The metrological culture in the context of big data: managing data-driven decision confidence

In recent years, data sets are steadily growing in size because of the huge sensing, storage, computation, and transmission capabilities offered at very low cost by current Information Technology systems, which are increasingly driving ubiquitous data acquisition from the empirical world. Moreover, massive amounts of data can be collected very efficiently- sometimes at zero marginal costs from the internet. The concept of Big Data (BD) epitomizes this phenomenon among the experts and the general public (for an accessible, application-oriented introduction to BD in the broad context of data science).This paper aimed at showing that a sound knowledge of measurement fundamentals can be helpful in managing such a risk, since it makes people aware about possible uncertainty sources and their effect on the confidence of the conclusions they draw. Specifically, a simplified model that synthesizes the structure of a D3M process has been proposed by identifying three main process stages: information generation, information processing, and decision making. Applying approaches used in the context of metrology, the main contributions that may affect the confidence of the final decision have been identified and highlighted in a D3M model. The authors think that the obtained scheme can provide useful guidelines to estimate the confidence of the D3M outcomes and manage the risk of making wrong decisions.