Abstract Machine Learning applications are using a wide variety of data sources, some real and some artificial, such as data generated through numerical simulations. The latter mode has been gaining rapid popularity for engineering design and analysis. Early work in this arena centered on the data being generated by developers of ML applications themselves. We are seeing the beginnings of publicly shared data sets. Thus, the quality and efficacy of such data need to be considered before their use. In this paper, we attempt to outline systematic principles and quantifiable quality and efficacy metrics based on insights gained collectively from both data curation projects and usage of large engineering data sets. Specifically, this paper addresses issues related to generating BIG datasets from CAD and FEA: Granularity and Modality, applicable to both input and output data; Variety and Balance, applicable to input data; also, Efficacy for ML. Generation of Big Data by simulation requires the use of commercial CAD and FE packages, which poses multiple challenges: automation, integration and balancing sample variants. This paper proposes metrics for evaluating data efficacy: size, quality, validity and balance. We propose Parametric Variety and Balance Matrices to study over or under representation of input attributes. Even if we have a large data set with good balance, it may not be suitable for ML if we do not see significant variation in response or performance variables. Several data generation, curation and utilization case studies are included in a variety of domains and usage of metrics is demonstrated.
Shah et al. (Mon,) studied this question.