Answers are grounded in this paper and cite their sources.
Creating datasets for ML is an inherently human endeavor, as the data’s heterogeneity mandates human intervention. However, most data workflows being one-time and hardly transferable leads to a lack of standardization and reusability. There has been a push to impose more structure on the data work process, but little is known about the implicit or "tacit" knowledge of data workers, i.e., "know-how"s that is difficult to transfer to others. Identifying and formalizing this knowledge can help data work improve, leading it from current "exploration" to more systematic "engineering." We interviewed 19 ML practitioners in this study to find "why" they use "what" tacit knowledge. As a result, we identified the following themes: 1) data is context/situation dependent, 2) human workers are inseparable from data, and 3) models must be understood to build data. We finally discuss future systematic supports and research to convert what is implicit to explicit.
No takes yet. Share an insight, caveat, or question.
Cha et al. (2023) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: