Answers are grounded in this paper and cite their sources.
High-quality training data form the foundation of reliable intelligent systems in knowledge-intensive and safety-critical domains such as healthcare, intelligent transportation, smart manufacturing, cyber–physical systems, and scientific discovery. However, constructing such data still depends heavily on domain experts, whose nuanced judgment remains essential in tasks that require deep domain knowledge, contextual reasoning, and adherence to professional standards. To bridge the gap between expert insight and scalable data production, a growing body of research has developed the expert-in-the-loop (EITL) paradigm, which integrates human expertise with AI methods and collaborative platforms throughout the data life cycle. In this survey, we present a unified, systems-oriented analysis of EITL methodologies along two orthogonal dimensions: training-data types and expert–AI collaboration paradigms. We systematically categorize existing approaches according to expert roles and the augmentation mechanisms of AI systems, ranging from rule-based preliminary labeling and active sampling to generative synthesis and model-assisted evaluation and quality control. By abstracting common architectural patterns and interaction mechanisms, this survey establishes a structured taxonomy and conceptual framework for Expert-Guided data engineering. It further outlines open challenges and future directions for scalable, reliable, and human-centered data construction in contemporary AI development.
No takes yet. Share an insight, caveat, or question.
Song et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: