Answers are grounded in this paper and cite their sources.
Active learning under hidden distribution shift requires selecting samples without access to the deployment distribution. We formalize this setting and introduce Composite Active Data Acquisition for Out-of-Distribution Generalization (CADA-OOD). Its central mechanism, validation error relevance, links unlabeled candidates to current model failures on an accessible validation set. Additional terms capture predictive uncertainty and group novelty while controlling batch redundancy and outlier selection. We report a retrospective evaluation on seven molecular and materials property prediction benchmarks that also informed method development. The benchmarks involve unseen molecular scaffolds and material composition groups. Across three acquisition budgets, CADA-OOD achieved higher mean out-of-distribution test scores than conventional acquisition strategies in most settings and stayed within 5% of the best method in all 21 benchmark and budget combinations. An approximation to Batch Active Learning by Diverse Gradient Embeddings (BADGE) showed comparable aggregate performance. Gains over conventional baselines were also observed with alternative molecular representations, with magnitudes varying across tasks. An external component comparison evaluated full CADA-OOD against a variant that removed only validation error relevance. Across three regression tasks under matched total label budgets, retaining this component yielded a mean relative reduction of 3.37% in root mean squared error (95% confidence interval 1.30−6.14%). Additional comparisons at equal total labeling cost, including a neural predictor with loss-based and gradient-based acquisition, showed task-dependent effects. These findings support validation error relevance as an interpretable complement to uncertainty and diversity for active acquisition under hidden distribution shift.
No takes yet. Share an insight, caveat, or question.
Zhanghua et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: