Human-AI Collaboration in Cyber-Physical Systems: Foundations and Applications

Decision-making in cyber-physical systems requires both computational models and human knowledge of the physical environment. In logistics, workers and planners possess contextual knowledge that is difficult to capture in operational data, while AI models offer scalable prediction and candidate generation. Combining these capabilities is challenging because human judgments vary in reliability, ground truth can be costly or unavailable, and model outputs are uncertain. This dissertation develops calibration methods for human-AI collaboration, connecting statistical foundations with deployment in large-scale logistics systems. First, we introduce RAICA, a framework for assessing delivery-area difficulty using workers' comparative judgments and model-generated scores. RAICA aggregates individual rankings into a consensus and uses isotonic regression to calibrate model assessments, supporting more equitable compensation and improved delivery performance. We then generalize this approach through Aggregate-then-Calibrate (AtC), a framework for human-centered assessment that explicitly accounts for differences in annotator reliability. The theoretical analysis establishes the efficiency benefits of modeling annotator heterogeneity, bounds calibration risk under ranking misspecification, and identifies conditions under which AtC asymptotically outperforms model-only assessment. Next, we study human-AI collaboration in geospatial repartition through CoAlign. This framework combines large language model generation with historical evidence to support the redesign of delivery zones. It generates explainable candidate partitions, applies conformal inference to characterize uncertainty in their rankings with coverage guarantees, and presents candidates through an interactive decision support system for human review. Experiments on semi-synthetic and real-world datasets demonstrate improvements in assessment accuracy, robustness, and alignment with human feedback. RAICA and CoAlign have both been deployed at one of the world's largest e-commerce logistics companies. RAICA's deployment across more than 1,200 delivery stations reduced the Gini coefficient of crowdsourcing couriers' earnings by 0.2, while CoAlign improved on-time delivery rates by 3\%. Together, these contributions establish calibration as a practical and theoretically grounded approach to integrating human knowledge with AI capabilities for assessment and decision support.