What does this research mean for the field?

Well-calibrated AI model confidence does not inherently produce well-calibrated human reliance, but remapping displayed confidence to account for human judgment systematically reduces this gap and improves team decision accuracy. Novelty: ClaimNovelty.CONTRADICTORY. Consensus alignment: ConsensusAlignment.CHALLENGES_CONSENSUS.

What question did this study set out to answer?

Investigate why well-calibrated AI model confidence fails to yield calibrated human reliance, and develop an interface framework that aligns displayed confidence with human-AI team performance.

August 27, 2026ACM Transactions on Social ComputingOpen Access

From Calibrated Confidence to Calibrated Reliance: Human-Aware Confidence Communication for AI-Assisted Decision Making

View Full Paper

Ask AI

Bookmark

Authors

ZWZijia WangBeijing University of Chemical Technology KHKejia HuNorthwestern University

Discussion

Member takes

Implication

Benchmark analysis reveals a calibration-reliance gap in human–AI decision making, indicating that remapping displayed confidence improves team performance.

Key Points

Investigate why well-calibrated AI model confidence fails to yield calibrated human reliance, and develop an interface framework that aligns displayed confidence with human-AI team performance.
Analyzed 35,670 human–AI decision interactions from the HAIID benchmark alongside validation data from the GRACE benchmark.
Formulated the Reliance Calibration Error (RCE) metric to quantify the divergence between displayed model confidence and realized team accuracy.
Designed and tested bounded and subgroup-aware confidence-remapping strategies to modify displayed probabilities without retraining the underlying predictor.
Demonstrated a systematic calibration–reliance gap where near-calibrated models still trigger miscalibrated human reliance due to human judgment filters.
Subgroup-aware remapping substantially reduced display–outcome misalignment on held-out HAIID data, while bounded and global policies improved team mean squared error (MSE) more reliably than aggressive remapping.
Bounded remapping variants avoided extreme boundary display concentrations (0/1 displays) while preserving simulated decision-quality gains.

Cite This Study

Wang et al. (2026) studied this question.

synapsesocial.com/papers/6a8fea1c10c91c1e9262234e https://doi.org/10.1145/3843742

View Full Paper

Ask AI

Bookmark

Implication

Benchmark analysis reveals a calibration-reliance gap in human–AI decision making, indicating that remapping displayed confidence improves team performance.

From Calibrated Confidence to Calibrated Reliance: Human-Aware Confidence Communication for AI-Assisted Decision Making

Discussion

Member takes

Implication

Key Points

Cite This Study

From Calibrated Confidence to Calibrated Reliance: Human-Aware Confidence Communication for AI-Assisted Decision Making

Discussion

Member takes

Implication

Key Points

Cite This Study

Also Consider

Also Consider