Calibration Inversion and Data-Freshness Govern AI Reliability in Indeterminate Domains: Evidence from 2,730 Data Points across 142 Cross-Protocol Evaluation Sessions

Key Points

• The research investigates the reliability of AI models in indeterminate domains, focusing on calibration inversion and data freshness failures.
• Analyzed 2,730 data points from 142 evaluation sessions across six AI models and three protocols.
• Assessed performance in financial markets, meteorology, sports, and cryptocurrency.
• Computed aggregate scores and robustness using cross-protocol evaluation metrics.
• Gemini achieved a point-estimate accuracy of 5.3% mean error but a confidence interval calibration of only 29.7%.
• DeepSeek's S&P 500 predictions exhibited 12–13% stale error due to information access issues.
• Spearman rank correlation of ρ = 0.619 indicated moderate ranking consistency across protocols.