Estimating LLM Consistency: A User Baseline vs Surrogate Metrics

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

1. 1What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineering2024 · 5 citations
2. 2Large Language Models are Inconsistent and Biased Evaluators2024 · 13 citations
3. 3Model Consistency as a Cheap yet Predictive Proxy for LLM Elo Scores2025
4. 4Factual Confidence of LLMs: on Reliability and Robustness of Current Estimators2024
5. 5SCORE: Systematic COnsistency and Robustness Evaluation for Large Language Models2025