Cross-Model Behavioral Measurement via Thermodynamic Peclet Number: Breaking Scoring Circularity with Public Benchmark Data

Key Points

• To explore whether the thermodynamic Peclet number can predict cognitive performance across various models using public benchmarks.
• Mapped 27 large language models to a three-dimensional behavioral space using public benchmark scores.
• Tested the predictive ability of the Peclet number on cognitive performance while controlling for specific benchmarks.
• Conducted paired analysis of model pairs to assess alignment and its effect on Peclet number.
• Partial correlations indicate that Peclet number significantly predicts performance on MMLU, HellaSwag, and ARC-Challenge.
• Alignment of models consistently increases Peclet number, with results showing perfect sign consistency.
• Statistically significant findings with p-values less than 0.02 demonstrate robust correlations.

Abstract