SuperARC: a test for artificial superintelligence based on compressed modelling, recursive prediction and problem complexity

We introduce an increasing-complexity, open-ended, and human-agnostic metric to evaluate foundational and frontier AI models in the context of Artificial General Intelligence (AGI) and Artificial Super Intelligence (ASI) claims. Unlike other tests that rely on human-centric questions and expected answers, or on pattern-matching methods, the test here introduced is grounded on fundamental mathematical areas of randomness and optimal inference. We argue that human-agnostic metrics based on the universal principles established by Algorithmic Information Theory (AIT) formally framing the concepts of model abstraction and prediction offer a powerful metrological framework. When applied to frontiers models, the leading LLMs outperform most others in multiple tasks, but they do not always do so with their latest model versions, which often regress and appear far from any global maximum or target estimated using the principles of AIT defining a Universal Intelligence (UAI) point and trend in the benchmarking. Conversely, a hybrid neuro-symbolic approach to UAI based on the same principles is shown to outperform frontier specialised prediction models in a simplified but relevant example related to compression-based model abstraction and sequence prediction. Finally, we prove and conclude that predictive power through arbitrary formal theories is directly proportional to compression over the algorithmic space, not the statistical space, and so further AI models’ progress can only be achieved in combination with symbolic approaches that LLMs developers are adopting often without acknowledgement or realisation. Here the authors propose a human-agnostic, algorithmic complexitybased benchmark to assess frontier AI models and AGI or ASI claims. Results show leading LLMs are inconsistent and often divergent with newer versions sometimes less capable than older versions, while a hybrid neuro-symbolic approach outperforms specialised LLM predictors, underscoring the need for symbolic components.