Evaluating Abstract Reasoning and Problem-Solving Abilities of Large Language Models Using Raven's Progressive Matrices

Abstract Artificial intelligence has rapidly evolved, leading to the development of powerful models capable of performing complex cognitive tasks. Evaluating the cognitive abilities of these models through established human intelligence tests such as Raven's Progressive Matrices (RPM) offers a novel and significant approach to understanding their abstract reasoning capabilities. The study adapted RPM for text-based interactions, enabling the evaluation of Mistral and Llama without human intervention. Results revealed that both models surpass average human performance in overall accuracy, demonstrating their advanced problem-solving skills. However, the analysis also highlighted variability in performance across different types of reasoning tasks, with Llama excelling in sequential pattern recognition and Mistral showing weaknesses in spatial awareness. These findings provide valuable insights into the strengths and limitations of Mistral and Llama, offering a comprehensive understanding of their cognitive abilities and guiding future advancements in artificial intelligence.