Assessing RLVR’s Efficacy in Solving Previously Intractable Problems with LLMs

Reinforcement Learning from Verifiable Rewards (RLVR) has emerged as a promising method for enhancing the reasoning capabilities of large language models (LLMs). There is an ongoing debate, however, as to whether RLVR genuinely expands reasoning capacities in LLMs and enables the emergence of new strategies in a manner similar to that of traditional RL agents. Seeking to objectively evaluate behavioral emergence in RLVR, this thesis assesses RLVR’s efficacy in solving previously intractable problems. By analyzing various relevant and contemporary implementations, this work shows that problems that cannot be solved by a base model often become solvable through RLVR training. Even though many of these problems are highly constructed, this shift represents evidence of a genuine expansion of reasoning capabilities through reinforcement learning. A deeper understanding of the limitations of current methods will enable further advances in research and the development of LLMs with greater success in complex reasoning tasks.