What question did this study set out to answer?

This research aims to enhance the effectiveness of language models in long-context settings through the RST framework.

July 12, 2026Open Access

Recursive–Structured-State–Termination (RST) - Improving the performance of RLM for long-context reasoning with the help of structured graph state

Key Points

This research aims to enhance the effectiveness of language models in long-context settings through the RST framework.
Proposed Recursive-Structured-State-Termination (RST) Reasoning Engine for adaptive processing.
Evaluated RST against multiple benchmarks including OOLONG and MuSiQue.
Compared performance metrics like F1 scores across various model scales and configurations.
RST achieves 44.2% mean gains over RLM on OOLONG.
On OOLONG-Pairs, RST reaches 76.6% F1 with helper functions on the 32K setting.
RST consistently shows best performance on MuSiQue benchmarks, with F1 scores up to 40.8%.

Abstract

Transformer-based language models achieve strong performance across NLP tasks, but their effectiveness degrades in very long-context settings due to context rot. Although recursive language models mitigate this limitation by processing inputs in chunks, they introduce a control challenge in determining when to continue reasoning, consolidate evidence, or terminate. WeproposetheRecursive-Structured-State-Termination (RST) Reasoning Engine, an adaptive recursive framework that incrementally extracts knowledge, maintains a structured reasoning graph of intermediate states, and employs convergence-based termination to improve reasoning efficiency, transparency, and scalability. We evaluate RST on OOLONG, OOLONG-Pairs, LongBench-v2 CodeQA and MuSiQue. RST consistently improves over RLM across model scales and domains. On OOLONG, RST achieves mean gains of 44. 2% over RLM with Qwen3-8B and 33. 9% over the base Qwen3-235B-A22B model. On OOLONG-Pairs, RST outperforms both base models and RLM without helper functions, reaching 14. 2% and 41. 9% F1 on the 32K setting with Qwen3-8B and Qwen3-235B, respectively, and 12. 1% F1 on the challenging 131K setting. With helper functions, RST further improves to 76. 6% F1 on 32K and 72. 1% F1 on 131K using Qwen3-235B, while running approximately 2–3×faster than RLM. RST also transfers effectively to LongBench-v2 CodeQA, yielding large gains across code-capable models. Across the MuSiQue benchmarks, the RST framework consistently achieves the best performance, attaining F1 scores of 40. 8% on MuSiQueₐns and 19. 3% on MuSiQue₁28K, respectively. Finally, integrating Mamba2-1. 3B for long-context extraction and compaction further improves effectiveness and efficiency, highlighting the complementarity of structured state-space modeling and adaptive recursive reasoning for long-context tasks.

Ask AI

Helpful

Bookmark

View Full Paper