What question did this study set out to answer?

The aim is to enhance long-context reasoning performance in language models using the RST framework.

June 29, 2026Open Access

Recursive–Structured-State–Termination (RST) - Improving the performance of RLM for long-context reasoning with the help of structured graph state

Key Points

The aim is to enhance long-context reasoning performance in language models using the RST framework.
Implemented Recursive-Structured-State-Termination (RST) to manage long document chunks.
Evaluated RST against OOLONG, OOLONG-Pairs, and LongBench-v2 CodeQA datasets.
Utilized a structured reasoning graph to maintain evidence and guide reasoning.
RST achieved a mean gain of 44.2% over RLM with Qwen3-8B on OOLONG.
In OOLONG-Pairs, RST reached 76.6% F1 on the 32K setting using Qwen3-235B.
RST operated 2–3× faster than RLM while also transferring effectively to LongBench-v2 CodeQA.

Abstract

Transformer-based language models achieve strong performance across natural language processing tasks, but their effectiveness degrades in very long-context settings due to context rot and the quadratic computational cost of self-attention. Although recursive language models mitigate this limitation by processing inputs in chunks, they introduce a control challenge in determining when to continue reasoning, consolidate evidence, or terminate. We propose the Recursive-Structured-State-Termination (RST) Reasoning Engine, an adaptive recursive framework for long-context understanding. RST partitions long documents into chunks, incrementally extracts knowledge, and maintains a structured reasoning graph composed of Claims, Partial Answers, and Open Questions. This graph preserves evidence across iterations, tracks dependencies, identifies contradictions, and guides further exploration through explicit information gaps. A convergence-based termination mechanism determines when sufficient evidence has been accumulated to generate a reliable response, thereby improving reasoning transparency while reducing unnecessary computation. We evaluate RST on OOLONG, OOLONG-Pairs, and LongBench-v2 CodeQA. RST consistently improves over RLM across model scales and domains. On OOLONG, RST achieves mean gains of 44.2% over RLM with Qwen3-8B and 33.9% over the base Qwen3-235B-A22B model. On OOLONG-Pairs, RST outperforms both base models and RLM without helper functions, reaching 14.2% and 41.9% F1 on the 32K setting with Qwen3-8B and Qwen3-235B, respectively, and 12.1% F1 on the challenging 131K setting. With helper functions, RST further improves to 76.6% F1 on 32K and 72.1% F1 on 131K using Qwen3-235B, while running approximately 2–3× faster than RLM. RST also transfers effectively to LongBench-v2 CodeQA, yielding large gains across code-capable models. Finally, integrating Mamba2-1.3B for long-context extraction and compaction further improves effectiveness and efficiency, highlighting the complementarity of structured state-space modeling and adaptive recursive reasoning for long-context tasks.

Read Full Paperexternally

Demander à l'IA

Bookmark

View Full Paper