Static Oracles Fail under Distribution Shift: The Primacy of Endogenous Horizon Generation in Reinforcement Learning

Traditional reinforcement learning (RL) typically frames time as an exogenous budget: a fixed horizon or discount factor that mechanically truncates agent trajectories. This paper explores an alternative, complementary perspective: time may be treated as an endogenous resource that agents can extend through targeted exploration. We introduce the Endogenous Horizon Markov Decision Process (EH-MDP), where the remaining temporal budget τₜ is included in the augmented state space, and its depletion rate is tied to the agent’s information gain. Within this framework, agents learn to adaptively allocate temporal resources: visiting novel states slows time consumption, effectively granting additional steps to reach sparse, delayed rewards. We first validate the core mechanism in the MiniGrid-Empty-8x8 environment using a strict causal identification protocol with six orthogonal ablations across 15 random seeds. Our full model achieves a mean success rate of 10. 4% (90% CI 3. 7%, 17. 8%), while all baseline models (Reward Only, Random Time, Time Only, Permuted Time Placebo) yield near-zero success rates (<2. 0%). We then evaluate the framework under extreme distribution shift: agents trained exclusively on the 8x8 maze are directly transferred to a 16x16 maze (shortest path ≈30 steps) while the initial physical time budget remains fixed at 12 steps. The Placebo model—which shares the same intrinsic reward and empirical time-consumption distribution as our full model, but lacks the state-dependent causal structure—collapses to a 2. 0% success rate, exhibiting what we term cognitive rigidity. In contrast, our full model adaptively generates a new temporal distribution, achieving a 12. 4% success rate (p = 0. 0305 vs. placebo) and, in some seeds, reaching the goal in 40 physical steps—3. 33 times the initial budget. Wasserstein distance analysis confirms a measurable shift between time-consumption distributions in the 8x8 and 16x16 environments (unconditional W-dist = 0. 0825, p < 0. 01). These results suggest that the ability to dynamically generate adaptive temporal distributions, rather than relying on a static distribution learned in training, may support more robust generalization under distribution shift, and that fixed oracles can be brittle in out-of-distribution settings.