A Reinforcement Learning Calibration Benchmark of Tabular Q-Learning and One-Hot DQN on Enumerable Maze Navigation

This paper presents a reinforcement learning calibration benchmark comparing tabular Q-Learning and one-hot Deep Q-Networks (DQN) on enumerable maze navigation tasks. Across deterministic grid-world environments with matched interaction budgets and multi-seed evaluation, tabular Q-Learning achieves equivalent final policy quality with superior sample efficiency and lower wall-clock cost. Additional ablations isolate the contribution of depth, replay buffers, target networks, and state representation. Results suggest that, in small enumerable MDPs, optimization overhead from deep RL infrastructure can dominate any benefit from function approximation. The repository includes a fully reproducible single-file NumPy implementation, statistical evaluation utilities, and benchmark scripts.