DisasterSim: A Reproducible Benchmark for Navigation and Coverage in Collapsed Structures with Empirical Analysis of Classical Exploration and Goal-Conditioned Learning-Based Methods

Autonomous navigation of collapsed buildings is critical for disaster response, yet no standardized simulation benchmark exists for reproducible evaluation of robot navigation and coverage policies in such environments. We present DisasterSim, an open-source benchmark built on ROS 2 Humble and Gazebo Classic that provides a physically realistic post-earthquake building interior with configurable obstacle density, a multimodal sensor suite with Extended Kalman Filter (EKF)-based fusion, four formally defined evaluation metrics with automated computation, and four reference baseline policies. The entire system—environment, robot, SLAM, navigation stack, metrics, and automated experiment runner—executes from a single command with frozen parameters to ensure full reproducibility. Our empirical study across 39 trials reveals a striking result: three fundamentally different classical exploration paradigms—reactive FSM, frontier-based, and potential field—converge to a statistically indistinguishable performance plateau of approximately 30% area coverage (p > 0.79, |d| ≤ 0.27). This convergence suggests that navigation constraints, not exploration strategy, form the primary performance bottleneck in cluttered disaster environments. A partially trained goal-conditioned PPO policy (370k of 600k planned steps)—which navigates toward a fixed, known survivor location rather than exploring freely—achieves higher incidental coverage (36.9% mean, 61.1% peak, Cohen’s d = 0.78), indicating that goal-directed learned navigation traverses more of the environment en route than classical explorers manage in the same time budget. We additionally identify a quantifiable coverage–localization trade-off (Pearson r = 0.85, p < 0.001), correct a data error present in an earlier draft, and discuss the design of a goal-free RL explorer as the next step toward a fully autonomous learned baseline. All code, configurations, experiment logs, and trained models are publicly available.