Generative Video-LiDAR world models for long-horizon multimodal scene simulation with temporal and geometric consistency

The training and evaluation of autonomous-driving perception systems rely on multimodal scene data. However, limited real-world long-tail scenarios have made generative world models an important complementary data source. Existing methods mainly focus on unimodal video generation or implicit training supervision, making test-time Camera–LiDAR conflict resolution and explicit consistency assessment difficult. This paper proposes a Video–LiDAR joint generation framework for extended-horizon scene synthesis that introduces consistency constraints into sampling and state space through three mechanisms. First, projection, depth–range consistency, and ray-visibility gradient corrections are incorporated into diffusion sampling to maintain cross-modal geometry. Second, a recoverability-based formulation compares predicted and re-encoded states to constrain temporal and identity consistency over rollout horizons up to 10 s. Third, failure-feedback-driven counterfactual interventions are designed to cover selected long-tail conditions. Experiments on Waymo and nuScenes show that, compared with the MagicDrive-V2-adapted baseline, the proposed method reduces Waymo Proj.Err. by 16.6% (Holm-adjusted p = 0.030) and Drift by 15.5%, while improving IDF1 by 1.6 points. Improvements on nuScenes are smaller and less consistent. With 30% LiDAR retention, the Long-tail mAP decline is reduced from 18.1% to 14.0%. On held-out real long-tail scenes, counterfactual augmentation improves Long-tail mAP from 34.1 to 35.5, compared with 34.6 using ordinary generated data, showing potential utility for challenging long-tail evaluation. Ablation studies reveal distinct effects of the three modules on geometry, identity, and long-tail performance. This work integrates training-stage modeling and sampling-stage correction for recoverability-based multimodal geometric and temporal consistency.