This paper presents Dynamic Reconnaissance-Flattening (DRF), a four-layer acceleration pipeline for NP-hard optimisation. The pipeline consists of: (1) Perspective — multi-resolution observation of the loss landscape using a hierarchy of collapse chains; (2) Flattening — active reshaping of the landscape to reduce ruggedness before search; (3) Diagnosis — real-time classification of residual landscape topology; and (4) Tool Adaptation — dynamic selection and configuration of solvers based on the diagnosed topology. Validated across all seven NP-hard problem boxes: MKP +48% (four-layer stack vs blind search) and +6.7% vs strong LP-guided baseline; Graph Colouring −56% conflict reduction; VRP +74%; TSP +4.1% at 200 cities; QAP +6.1%; Sequential Decision 3.56× speedup; SAT correctly identified as hard wall (abstain). Three subsidiary results: adaptive chain configuration (principal–adjutant structure, +7% over uniform allocation); multi-resolution perspective chains (rescuing Graph Colouring from degradation); MCCO+CMA tool adaptation (+98% on continuous benchmarks). Part of the NSD / MOEH research programme. Companion paper to "Collapse as a Unifying Language for NP-Hard Optimisation" (Zenodo: 10.5281/zenodo.19574013). v2 update: Added Section 1.4 "Prerequisite: The Landscape Classification Phase" — explicitly establishes the upstream-downstream relationship between the NP Landscape Classification paper (Phase 1: independent reconnaissance → terrain reshaping → algorithm selection) and the DRF pipeline (Phase 2: embedded reconnaissance → perspective → flattening → diagnosis → tool adaptation). The two papers form a two-phase scout operation: Phase 1 scouts operate independently to classify and reshape the landscape; Phase 2 scouts accompany the main force for real-time tactical support. Scouts are the brain of the operation; the solver provides muscle. Approximately 1% of computational resources produces the intelligence that directs the other 99%. v3 update: Added Section 9 "Two-Phase Validation: NP Landscape + DRF" with complete experimental results. Three-way comparison (Blind vs DRF Only vs Full Two-Phase) on six MKP configurations, 30 independent runs each, Wilcoxon signed-rank tests. Key finding: Phase 1's iterative reconnaissance-transformation loop adds no significant value at small scale (n=100, p>0.2) but becomes highly significant at large scale (n≥250, p<0.001; n=500, p<10⁻⁹). Added scalability validation from n=100 to n=5,000: improvement curve is monotonically increasing (+44% at n=100, +56% at n=500, +61% at n=5,000). At n=5,000, DRF is both better (+61%) and faster (4.5x wall-clock speedup), with reconnaissance overhead below 0.5%. All results: p < 10⁻⁹, 30/30 wins. v4 update: Added cross-box validation on VRP (Box 4) with full two-phase pipeline (30 independent runs, n=50/100/200/500, Wilcoxon signed-rank tests). VRP confirms near-identical scalability curve as MKP: +44% at n=50, +51% at n=100, +54% at n=200, +56% at n=500. Unlike MKP, Phase 1's reconnaissance-transformation loop is statistically significant at ALL scales including n=50 (p=1.3×10⁻⁶), because VRP's spatial landscape is more rugged than MKP's combinatorial landscape. Cross-box comparison table shows MKP and VRP produce nearly identical improvement percentages at matching scales (+44%/+54%/+56%), suggesting the improvement range reflects a structural property of the reconnaissance-transformation architecture rather than a box-specific artefact. Two boxes, two fundamentally different landscape types, same scalability story. v5 update: Added cross-box validation on Graph Colouring (Box 2) with full two-phase pipeline (30 independent runs, n=50/100/200/500, Wilcoxon signed-rank tests). Graph Colouring achieves -65% conflict reduction at n=50, stabilising at -57% for n≥100. Phase 1 (Gumbel-Softmax temperature annealing with difficulty score monitoring) is significant at all scales including n=50 (p=0.012). Three-Box Cross-Validation Summary table now covers MKP (Box 1, combinatorial/LP-transformable), VRP (Box 4, spatial/geometry-transformable), and Graph Colouring (Box 2, constraint-dominated/Gumbel-Softmax-transformable). Three fundamentally different landscape types, three different Phase 1 transformation methods, same scalability story. Near-identical improvement magnitudes across boxes suggest the improvement range reflects a structural property of the reconnaissance-transformation architecture itself. Updated abstract, scout cost analysis, and conclusion to reflect three-box validation. v6 update: This version introduces the Scout-Modeler Feedback Loop: Learning to Build the Mountain (Section 7), which closes the previously open expedition cycle. Prior versions had the expedition direction correct—modeler builds landscape, scouts survey, solver climbs—but lacked the return arrow from expedition to modeler. v6 adds this arrow and specifies what flows through it: Three roles explicitly separated: Modeler (author of the landscape), Scout (landscape observer), Solver (landscape climber). Four categories of expedition signal that return to the modeler: predicted-vs-actual divergence, scout-vs-actual divergence, basin structure mismatches, and transformation effectiveness. All four are free by-products of expeditions that were going to happen anyway. Modeler update rule: inverse-variance weighted least squares over the expedition log, operating at a slower timescale (weeks-to-months) than scouts (per-expedition) or solver (daily). The separation of timescales is essential: fast modeler updates chase noise; slow updates capture durable regime structure. Connection to MCCO's outer level (MCCO v3, §8.3): MCCO introduced "learning how to build the mountain, not just how to find its peak" as a Box-6-specific mechanism. v6 elevates this to a pipeline-level construct applicable across all seven Boxes. Empirical evidence: The NSD production system's 27-dimensional shadow logger~zenodo.19456455 is the first live implementation. The first formal modeler-calibration event is pre-registered in the strict sense: calibration date, update rule, and evaluation criterion were fixed before the 30-day accumulation window began. From two-phase to closed-loop three-phase: Phase 1 independent reconnaissance → Phase 2 embedded reconnaissance → Phase 3 expedition debrief. Phase 3's value does not appear in any single expedition's metrics; it compounds across expeditions as the modeler's landscapes become progressively better calibrated. Philosophical claim. Expedition-to-modeler feedback is not a tuning trick but a structural necessity. An expedition system that does not close the loop cannot distinguish a well-calibrated modeler from a lucky climb. Only accumulated residuals discriminate between these cases. Closing the loop is the mechanism by which the pipeline earns its claims of generality. === Version 7 (Standalone) — April 2026 === Major restructuring to make DRF a fully self-contained paper, independent of any prerequisite framework: - Pipeline reframed as five stages (Reconnaissance, Flattening, Perspective, Diagnosis, Tool Adaptation), with an optional sixth stage for the scout-modeler feedback loop. Previous "four-layer" framing absorbed Reconnaissance and Flattening as explicit stages of the DRF pipeline itself. - Algorithm-agnostic positioning made explicit: DRF assumes the user has already chosen a solver family, and accelerates that solver. The pipeline does not perform problem classification or algorithm-class selection. - Section 10 reframed as a Scalability Ablation experiment (Full DRF vs Streamlined DRF), replacing the previous Two-Phase Validation framing. All experimental data preserved unchanged. - Problem classes referred to by canonical names (MKP, TSP, Graph Colouring, SAT, VRP, QAP, Sequential Decision) throughout, replacing the box-numbered taxonomy of prior versions. - Section 7.7 reframed from "Two-Phase to Three-Phase" to "Open-Loop to Closed-Loop Pipeline," consistent with the standalone five-stage presentation. - All experimental results (VRP +74%, Graph Colouring +56%, MKP +48%, etc.) preserved from v6.
H Y Rao (Wed,) studied this question.