Silent Modality Collapse and the Measurement of Fusion Gains: A Nationwide Out-of-Time Test of Earth Observation for Agricultural Default Forecasting

Satellite Earth observation (EO) is widely proposed as alternative data for agricultural credit risk, yet reported fusion gains are rarely tested against the protocol producing them. We specify three inexpensive instruments—matched alert volumes instead of a fixed threshold, neutralization of a modality inside a fitted model, and diagnostics of whether a fused branch is used—and apply them to ST-CRAAN, a dual-branch forecaster combining an eleven-channel monthly EO tensor with borrower financial covariates, on 1,496,721 Thai rice-cultivating borrowers with the 2022 crop year withheld. The diagnostics detect silent modality collapse: weight decay drove the recurrent input kernel from 0.0750 at initialization to 7.0 × 10−8 (about 10−6 of its initial value), and replacing the EO tensor by its population mean changed test AUC by 0.00000. An ablation traces the collapse to gradient starvation of the branch, which weight decay completes rather than causes. With the branch in use, a factorial of five model families by eight feature sets, refitted under several seeds, finds that EO lowers test AUC in every fit of the three tabular families when it joins financial and spatial covariates, and gives the two neural families no reliable gain. Added to sub-district repayment history alone, it lowers test AUC in every fit of all five families. Ranking borrowers by the realized default rate of their sub-district–year cell bounds any predictor constant within such a cell at AUC 0.67721; every family stays below it, by 0.029 to 0.031. On this panel and across this year boundary, areal EO does not improve borrower-level default forecasting for a lender holding its own records: the binding constraint is resolution, not architecture.