Machine Learning Models for Flood-Risk Index Classification: A Comparative Evaluation with Multi-Scale Spatial Replication Using the FEMA National Risk Index

Flood-risk index classification asks whether a classifier can distinguish high- from low-risk locations from composite risk indicators, and whether that ability survives contact with real geography. Six classifiers—Logistic Regression, Decision Tree, Random Forest, Gradient Boosting, LightGBM, and CatBoost—were benchmarked on a 1,117,957-record synthetic dataset (Resurrectum Diluvium, Kaggle); Logistic Regression achieved the highest AUC-ROC (0.93) by a small but significant margin, hypothetically attributable to the additive construction of the synthetic indices. The same pipeline was applied to the Federal Emergency Management Agency (FEMA) National Risk Index at county resolution (n = 3140), using a loss-per-exposure target after raw dollar losses proved population-correlated (r = 0.63). The ranking inverted: Random Forest reached AUC = 0.79 against 0.67 for Logistic Regression, and leave-one-region-out cross-validation collapsed every model toward chance (mean AUC 0.52–0.55). A census-tract replication within the same NRI framework (n = 84,034) reproduced the pattern: Random Forest again led under random splitting (AUC = 0.85), tree ensembles lost the most under spatial cross-validation, and Logistic Regression, weakest under random splitting, generalized best across untrained regions (mean AUC = 0.64). Across three stages and 1.2 million observations, random-split validation overestimated spatial transferability, which becomes visible only when geography is excluded from at least one evaluation fold.