Toward Robust Evaluations of Flood Inundation Predictions Using Remote Sensing Derived Benchmark Maps

Abstract Remote Sensing‐derived Flood Inundation Maps (RS‐FIM) are an attractive and commonly used source of evaluation benchmarks. In this paper, we investigate several sources of bias in RS‐FIM benchmarking and their effect on model‐predicted FIM (M‐FIM) evaluation results. We do so by comparing M‐FIM evaluation results using a high‐confidence benchmark against degraded benchmarks. The evaluation results show considerable differences in M‐FIM accuracy assessment when using lower‐quality benchmarks. An RS‐FIM enhancement (gap‐filling) procedure is presented, and its effect on FIM evaluation results is analyzed. The results show that the enhancement can significantly improve the robustness of the evaluation, but can also degrade the benchmark when a considerable number of false‐positive grid cells are present in the RS‐FIM. The impact of including/excluding Permanent Water Bodies (PWB) on FIM evaluation results is analyzed. The results show that including PWB in FIM evaluation can significantly inflate the model accuracy. A novel evaluation strategy is proposed, based on excluding low‐confidence grid cells and PWB from the M‐FIM evaluation analysis. Low‐confidence grid cells are those that were estimated to be flooded by the gap‐filling procedure, but were not classified as such by the remote sensing analysis. The results show that the proposed evaluation strategy can considerably improve the robustness of the evaluation. The analyses showcase the many challenges in FIM evaluation. We provide an in‐depth discussion about the need for standards, user‐centric evaluation, the use of secondary sources, and qualitative evaluation.