Diagnostic-Gap-Driven Class-Conditional Modality Regulation for Multimodal Building Defect Segmentation

RGB–IR multimodal inspection can combine visible surface information with thermal-response patterns for building-defect assessment, but joint learning may develop class-specific information unevenly across modality branches. This study proposes adaptive class-conditional modality regulation for RGB–IR façade-defect semantic segmentation. Matched unimodal models establish an empirical diagnostic reference for each modality–defect pair. During multimodal training, periodic diagnostic branch scores are compared with these references, and the resulting diagnostic gaps redistribute auxiliary supervision while leaving the fused inference pathway unchanged. On the held-out test subset of the Building Façade Defect Detection dataset, plain dual-branch fusion provided only a modest improvement over RGB alone, while calibration-set diagnostics showed larger gaps in the IR branch. Across five matched test runs, the proposed method increased mIoU from 0.6097±0.0034 to 0.6568±0.0069 and Boundary F1 from 0.5841±0.0042 to 0.6372±0.0091, exceeding early fusion and the selected global balancing methods OGM-GE and AGM. Ablation results associate the largest component-wise difference with class-specific allocation. The method adds training-time computation but does not change the deployed network or its forward computation.