Gestational diabetes mellitus prediction using image-encoded electronic medical records and Transformer-based fusion

Accurate prediction of gestational diabetes mellitus (GDM) is critical for improving maternal and fetal outcomes. This study develops a Transformer-based multimodal fusion model that integrates tabular clinical features and image-encoded electronic health records (EHRs), aiming for accurate end-to-end classification of GDM. Preprocessed EHRs were transformed into grayscale, RGB, and heatmap, with visual features were extracted by a Vision Transformer and tabular features by an MLP. A modality-aware attention mechanism enhances cross-modal fusion. Evaluated on two public datasets, performance gains over the strongest single-modality models reached 3.95% and 0.38% in accuracy.