From Data to Dialogue: A Communication-Centered Perspective on AI-Based Multimodal Prediction of Neonatal Jaundice

Neonatal jaundice is a common issue in the newborn period, and timely, accurate monitoring and prediction are crucial to prevent severe complications (e.g. kernicterus). However, traditional diagnosis relies on invasive serum bilirubin tests or subjective judgment, which limits applicability in developing countries and home settings. In recent years, machine learning techniques have been introduced for non-invasive neonatal jaundice prediction: convolutional neural networks (CNNs) have been used for analyzing skin images of jaundice, and long short-term memory (LSTM) networks for time series prediction of bilirubin changes, achieving some success. Yet, CNNs and LSTMs have limitations such as local receptive fields and difficulty capturing long-term dependencies. This paper comprehensively reviews and reconstructs the machine learning framework for neonatal jaundice prediction by introducing the Transformer model as the core. We use a Vision Transformer (ViT) instead of CNN for processing skin images, and a time-series Transformer in place of LSTM for modeling dynamic data like transcutaneous bilirubin readings, and we fuse multimodal information to improve prediction accuracy. We highlight the potential of this Transformer-based framework in the health communication domain: for example, using attention heatmap visualization to improve model interpretability, and integrating with mobile health (mHealth) applications for remote monitoring and interaction, thereby increasing public awareness and trust in new technology. We detail the methodological architecture, demonstrate its promising application in early prediction of neonatal jaundice, discuss current challenges, and envision future research directions. By leveraging a Transformer-centric multimodal deep learning framework, neonatal jaundice prediction could achieve higher accuracy and interpretability, better serving clinical decision-making and family healthcare.