COMFORT: A Continual Fine-Tuning Framework for Foundation Models Targeted at Consumer Healthcare

Wearable medical sensors (WMSs) are revolutionizing smart healthcare by enabling continuous, real-time monitoring of user physiological signals, especially in the field of consumer healthcare. The integration of WMSs and modern machine learning enables unprecedented solutions to efficient early-stage disease detection. Despite the success of Transformers in various fields, their application to sensitive domains, such as smart healthcare, remains underexplored due to limited data accessibility and privacy concerns. A key challenge in disease detection using Transformers with WMS data is the difficulty of obtaining sufficient labeled training data from individuals suffering from a specific disease, as this requires labeling by experts and extensive time and effort devoted to data collection and data privacy regulation compliance. The phrase “foundation models” conjures up an image of a large language model trained on text corpora. However, foundation models need not be large, nor necessarily trained on text. In this work, we propose a foundation model trained on WMS data under a continual fine-tuning framework called COMFORT. COMFORT introduces a novel approach for pre-training a Transformer-based foundation model on a large dataset of physiological signals exclusively collected from healthy individuals with commercially available WMSs. We adopt a masked data modeling (MDM) objective to pre-train this health foundation model. MDM is inspired by the mask language modeling approach employed in self-supervised training of language models. Our WMS-based foundation model can then be rapidly adapted to multiple disease detection tasks through various parameter-efficient fine-tuning (PEFT) methods, such as Low-Rank Adaptation and its variants. COMFORT continually stores the low-rank decomposition matrices and classifiers obtained using the PEFT algorithms to construct a library for multi-disease detection. This library enables scalable and memory-efficient disease detection on edge devices. Our experimental results demonstrate that COMFORT achieves highly competitive performance while reducing memory overhead by up to 52% relative to conventional methods. Thus, COMFORT paves the way for personalized and proactive solutions to efficient and effective early-stage disease detection.