Contextual Health State Inference from Lifelog Data Using LLM

This study investigates the application of Large Language Models (LLMs) for health state classification using multimodal sensor data from mobile and wearable devices. We address the limitations of existing approaches by integrating diverse sensor data with contextual information, aiming to improve classification accuracy in real-world, data-scarce scenarios. Our novel methodology involves creating conversation-like datasets that contextualize sensor data, which are then used to fine-tune an LLM (Meta-Llama-3-8B). We compare this approach with traditional machine learning methods (XGBoost and Random Forest) and deep neural networks (DNN and Conv+GRU model) across various data configurations. Results demonstrate that our LLM-based approach, augmented with conversational data, achieves a 34.18% performance increase over the benchmark features configuration, outperforming other models with a score of 5.983 out of 10. This study highlights the potential of LLMs in revolutionizing health state classification by effectively leveraging both quantitative sensor data and qualitative contextual information. Our findings open new avenues for developing more accurate and personalized health monitoring systems, with significant implications for real-world applications in healthcare and wellness.