Bridging time series and large language models via symbolic representation for human activity recognition

Time-series classification with Large Language Models (LLMs) remains challenging because raw continuous signals lack the discrete sequential structure that language models are designed to process. We observe that Symbolic Aggregate approXimation (SAX) naturally produces ordered token sequences structurally compatible with LLM inputs, making the symbolic-LLM connection principled rather than ad hoc. Building on this observation, we propose SAX_HAR-LLM, a dual-stream framework that encodes inertial sensor data as symbolic sequences augmented with lightweight kinematics-informed descriptors, processed jointly by a fine-tuned causal language model for human activity recognition. The framework is evaluated on two benchmark inertial HAR datasets — WISDM and MotionSense — under a subject-independent protocol against symbolic, deep learning, and kernel-based baselines. Ablation experiments provide direct empirical evidence for the central theoretical claim: the fine-tuned LLM benefits from pre-trained sequential priors when processing symbolic time-series representations. Dynamic activities are discriminated from symbolic sequences alone, while kinematics-informed descriptors function as a targeted auxiliary mechanism for classes becomimg symbolically indistinguishable under z-normalization. SAX_HAR-LLM outperforms all symbolic baselines and most deep learning architectures, remaining competitive with state-of-the-art kernel-based approaches. Unlike competing methods that sacrifice transparency for accuracy, SAX_HAR-LLM’s predictions are fully auditable: an attention-based interpretability analysis reveals that the model’s predictive focus aligns with physically meaningful sensor axes and class-specific symbolic motifs, enabling decisions to be traced and validated in domain terms. These findings establish symbolic time-series representations as an interpretable and effective interface to LLMs — one where competitive performance and decision transparency are achieved simultaneously, at a modest but measurable cost in accuracy and inference overhead relative to purely numerical approaches.