Fine-Grained Toddler Activity Recognition through Domain Adaptation of a Pretrained Hierarchical Vision Transformer

Under the broad spectrum of vision-based Human Activity Recognition (HAR), infant and toddler activities are underrepresented. Although some toddler activities resemble those performed by adults, their daily routines, play behaviors, and other developmental activities are specific to their age group. Furthermore, their activities differ from those of adults due to developmental characteristics, including body proportions, gait, postural stability, and cognitive and emotional development. These differences make fine-grained toddler activity recognition a distinct and challenging research problem. To address the limited availability of fine-grained toddler activity datasets, this study introduces a new dataset, TodAct16, comprising 16 distinct toddler activity classes spanning daily routines, developmental behaviors, and safety-critical activities. Motivated by the need for accurate, scalable, non-invasive, and computationally efficient models for toddler activity monitoring, this study investigates the transferability of hierarchical spatiotemporal representations learned from large-scale adult activity videos to fine-grained toddler activity recognition. Therefore, a Hiera-Base hierarchical vision transformer initialized with Kinetics-400-pretrained weights is adapted by fine-tuning and evaluated using five-fold cross-validation. The fine-tuned Hiera-Base model achieves an accuracy of 97.02% ± 0.70%, a macro-F1 of 97.06% ± 0.71%, and an AUC of 99.85% ± 0.10%, outperforming the VideoMAE-Base video transformer on all three metrics. In addition to classification performance, this study evaluates the computational complexity, model size, inference latency, throughput, and memory requirements of the high performing Hiera-Base and its Stage-3-pruned variant to investigate their potential suitability for edge deployment. Overall, the experimental findings demonstrate the potential of transferring pretrained hierarchical spatiotemporal representations from general human activity videos to the toddler activity recognition domain.