HTMA-CL: A Hierarchical Tokenization and Multiscale Attention Framework for Compressive Domain Multimedia Inference

Compressed learning (CL), integrating compressed sensing (CS) and machine learning (ML), enables direct inference from few CS measurements. However, existing CL methods either heavily rely on pre-trained models built upon extremely large-scale datasets or perform only relatively simple tasks on small datasets, restricting their scalability in real-world multimedia scenarios. To address these limitations, we propose an efficient CL framework named HTMA-CL for practical multimedia acquisition and edge intelligent processing. HTMA-CL employs CNN-based learnable sampling to realize block-based CS for high-resolution images, significantly decreasing the transmission bandwidth and storage overhead. A hierarchical tokenization module together with a deep-narrow Transformer module progressively models local and global dependencies within the measurements, enabling accurate inference directly in the compressive domain. Various task heads are constructed for performing diverse multimedia analysis tasks such as image classification and semantic segmentation. Extensive experiments demonstrate that our HTMA-CL achieves state-of-the-art performance compared to other CL methods, and nearly comparable performance to the image domain methods at CS ratio of 10%. Our method is further verified the strong robustness against external interference in the disturbance-prone IoT multimedia environments. The source code is publicly available at https://github.com/acrlife/HTMA-CL.git .