MCANet: A lightweight action recognition network with multidimensional convolution and attention

Abstract The majority of lightweight networks currently employed in action recognition tasks are based on convolutional neural networks. The spatial inductive biases of convolutional neural networks, which enables them to complete action recognition tasks with fewer parameters and faster reasoning speeds. Although convolutional neural networks are highly effective at extracting local spatiotemporal features, they lack the global spatiotemporal modelling capabilities of transformer-based networks. However, the transformer-based network has a considerable number of model parameters and a relatively slow inference time, which makes it incompatible with the lightweight requirements. In order to meet the requirements of both lightweight design and high recognition accuracy, this paper proposes MCANet, an action recognition network that is suitable for deployment on lightweight devices. MCANet integrates the strengths of convolutional neural networks and transformer-based networks. Furthermore, the network maintains a low number of model parameters and a fast reasoning speed, while also exhibiting local and global spatiotemporal modelling capabilities, thus achieving high recognition accuracy. The efficacy of the proposed methodology was validated on a diverse array of datasets, including Kinetics400, UCF101, and HMDB51, demonstrate the effectiveness of the proposed method.