Food Image Ingredient Recognition Based on VMamba and Cross-layer Attention

This is the authors' abstract. We don't add key points for this paper.

Food image analysis remains challenging owing to substantial visual variations, multiscale ingredient appearances, and the computational demands of global visual modeling.In practical dietary sensing systems, food images are acquired by vision sensors and processed on edge devices, placing additional requirements on both recognition accuracy and computational efficiency.To address these issues, we propose a lightweight multitask framework for joint dish classification and ingredient recognition.The framework employs VMamba-Tiny to provide efficient long-range visual modeling and introduces a cross-layer attention block to integrate fine-grained texture cues with high-level semantic information for multiscale ingredient representation.In addition, a decoupled feature integration strategy (DFIS) adopts a forwardshared and backward-decoupled design, preserving ingredient representations for dish classification while restricting the backward effect of the dish-classification loss on ingredientspecific representations.Experiments on Vireo Food-172 and ISIA Food-200 demonstrate that the proposed model achieves Micro-F1 scores of 81.41 and 72.78%, respectively, with only 7.1 giga floating-point operations.Further analysis shows that the improvement introduced by DFIS becomes increasingly pronounced for low-frequency ingredients, with gains on Tail classes approximately 2.4-3.2×larger than those on Head classes.Moreover, the model achieves 25.8 frames per second on an NVIDIA Jetson Orin Nano with a batch size of 1, supporting its use as the inference component of a vision-sensor-based edge dietary monitoring system.