Answers are grounded in this paper and cite their sources.
In this study, we leverage advancements in large language models (LLMs) for fine-grained food image classification. We achieve this by integrating textual features extracted from images using an LLM into a multimodal learning framework. Specifically, semantic textual descriptions generated by the LLM are encoded and combined with image features obtained from a transformer-based architecture to improve food image classification. Our approach employs a cross-attention mechanism to effectively fuse visual and textual modalities, enhancing the model’s ability to extract discriminative features beyond what can be achieved with visual features alone.
No takes yet. Share an insight, caveat, or question.
Kim et al. (2024) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: