Multimodal RAG System for Rose Cultivation Using Paragraph-level Chunking and VLM-based Image Captioning

Key Points

• The study aims to enhance the accuracy of RAG systems in agriculture by integrating textual and visual data for rose cultivation.
• Developed a multimodal RAG system combining paragraph-level chunking and VLM-based image captioning.
• Isolated text and images for extraction and built a vector database using context-integrated captions generated by GPT-4o.
• Performance evaluated by comparing Recall@5 and Answer Relevancy metrics against traditional text-based RAG systems.
• The proposed system improved Recall@5 by 28.8% compared to traditional methods.
• Achieved a 33.3% increase in Answer Relevancy over conventional text-based RAG systems.

Abstract