Answers are grounded in this paper and cite their sources.
Recent studies have revealed the remarkable capabilities of multimodal large models (MLLMs) in general vision-and-language tasks, generating increasing interest in their application to geoscientific domains. Rock image recognition and lithological description constitute fundamental skills for geologists and represent one of the earliest application scenarios in geosciences where artificial intelligence technologies have been actively explored. As rock image recognition and description are inherently multimodal tasks, involving both visual and textual data, models are required to understand the interactions between modalities and how these interactions affect information transfer. However, the simultaneous comprehension of rock images and their corresponding lithological descriptions remains a significant challenge for current machine learning frameworks. The impact of multimodal co-training on downstream geoscientific performance is still unconfirmed. This study presents PetroMind, a domain-adapted multimodal petrographic foundation model built upon Qwen2.5-VL, together with SA-Rock, a novel LLM-based metric designed to assess the semantic accuracy of rock image generation descriptions to evaluate the capabilities of MLLMs in petrographic multimodal tasks. In the rock image classification task, PetroMind achieves performance comparable to ViT-Base-Patch16-224, with accuracy and Macro-F1 scores exceeding or approaching 97 %. This demonstrates PetroMind's strong capability in long-tailed image learning and highlights its effectiveness in substantially improving classification accuracy for few-shot rock image categories. In the lithological description generation task, PetroMind attains BLEU-4 and SA-Rock scores of 0.644 and 7.864, respectively, indicating good few-shot learning performance. The SA-Rock metric shows that the model produces highly accurate descriptions of rock structure, texture, and colour, while leaving considerable scope for improvement in the description of mineral composition. Ablation experiments further indicate that task-specific LoRA adapters are more effective for high-resource tasks such as image classification, whereas a single shared LoRA adapter demonstrates superior multi-task interaction in low-resource captioning scenarios. This study demonstrates the potential of MLLM-based architectures in jointly understanding rock images and their associated geological descriptions.
No takes yet. Share an insight, caveat, or question.
Chen et al. (2025) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: