What type of study is this?

This is a Quantitative Study study.

October 13, 2025Open Access

Scientific Reasoning: Assessment of Multimodal Generative LLMs

Key Points

Gemini models achieve the highest accuracy in answering scientific questions within minimal context, demonstrating effective reasoning abilities.
On the ScienceQA dataset, Gemini models also exhibit the strongest textual similarity to human explanations with richer contexts, highlighting their advanced capabilities.
Adapter-tuning applied to smaller multimodal LLMs did not yield any significant performance improvements at any stage of assessment.
Training from Gemini outputs consistently resulted in lower performance compared to models trained on the original dataset, indicating a critical discrepancy.

Abstract

Large language models (LLMs) can answer questions and reason about complex tasks, also from the scientific domain. We assess several multimodal LLMs (MLLMs) on ScienceQA and find that Gemini models show the highest accuracy with little context, and the highest textual similarity to human explanations with richer context. Adapter-tuning of smaller MLLMs did not lead to any reliable performance. Training from Gemini outputs consistently underperformed training from the original data.

Connected Papers

Building similarity graph...

Analyzing shared references across papers

Discussion

Cite this study

Dreyer et al. (Sun,) studied this question.

www.synapsesocial.com/papers/68ece2abd1bb2827d1297179 — DOI: https://doi.org/10.48550/arxiv.2503.01064

Authors

Florian Dreyer

Ekaterina Kolos

Daria Matiash

Actions

References and Citations

Connected Papers

Building similarity graph...

Analyzing shared references across papers

Scientific Reasoning: Assessment of Multimodal Generative LLMs

Key Points

Abstract

Citation Network

Connected Papers

Discussion

Cite this study

Authors

Actions

References and Citations

Citation Network

Connected Papers

Discussion