Challenges and feasibility of multimodal LLMs in ER diagram evaluation

This study investigates the capabilities of four multimodal large language models (MMLLMs), namely GPT-4o, Claude-3 Sonnet, Grok-4, and Gemini-2.5-pro, in assessing Entity-Relationship (ER) diagrams created by students. These diagrams are widely used in database design education, yet their evaluation is time-consuming and often subjective. Automated assessment may reduce instructor workload and provide timely, formative feedback. We developed a structured rubric to evaluate entities, attributes, relationships, and cardinalities, and conducted a controlled experiment with forty student diagrams created in Chen notation and Crow’s Foot notation Models were evaluated under four conditions that combined two types of input context, case description based and solution comparison, with two prompting strategies, with or without Chain of Thought(CoT) reasoning. The results of the structural analysis showed that the models extracted the main components of the diagrams reasonably well, although cardinalities were the most difficult to interpret. For rubric-based scoring, alignment with human grading improved when CoT reasoning was used with Chen notation, although its effect in Crow’s Foot notation was mixed. These findings indicate that notation style, input context, and reasoning play important roles in automated evaluation, and that multimodal models can offer scalable support for formative assessment in database education.