GraphRAG-Based Generation of Mathematics Assessments Using Prerequisite Knowledge Graphs

Large language models (LLMs) provide new opportunities for the automatic generation of educational assessments. However, their application in mathematics remains challenging because they do not explicitly represent prerequisite knowledge and may generate mathematically incorrect content. To address these limitations, this study proposes a GraphRAG-based architecture for generating multiple-choice mathematics assessments. The architecture integrates a directed acyclic knowledge graph (DAG), semantic embeddings, hybrid retrieval, LLM-based generation, and independent mathematical validation using SymPy. The knowledge graph represents prerequisite relationships among mathematical concepts, allowing the system to retrieve structured information to guide question and distractor generation. The evaluation included 154 questions covering basic fraction operations and 462 distractors, of which 53.2% showed direct correspondence with prerequisite knowledge and 30.7% showed indirect correspondence. Mathematical validation accepted 137 questions (89.0%) without correction, while 17 (11.0%) required correction and subsequently passed revalidation. In addition, a controlled comparison involving 80 paired generation requests examined conventional RAG and GraphRAG under comparable generation conditions and the same maximum context limit. GraphRAG produced a higher proportion of distractors with identifiable prerequisite relationships than conventional RAG (91% versus 77%). A complementary evaluation by 24 mathematics teachers using a five-point Likert scale showed favorable perceptions across five dimensions, with coherence receiving the highest mean score (M = 4.68) and usefulness for teachers the lowest (M = 4.36). Overall, the findings provide preliminary evidence of the technical feasibility of the proposed architecture and suggest that the complete GraphRAG retrieval configuration is associated with stronger prerequisite–distractor correspondence. However, the comparison does not isolate the contribution of graph relationships from possible differences in retrieved context, and the study does not establish diagnostic validity or improvements in student learning outcomes.