Controlled thematic prompting for Indonesian Quran QA dataset construction with matched cross-generator LLM validation

Constructing high-quality grounded question-answer (QA) datasets remains a major bottleneck in developing reliable retrieval-augmented generation and question-answering systems, particularly for domain-specific and low-resource corpora. Although large language models (LLMs) can automatically generate QA pairs, the quality of generated datasets depends heavily on prompt design and reliable validation mechanisms. This paper presents a controlled comparison of Single-Path Thematic Prompting (SPTP) and Multi-Path Thematic Prompting (MPTP) for automated grounded QA dataset construction from Indonesian translations of the Quran. Within each generator condition, thematic path structure is treated as the manipulated prompt variable while the task instructions, output schema, grounding constraints, and validation criteria remain fixed. The primary full-corpus experiment generates QA pairs from 4,685 thematically indexed verses using GPT-5.2 and validates them using DeepSeek-V3.2, Mistral-Large-3, and Llama-4-Maverick. To directly evaluate cross-generator portability, we additionally compare GPT-5.2 and GPT-OSS-120B on a four-way matched subset of 2,571 verses. Outputs from both generators under SPTP and MPTP are evaluated using the same open-weighted validator ensemble consisting of Gemma4:31B, Granite4.1:30B, and Qwen3.6:35B. Validation adopts two acceptance criteria: majority consensus (at least two of three validators support the generated answer) and strict consensus (all three validators support the generated answer). On the full proprietary track, MPTP significantly improves strict consensus over SPTP ( p = 0.00415 , McNemar’s test), while majority acceptance remains similarly high. On the matched cross-generator subset, GPT-5.2 achieves majority acceptance rates of 97.3% under SPTP and 97.5% under MPTP, while strict acceptance increases from 86.4% to 88.7% ( p = 0.00274 ). GPT-OSS-120B achieves majority acceptance rates of 94.3% and 94.7%, respectively, whereas strict acceptance remains identical at 80.4% under both prompting conditions ( p = 0.966 ). Majority-level acceptance decisions agree between the two generators for 93.4% of SPTP items and 93.5% of MPTP items. The resulting dataset contains 4,644 automatically validated grounded QA pairs covering 99.1% of the attempted verses. The results provide evidence that the overall generation–validation framework is applicable across different generator families. However, the magnitude of the benefit produced by multi-path thematic prompting is generator-dependent rather than universally invariant.