A Cognitive Design System Approach to Enhancing Cognitive Alignment and Difficulty Control in Gen-AI Item Generation

Automatic item generation (AIG) has been transformed by generative AI (Gen-AI), offering high efficiency and flexibility. However, post-hoc explanations of item difficulty do not provide the a priori control necessary for robust psychometric quality. This proof-of-concept study integrates Gen-AI with the cognitive design system approach grounded in information-processing theory. A structured prompt template operationalized difficulty manipulations across three cognitive components (Encoding, Inference/Mapping, Response) derived from verbal analogical reasoning theory. Three Gen-AI models (Claude Sonnet 4, Gemini 2.5 Pro, GPT-4o mini) generated 36 items. Component-level adherence was generally strong but uneven, and 19 items (52.8%) met all specifications under a conservative consensus standard. Gen-AI self-evaluation showed high agreement with human consensus but low chance-corrected agreement, indicating systematic leniency and supporting its use as a supplement rather than a replacement for human review. An iOS prototype demonstrated implementation feasibility. Overall, this study provides initial evidence for theory-driven, cognitively aligned Gen-AI item generation.