AI-Assisted Design Judgment: Perceived AI Interaction Conditions, Critical Evaluation of AI Advice, and Expert-Rated Design Judgment Quality

Generative artificial intelligence (AI) is increasingly used to analyze and compare design alternatives, yet how design students evaluate AI advice and how such evaluation relates to final design judgments remain insufficiently understood. This study examines the associations among perceived AI interaction conditions, critical evaluation of AI advice, and expert-rated Design Judgment Quality. A cross-sectional study combined a standardized poster comparison task with a post-task questionnaire involving 118 visual communication design students. Participants interacted with Doubao-Seed-2.1-Pro and submitted design choices, rationales, and revision recommendations. Perceived Feedback Diagnosticity, Interaction Controllability, and three critical evaluation activities were measured through post-task self-reports, while Design Judgment Quality was independently assessed by three experts. Partial least squares structural equation modeling (PLS-SEM) was used for hypothesis testing. Feedback Diagnosticity was positively associated with Evidence Verification, Argument Scrutiny, Contextual Fit Evaluation, and Design Judgment Quality. Interaction Controllability was positively associated with Argument Scrutiny and Contextual Fit Evaluation, but not directly with Design Judgment Quality. Among the three evaluation activities, only Argument Scrutiny showed a significant positive association with expert-rated Design Judgment Quality. These findings suggest that AI-assisted design judgment requires attention to both the perceived value of AI feedback and how users examine AI recommendations and their underlying rationales.