High Coverage, Low Information? Careless Responding and the Validity of Institutional Student Evaluation in Chinese Higher Education

Large student surveys increasingly inform educational planning and organisational comparison, but high participation does not guarantee attentive responses. This study examined response quality and its institutional consequences in an administratively mobilised survey at a public university in northern China (N = 5,144). Response patterns were evaluated across four consecutive ordered-response blocks comprising 67 items on three scale metrics. A conservative primary rule flagged respondents who gave completely invariant answers in at least two of three core blocks; separate blocks were then used for cross-block sensitivity analysis. Within-block invariance ranged from 40.2% to 60.9%, and 49.4% of complete cases met the primary rule. After screening, Cronbach’s alpha decreased for all seven scales, top-category endorsement fell by 6.7–10.8 percentage points, a focal correlation declined from .644 to .495, and the ordering of 11 colleges changed. Estimated inflation was greater for indicators with lower endorsement in the screened sample, a descriptive pattern consistent with compression of the differences that institutional evaluation is intended to detect. Cross-block analyses produced the same directional pattern, although with smaller effects. The findings illustrate a coverage–quality paradox: administratively successful data collection can still yield weak evidence for educational governance. Response-quality assessment should therefore accompany response-rate reporting, group comparison, and institutional use of student survey results.