• PISA 2022 creative thinking tasks exhibit Differential Item Functioning (DIF) across cultural contexts. • We combine an IRT-based DFIT framework with qualitative item diagnosis. • Our analysis across cultural clusters reveals significant cultural differences. • Creative thinking studies could draw more on item-design teams & cultural experts. Conceptions of creativity differ systematically across cultures, shaping how they are expressed and valued. The introduction of a Creative Thinking (CT) domain in PISA 2022 marks the first large-scale, internationally comparable assessment of this competence among 15-year-olds. However, the availability of this rich cross-national data does not resolve the fundamental validity question of whether the creative thinking tasks function equivalently across diverse cultural contexts at the item level. This study investigates the extent, patterns, and mechanisms of cultural differences in the PISA 2022 CT assessment. We employ a sequential explanatory mixed-methods approach, pairing an IRT-based Differential Functioning of Items and Tests (DFIT) framework with qualitative item diagnosis. DFIT analyses of 30 PISA 2022 CT items across 10 Global Leadership and Organizational Behavior Effectiveness (GLOBE) cultural clusters reveal that while cultural non-invariance is pervasive (flagging 83.3% of items), the severity of bias is uneven. At the cluster level, bias does not suggest an “East–West” divide but distinct regional assessment profiles. To explain these quantitative patterns, a subsequent qualitative diagnosis of the most severely biased items was conducted. This analysis located a four-dimensional bias architecture that may contribute to the observed disadvantage: (1) context universality and familiarity, (2) the cultural match of scenario-activated cognitive prototypes, (3) implicit value assumptions, and (4) response-mode alignment. These findings call for a shift in the field from reactive, post-hoc bias screening to a proactive approach of embedding fairness considerations into the design process, aimed at creating culturally inclusive assessments.
Cen et al. (Sun,) studied this question.