Observer-based assessment in authentic management simulations: multi-source validity evidence and construct boundaries

Business schools increasingly use experiential simulations to develop applied strategic thinking, intercultural communication, and managerial decision-making. However, the validity of observer-based live assessments remains under-theorised, as existing research has focused mainly on written assignments and presentations. Following Messick’s unified validity framework, this study conceptualised observer ratings as score interpretations that require empirical justification from multiple sources of evidence to be valid. A multi-source, mixed-methods design integrated observer rubric evaluations, peer assessments, pre- and post-program surveys, and reflective reports from multicultural student teams participating in international business strategy battles involving headquarters–subsidiary negotiations (treatment: N = 42; control: N = 31; peer ratings: N = 412; reflection reports: N = 80). The results indicated strong internal consistency, meaningful convergence between observer and peer ratings, and substantial developmental gains across the program’s duration. Convergence patterns suggest that observer and peer systems capture the socially visible dimensions of enacted strategic performance more effectively than latent cognitive processes. The findings position observer-based assessments as a structured validity argument for externally enacted strategic performance in authentic higher-education simulations.