Red teaming for culturally grounded LLM evaluation: lessons learned from a Japanese team’s engagement with stereotype risks

Abstract This paper examines how culturally grounded red teaming operates in a non-English context through a qualitative case study of Japan’s participation in the 2024 IMDA Red Teaming Challenge. While the challenge provides the shared institutional framework, the analysis focuses on Japan’s four-phase implementation, including expert-driven stereotype list development, in-person and virtual prompt generation, independent annotation, and interdisciplinary reflection. The findings show that Japan’s monolingual environment shaped the emergence of region-, gender-, and appearance-related stereotypes, and that participants intentionally selected Japanese-language prompts to test culturally embedded harms. The stereotype list lowered cognitive burden but sometimes oversimplified context, while annotation revealed recurring gray-zone dilemmas, such as distinguishing descriptive inequalities from normative generalizations and separating stereotypes from inference-level hallucinations. Interdisciplinary exchanges further showed that stereotype evaluation is an interpretive rather than purely technical task. The study positions culturally grounded red teaming as a sociotechnical process shaped by local norms and human judgment, offering methodological implications for more context-sensitive evaluation practices in non-English and culturally specific settings.