Accentedness ratings of Brazilian speakers’ L2 English by LLMs: Systematic upward shift from the human baseline

This study examines whether audio-capable large language models (LLMs) align with human patterns of L2 English accentedness judgment. Google Gemini Flash and Gemini Pro rated the L2 English speech of Brazilian Portuguese speakers from the CSLU Foreign Accented English Corpus ( Lander, 2007 ) and were compared against native American English listeners. Both models assigned systematically more severe ratings, using the highest category of the rating scale at frequencies not observed in the human data. The shift was most pronounced in Gemini Pro ( OR = 8.25), suggesting that greater model capacity does not yield closer alignment with human judgment. The two models also failed to agree with each other. These divergences raise concerns about the validity of LLM accentedness ratings as proxies for human judgment.