What question did this study set out to answer?

The study aims to investigate the extent of social biases present in large language models during role-playing scenarios.

Fairness Testing of Large Language Models in Role-Playing

View Full Paper

Ask AI

Bookmark

Authors

Empirical study evaluates bias in large language models during role-playing, highlighting prevalent discrepancies.

The study aims to investigate the extent of social biases present in large language models during role-playing scenarios.
Conducted an empirical study on 10 large language models (LLMs) using 33,000 role-specific questions.
Generated questions targeting 11 demographic attributes to assess biases.
Utilized rule-based and LLM-based strategies for identifying biased responses, validated through human evaluation.
Identified 107,580 biased responses across the evaluated LLMs, with individual models producing between 7,579 and 16,963 biased responses.
Demonstrated significant prevalence of bias in role-playing contexts.
Publicly released dataset and evaluation scripts for future research.

Li et al. (2026) studied this question.

View Full Paper

Ask AI

Bookmark

View Full Paper

Ask AI

Bookmark

Authors

Empirical study evaluates bias in large language models during role-playing, highlighting prevalent discrepancies.

The study aims to investigate the extent of social biases present in large language models during role-playing scenarios.
Conducted an empirical study on 10 large language models (LLMs) using 33,000 role-specific questions.
Generated questions targeting 11 demographic attributes to assess biases.
Utilized rule-based and LLM-based strategies for identifying biased responses, validated through human evaluation.
Identified 107,580 biased responses across the evaluated LLMs, with individual models producing between 7,579 and 16,963 biased responses.
Demonstrated significant prevalence of bias in role-playing contexts.
Publicly released dataset and evaluation scripts for future research.

Li et al. (2026) studied this question.

View Full Paper

Ask AI

Bookmark