What question did this study set out to answer?

To develop and evaluate a human-and-bot adversarial framework that identifies, collects, and mitigates unsafe responses in conversational AI systems.

January 1, 2021Open Access

Bot-Adversarial Dialogue for Safe Conversational Agents

View Full Paper

Ask AI

Bookmark

To develop and evaluate a human-and-bot adversarial framework that identifies, collects, and mitigates unsafe responses in conversational AI systems.
Crowdworkers engaged in multi-turn dialogues with conversational bots, actively attempting to provoke the agents into generating unsafe or toxic responses.
Collected adversarial conversation logs to create the Bot-Adversarial Dialogue (BAD) dataset.
Trained safety classifiers and generative dialogue models iteratively using the collected adversarial data alongside standard safety benchmarks.
Bot-adversarial training significantly improved safety classifier accuracy in detecting nuanced and covertly toxic conversational inputs.
Generative dialogue models trained with adversarial data generated markedly fewer offensive or toxic utterances when challenged by adversarial prompts compared to standard baselines.

Implication

Computational study demonstrates adversarial dialogue training reduces toxic responses in conversational agents, indicating improved safety during interaction.

Abstract

Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, Emily Dinan. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

Discussion

No takes yet. Share an insight, caveat, or question.

Cite This Study

Xu et al. (2021) studied this question.

synapsesocial.com/papers/6a1280c78edbaba0bf676556 https://doi.org/10.18653/v1/2021.naacl-main.235

View Full Paper

Ask AI

Bookmark

Implication

Computational study demonstrates adversarial dialogue training reduces toxic responses in conversational agents, indicating improved safety during interaction.

Bot-Adversarial Dialogue for Safe Conversational Agents

Key Points

Implication

Abstract

Discussion

Cite This Study

Bot-Adversarial Dialogue for Safe Conversational Agents

Key Points

Implication

Abstract

Discussion

Cite This Study

Keep reading

Keep reading