The siren song of the Chatbot: persuasive hallucinations and automation bias in OMI detection

Key Points

• Persuasive hallucinations can lead to misleading OMI detection outcomes, impacting reliability.
• In automated systems, user reliance on chatbot algorithms can amplify automation bias, skewing results.
• The approach utilizes qualitative analysis of user interactions to assess the chatbot's influence on OMI detection tasks.
• Awareness of these biases is crucial, as they may affect the determination of critical outcomes in real-world settings.