Social reputation and AI overreliance: an experimental test with ChatGPT-3.5

Understanding an agent's true competencies is crucial for a principal, particularly when delegating tasks. A principal may assign a task to an AI system, which is often perceived as highly competent, even in domains where its actual capabilities are limited. This experimental study demonstrates that participants mistakenly bet on ChatGPT-3.5's ability to solve mathematical tasks, even when explicitly informed that it only processes textual data. This overestimation leads participants to earn 67.2% less compared to those who rely on the competencies of another human. Overconfidence in ChatGPT-3.5 persists irrespective of task difficulty, time spent using ChatGPT-3.5, nor prior experience posing mathematical or counting questions to it mitigates this bias. I highlight that overconfidence in ChatGPT-3.5 seems to be explained by the algorithm's social reputation. The more participants perceive ChatGPT-3.5 as socially trusted, the more they tend to rely on it.