Chatbot or Humanaut? How the Source of Advice Impacts Behavior in One-shot Social Dilemmas

Abstract This paper explores how the source of advice (human vs. AI chatbot) impacts behavior in three classic social dilemmas. Utilizing a novel experimental design, we show that advice enhances welfare, but when it comes to prosociality, the source matters. Players exhibited a preference for human advice over that from generative AI (genAI) and were more willing to pay for it. Prosocial behavior was more prevalent when players received human advice compared to genAI, increasing the probability of the cooperative stag hunt strategy and significantly driving contributions in public goods and dictator games. Potential explanations include genAI's limitations in being truly empathetic and the possibility of moral wiggle room. Leveraging natural language processing (NLP) advances through large language models (LLMs) topic modeling, we demonstrate that the human and genAI advice differ significantly. Human advice was more objective, specific, intuitive, and norm-oriented, while AI advice tended to offer reasoning and targeted concepts of risk. We conclude that the adoption of genAI technologies should be balanced with an investment in human connectivity and organizational input with a goal of the constrained agency. Preregistered at Aspredicted.org #128933