Evaluating the Effectiveness of AI Source Disclosure in Human–AI Communication

Large language models can generate language that is often indistinguishable from language generated by humans, but they lack human motivations, beliefs, and accountability. These systems pose risks to our information ecosystem: bias amplification, fabrication, and misinformation are some of the forecasted negative consequences of mass adoption of the technology. The recurring regulatory proposal to mitigate such risks is obligatory source disclosure. The underlying assumption is that if people know the origin of AI-generated language, they can exercise appropriate caution when engaging with it. Here we apply concepts from linguistics and cognitive science to ask what appropriate caution means when engaging with AI-generated linguistic content. We discuss an idealized model of human communication as a motivated activity aimed at increasing the mutually shared beliefs among conversation partners. Building on this model, we develop a set of conceptual tools and empirical signatures to evaluate whether humans engage with AI-generated linguistic messages in the same way as they would with a fellow human or if they approach it differently. Our preliminary empirical investigation implies that even when humans know that some language was generated by AI, they nevertheless treat it on par with human language, albeit with somewhat diminished trust. The main implication of this finding is that source disclosure is not a sufficient regulatory strategy to manage risks that will arise with the proliferation of synthetic language.