Abstract While ethical and existential concerns about advanced AI are widespread, on the epistemic side there remains considerable optimism. This paper strikes a different tone, warning of the epistemic costs of artificial intelligence that is both extremely capable and widely available. I begin by drawing on literature in computer science to motivate two claims. First, that AI will achieve persuasive capabilities far exceeding those of humans within our lifetimes. Second, that technical challenges in AI training make it plausible that such systems will not be reliably truthful. I then undertake epistemological analysis, identifying the epistemic costs of AI which is super-persuasive but not super-truthful. Chief among these, I argue, is that the prevalence of highly persuasive AI-generated arguments undermines our warrant for beliefs formed through argumentation, generating pervasive undercutting defeat. Additionally, such systems threaten to intensify epistemically defective practices such as motivated reasoning. The paper concludes by suggesting concrete avenues for research, education, and policy aimed at mitigating these risks.
Ari Deller (Sat,) studied this question.