Key points are not available for this paper at this time.
Abstract Recent advances in natural language generation (NLG), such as public accessibility to ChatGPT, have sparked polarised debates about the societal impact of this technology. Popular discourse tends towards either overoptimistic hype that touts the radically transformative potentials of these systems or pessimistic critique of their technical limitations and general ‘stupidity’. Surprisingly, these debates have largely overlooked the exegetical capacities of these systems, which for many users seem to be producing meaningful texts. In this paper, we take an interdisciplinary approach that combines hermeneutics—the study of meaning and interpretation—with prompt engineering—task descriptions embedded in input to NLG systems—to study the extent to which a specific NLG system, ChatGPT, produces texts of hermeneutic value. We design prompts with the goal of optimising hermeneuticity rather than mere factual accuracy, and apply them in four different use cases combining humans and ChatGPT as readers and writers. In most cases, ChatGPT produces readable texts that respond clearly to our requests. However, increasing the specificity of prompts’ task descriptions leads to texts with intensified neutrality, indicating that ChatGPT’s optimisation for factual accuracy may actually be detrimental to the hermeneuticity of its output.
Henrickson et al. (Mon,) studied this question.