Evaluating the Performance of Chatbots in Literary Criticism on Lima Barreto

In the early 2020s, with the popularization of generative artificial intelligences (GenAIs), discussions intensified regarding the utility of chatbot technology — generative text intelligences — in teaching-learning dynamics. Aiming to verify the capacity of these systems to produce utterances suitable for literary criticism, and to understand their practical limitations. This research compared qualitative and quantitative evaluations of interactions conducted by a human model and by artificial models (ChatGPT, Gemini, and GPT4ALL). The analysis of their performances demonstrated that ChatGPT presented the best results, followed by Gemini and GPT4ALL. Among the discrepant aspects observed in the responses, the following stood out: originality, response length, continuity between responses, vocabulary variability, objectivity of enunciation and the complexity of adapting terms to context. It was also observed that the models' tendency to achieve persuasion through discursive simplification and rhetorical standardization represents a problem for pedagogical dynamics, which should stimulate academic erudition and deep critical thinking.