Reliability of a large language model in supporting agricultural extension officers for land management practices in Indonesia

This study aims to evaluate the effectiveness of a large language model (LLM)-based chatbot in responding to technical agricultural questions commonly asked by farmers, with a focus on its potential to support agricultural self-sufficiency while considering the risks of misinformation. The chatbot’s responses to five frequently asked questions were tested across two contrasting agroecosystems in Indonesia: acidic drylands and swamplands. A total of 33 extension officers in acidic drylands and 37 in swamplands, along with five soil scientists in each ecosystem, assessed the chatbot’s responses using a 0–100 scoring system equivalent to a 1–4 Likert scale. The results showed that extension officers rated the chatbot’s answers higher than soil scientists, with average scores of 88.1 and 80.4 in acidic drylands, and 83.8 and 75.4 in swamplands, respectively. Extension officers categorized the chatbot’s responses as “very accurate,” whereas soil scientists considered them “fair” to “quite accurate.” These findings indicate that the chatbot has the potential to complement extension services by providing farmers with quick, preliminary information, although further refinement is still required for specialized agroecosystem contexts. The study contributes to the growing body of research on digital agricultural advisory services by highlighting both the potential and limitations of LLMs as knowledge intermediaries. The results reinforce the idea that LLMs primarily function as language models rather than knowledge models. They can simulate expertise based on patterns in data but cannot independently verify causal relationships or scientific validity. This research offering insights into the role of AI in supporting agricultural self-sufficiency while mitigating misinformation risks.