Answers are grounded in this paper and cite their sources.
User reviews of mobile applications provide a source of citizen feedback on digital public services, yet their automated analysis in non-English settings is constrained by the limited availability of domain-adapted classifiers. This study develops and evaluates a documented procedure for three-class sentiment classification of Czech reviews of government mobile applications using a large language model without task-specific fine-tuning. The corpus comprised 10,198 reviews of six applications. An adjudicated human reference was created for a stratified random sample of 500 reviews, and LLM performance was compared with a majority-class baseline, two lexicon-based methods, and two readily deployable encoder-based transfer models. The validation included five repeated runs under identical settings and an analysis of classification errors. Across five GPT-5.6 Sol validation runs, mean accuracy was 93.0%, mean macro-F1 was 0.827, and mean Cohen's kappa was 0.857. Macro-F1 ranged from 0.814 to 0.845, while agreement among the five runs was high (Fleiss' kappa = 0.918). The neutral class was the most challenging, with most disagreements occurring at its boundaries with negative and positive sentiment. In the full corpus, 65.9% of reviews were classified as negative, 7.9% as neutral, and 26.2% as positive. Textual sentiment was strongly associated with star ratings but was not identical to them. These findings provide corpus-specific evidence that a standardized LLM procedure can help analyze short-form non-english citizen feedback. They also underscore the need for human reference labels, stability checks, and precise documentation of the model, prompt, settings, and execution workflow.
No takes yet. Share an insight, caveat, or question.
Johanovský et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: