A framework evaluates emotional individuality in large language models, highlighting methodological implications for emotion scoring tasks.
Evaluating emotional understanding in Large Language Models (LLMs) is challenging because assessments are subjective, ambiguous, multidimensional, and sensitive to controllable generation parameters. We developed a unified mathematical framework for characterizing LLM “emotional individuality” that integrates softmax sampling–temperature control (the decoding-time temperature parameter exposed by the API and typically used to modulate output randomness during token generation), fuzzy set theory with Shannon-type fuzzy entropy, and persona-based cognitive diversity analysis. We evaluated 36 API-accessible LLMs from seven major vendors on Japanese literary texts, using four personas each assigned a sampling temperature (T∈{0.1,0.4,0.7,0.9}), yielding 4227 /4320 trial responses (97.8% coverage), of which 4067/4227 contained valid numeric emotion scores (96.2%). Temperature controllability varied approximately 25-fold (κM∈[0.039,0.982]) with both positive and negative temperature–variance relationships across models. Because each sampling temperature is deterministically assigned to a persona in our design, κM should be interpreted as an operational temperature–variance association across persona conditions rather than an isolated causal temperature effect. The model-level mean fuzzy entropy ranged from approximately 0.40 to 0.66, and the numerical stability consistency scores ranged from approximately 0.548 to 0.780. We also observed text-dependent structure, including genre-specific variation in the Interest–Sadness relationship. For practitioners, the framework is most directly useful as a benchmark-design and model-screening template for structured emotion-scoring tasks; its empirical conclusions remain limited to the present Japanese literary, text-only setting.
No takes yet. Share an insight, caveat, or question.
Shirahama et al. (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: