A CLIP-Guided Neuro-Fuzzy Framework for Interpretable Affective Modeling of Artworks

Interpretable analysis of affect-related artwork scores requires an explicit link between visual attributes and model outputs. This study develops a CLIP-guided neuro-fuzzy framework that maps eight painting-domain attributes to four computational scores: comfort, pleasure, stress, and anxiety. CLIP-derived pseudo-labels provide weak supervision for four adaptive neuro-fuzzy inference system (ANFIS) branches, whose membership functions and first-order rules expose how attributes contribute to each output. The final deployment model comprises 56 rules and 332 parameters and operates on extracted attributes without invoking CLIP. Evaluation on 139 curated artworks uses 10 repetitions of nested five-fold cross-validation, with feature and training-duration selection confined to training data. ANFIS achieves a mean absolute error (MAE) of 0.437 and dimension-wise Pearson correlations of 0.213–0.325 with CLIP scores. Input-matched ridge regression and a cached CLIP-embedding ridge probe yield MAEs of 0.430 and 0.126, respectively, quantifying the fidelity of the rule-based representation relative to alternative predictors. Out-of-fold evaluation against 137 author-curated category records yields AUCs of 0.578–0.654. Complete rule bases, held-out rule traces, and sensitivity analyses make the learned attribute–score relationships available for inspection and reproduction. An exploratory survey of 54 completed questionnaires covering eight nominal-category items characterizes variation in reported affect, with limited inter-rater agreement (Fleiss’ κ=0.086). The framework connects explicit visual attributes, vision–language supervision, and inspectable fuzzy inference, providing a reproducible basis for computational analysis of artworks and subsequent dimension-specific human assessment.