Emotion Recognition Using Explainable Genetically Optimized Fuzzy ART Ensembles

There is a growing demand for explainability in complex artificial intelligence solutions to support critical applications’ decision-making processes. Barriers to explainable processes include black-box classifiers, such as deep learning, and noisy datasets. Affect recognition involving neural networks attempts to map complex human emotions onto Arousal and Valence scales based on physiological signal measurements. Datasets collected for this purpose are inherently noisy and may contain outliers and imbalanced classes, hindering accurate classification. In our approach, these issues are addressed using Fuzzy ART (FA) for clustering data samples into more condensed memory templates, introducing stochastic resonant noise to amplify signal-to-noise ratio, and SMOTE sampling to generate synthetic minority samples. A genetic algorithm is developed for FA optimization and ensemble model selection. Clusters obtained from the resulting ensembles are then used to train an ensemble of boosted decision trees for classification and to visualize the decision-making processes. Individual features such as heart rate variability and EEG band power, as well as feature interactions between pairs of features, may contain critical information as human affect indicators. Contributions of individual features and feature interactions toward describing human affect are quantified and interpreted using Shapley additive explanation values. Three established affect recognition datasets were considered for mapping physiological features onto a binary classification of Low/High Arousal and Positive/Negative Valence. Our framework was able to achieve good generalization for both classification tasks as well as provide detailed insights into the contributions of physiological features towards describing Arousal and Valence affects.