Output Correction of Recurrence-Aware Long-Term Cognitive Network Classifiers

Recurrence-Aware Long-Term Cognitive Network (rLTCN) classifiers have reported comparable performance to mainstream black-box models, including tree ensembles and support vector machines, in tabular pattern classification tasks. These classifiers use a two-step learning algorithm to address issues that arise during the training of recurrent neural networks. While the weights in the recurrent block are computed using unsupervised learning, recurrence-aware weights are determined using a one-step learning rule based on the Moore-Penrose inverse. However, the related least-squares learning problem tends to favor easy instances and common patterns, particularly those associated with the majority class in imbalanced datasets. In such scenarios, a loss function that directly optimizes a robust metric, such as the F1 score, would lead to models with stronger generalization capabilities. Unfortunately, incorporating such a metric into the Moore-Penrose inverse learning procedure presents challenges from a mathematical viewpoint. In this paper, we propose four gradient-based correction methods that modify the output logits of rLTCN classifiers once the two-step training process is done. Inspired by procedures such as Platt or Beta scaling, the proposed post-optimization correction methods seek to maximize the F1 score rather than produce calibrated probabilities. The simulations using real-world datasets show that adding a correction layer to rLTCNs improves their performance significantly at the expense of occasional reductions in the precision metric.