Loss Functions Forcing Cluster Separations for Multi-class Classification Using Deep Neural Networks

In Deep Neural Networks (DNNs), the learning process (updating the parameters) is driven by the gradients of a loss function; thus, the loss function does not only measures the "goodness" of classification, but also determines the form of representations. In this paper we propose two loss functions for Deep Neural Networks (DNNs), namely Multi-class Discriminant Analysis Loss Function and Maximum Scatter to Closest Neigh-bor Class loss function, which share the same idea, i.e., they minimize the within-class variance (scatter) and maximize the minimum between-class distance. They overcome the problem of overlapping class in the Linear Discriminant Analysis (LDA)-based loss functions and can be used to multi-class classification tasks. We show that our proposed loss functions achieve state-of-the-art accuracy in a medical image classification task. Particularly, in the Optical Coherence Tomography (OCT) image dataset, we got nearly perfect (99.6%) test accuracy.