Marginal Contrastive Loss: A Step Forward for Forward-Forward

The Forward-Forward algorithm was developed to increase the resemblance of artificial neural network training processes to those occurring in the brain, in contrast to the backpropagation algorithm, which has been shown to have less similarity to brain processes. While Forward-Forward is a fascinating and novel idea, it significantly differs in performance from backpropagation. Forward-Forward strives to achieve this similarity by updating each layer independently of the others, through the introduction of a loss function that facilitates the separability of data from different classes. This inherent nature of creating discrimination between data of different classes inspired us to take advantages of contrastive learning to improve Forward-Forward performance. We modified the contrastive loss to be used in Forward-Forward, and our experimental results show that the proposed method improves the model accuracy and increases the convergence speed by more than 20 times.