Balance Ratio Sum Versus Maximization Ratio Sum for Linear Discriminant Analysis

Linear discriminant analysis (LDA) is a widely used dimensionality reduction (DR) technique that is effective in extracting discriminative features across various fields. Ratio sum LDA (RSLDA), a variant of LDA, was developed to address the shortcoming of the LDA method, which tends to obtain features with weak discriminative information. However, the traditional ratio sum formulation is dominated by the maximum ratio, which makes it difficult to select highly discriminative features and contradicts the original goal of the ratio sum. In this article, we analyzed the underlying causes of the dominance problem in RSLDA. A novel discriminant feature learning method via balance ratio sum discriminant analysis (BRSDA) is proposed. BRSDA effectively balances the ratios of the model formulation, thereby mitigating the domination problem. It focuses on optimizing low-quality projection directions, thereby yielding a well-balanced solution with consistently strong projection quality. First, a minimization ratio sum (Min-RS) criterion is adopted, which leverages the balance property of the harmonic mean to balance the gaps between the ratios. Second, Min-RS is integrated with the ₏ -norm to further balance gaps between ratios. By amplifying the differences across projection directions, the ₏ -norm drives BRSDA to emphasize the optimization of low-quality directions, thus raising the lower bound of direction quality. Finally, since obtaining a closed-form solution for the BRSDA problem is challenging, the gradient descent method is employed to solve its optimization. Sufficient experimental results verify the effectiveness of BRSDA, and BRSDA can effectively solve the domination problem and extract discriminative features.