Comparison of Matrix Decomposition in Null Space-Based LDA Method

Problems with small sample sizes and high dimensionality are common in pattern recognition. Almost all machine learning algorithms degrade in high-dimensional data, so that singularities in the scatter matrices, the main problem of the Linear Discriminant Analysis (LDA) technique, might result. A null space-based LDA (NLDA) has been conceived to address the singularity issue. NLDA aims to maximize the distance between classes in the null space of the within-class scatter matrix. In the first research, the NLDA method was performed by computing the eigenvalue decomposition and singular value decomposition (SVD). This research led to several new implementations of the NLDA method that use other matrix decompositions. The new implementations include NLDA using Cholesky decomposition and NLDA using QR decomposition. This paper compares the performance of three NLDA methods that use different matrix decompositions, namely, SVD, Cholesky decomposition, and QR decomposition. Two sets of data were used in the experiments that used three different NLDA algorithms. To determine the performance of the NLDA methods, the classification accuracy of the three methods was measured using the confusion matrix. The results show that the NLDA method using SVD has the best performance when compared to the other two methods, achieving a precision of 77.8% accuracy for the Colon dataset and a precision of 98.8% accuracy for the TKI-resistance dataset.