Kernelized Manifold-Optimized Linear KNN for Nonlinear Data Classification

In sparse representation learning-based linear k-nearest neighbors methods, the linear representation assumption frequently fails when applied to nonlinear distributed data, leading to degraded generalization and a loss of physical interpretability. To address this, we propose the Kernelized Manifold-Optimized Linear Nearest Neighbor (KMOLNN) method. Methodologically, KMOLNN projects the data into a high-dimensional kernel space to capture the nonlinear relationships, while introducing an adaptive manifold-preserving regularization term—via an adaptive Laplacian matrix—to dynamically preserve the local geometric structures. Theoretically, this study provides a mathematical proof of the nearest neighbor group effect for the kernel framework and reveals that its weight optimization behavior implicitly implements the Bayesian decision rule. Furthermore, we derive a rigorous generalization error bound using Rademacher complexity to validate its theoretical robustness. Empirically, we evaluate KMOLNN on 15 small-to-medium-scale benchmark datasets against eight comparative methods, including recent variants. The results demonstrate significant numeric superiority, with KMOLNN achieving an average accuracy of 90.76% and a Macro F1-score of 88.62% across the evaluated datasets. Finally, we present a comprehensive runtime analysis, explicitly acknowledging that these gains in generalization capability and theoretical interpretability present a practical trade-off, requiring increased computational runtime due to the iterative alternating optimization process.