A Deep Neural Network Optimization Framework Based on Optimal Transport Bridge Feature Selection and Sparse Representation

The performance of deep neural networks (DNNs) in accomplishing tasks heavily relies on feature selection and sparse representation of high-dimensional data. Previous work has treated feature selection and sparse representation as separate mechanisms for improving DNNs performance, focusing on identifying and leveraging informative features to enhance task-specific outcomes. However, few studies have established a connection between feature selection and sparse representation. To address this gap, this article proposes an optimization framework termed informative sparse transport (IST), which integrates feature selection and sparse coding into a unified multiobjective optimization framework. Using optimal transport as a bridge, the IST framework harmonizes the relationship between feature selection and sparse representation, offering an informational advantage. In the IST framework, feature selection aims to identify an optimal subset of features to maximize mutual information or minimize redundancy, while sparse representation seeks to approximate data with the fewest possible features. Although these objectives differ, they are fundamentally complementary, as both emphasize extracting task-relevant information while eliminating redundancy. By unifying feature selection and sparse representation, the IST framework effectively mitigates challenges posed by high-dimensional data, delivering a robust solution for enhanced feature extraction and representation. We validate the IST framework on generative and classification tasks, demonstrating IST framework improves model performance through the complementary synergy of feature selection and sparse representation.