Deep learning has shown promise in tabular data modeling, yet challenges such as feature heterogeneity, sparse interactions, and expert prediction collapse remain unresolved. To address these issues, we propose DETTab (Diversity-Enhanced Tabular Experts), a framework that integrates feature gating, multi-expert fusion, and structure-aware regularization. DETTab first employs a Feature Gating Encoder to perform soft selection over input fields, enhanced by a Field Decorrelation Loss to promote embedding diversity. A Feature Interaction Encoder is then used to capture high-order dependencies among features via multi-head self-attention. Finally, a Multi-View Expert Fusion Module aggregates predictions from multiple experts through a soft routing mechanism, guided by an Expert Diversity Loss to mitigate prediction collapse and improve training stability. Extensive experiments on public tabular datasets demonstrate that DETTab achieves consistent improvements in predictive performance and training robustness across different settings, particularly in alleviating expert convergence collapse, thereby validating its effectiveness for tabular learning.
Wang et al. (Fri,) studied this question.