Abstract The effectiveness of machine learning models heavily depends on the quality and relevance of the features extracted from raw data. Automated Feature Engineering (AutoFE) offers a scalable solution by generating large pools of candidate features. However, the unfiltered expansion of features introduces redundancy, exacerbates computational overhead, and may impair model generalization due to the curse of dimensionality. To address this, we propose Feature Forgetting, a proactive, dynamic pruning mechanism that discards redundant or non-informative features during the feature construction phase itself. Unlike conventional post-hoc feature selection methods such as Lasso or Recursive Feature Elimination, our approach integrates feature relevance evaluation into the generation loop using mutual information, variance inflation factor, and Shannon entropy. Empirical evaluations on three benchmark datasets — credit risk prediction, customer churn analysis, and healthcare diagnostics — reveal that Feature Forgetting can reduce feature dimensionality by up to 50%, cut training time by 40%, and improve classification accuracy by as much as 5%. These findings support the utility of real-time feature curation in constructing interpretable and efficient AutoML pipelines.
Patankar et al. (Wed,) studied this question.