Class imbalance is a real‐life issue that introduces bias into the training and test sets and the overall model’s performance. Therefore, class imbalance should be handled appropriately. The question of whether the cross‐validation type has an impact on the training/test split and, consequently, on the model’s performance in imbalanced classification has not been sufficiently explored in academic literature. Some authors recommend applying stratified k‐fold cross‐validation to tackle the effects of class imbalance on the training and test sets and avoid the overfitting of the majority class in the final model. However, scientific proof is not enough to confirm that the impact of class imbalance on the training and test sets can be mitigated by any type of cross‐validation, including the tenfold stratified cross‐validation. The aim of this research is to test whether the tenfold stratified or any other type of cross‐validation can be more suitable for a train/test split in imbalanced data so that the model’s performance is unbiased. The key findings from this research are that contrary to existing literature, the tenfold stratified cross‐validation cannot handle the bias in the training and test set coming from the imbalanced target variable. Moreover, there is not any other cross‐validation type that can be used for this purpose. In fact, other factors such as appropriate model selection and resampling techniques for class imbalance are key when dealing with imbalanced data rather than the type of cross‐validation.
Borislava Toleva (Thu,) studied this question.