Class Imbalance in IoMT Datasets: Evaluating Balancing Strategies for Learning-Based Attack Detection

Internet of Medical Things (IoMT) devices are inherently vulnerable to cyberattacks, typically due to their limited processing power and memory capacity. Their widespread use in healthcare poses a significant security risk, threatening patient data privacy and the continuity of services. This study examines the effects of data imbalance correction and balancing strategies on the performance of machine and deep learning models using openly available IoMT datasets. In this context, four different balancing methods—RandomUnderSampler, SMOTE, Borderline-SMOTE, and ADASYN—were applied to three open-access IoMT datasets: ECU-IoHT, WUSTL, and CICIoMT2024. Performance analyses were conducted using five machine learning algorithms (AdaBoost, Logistic Regression, Random Forest, XGBoost, and K-Nearest Neighbor (KNN)) and two deep learning algorithms (Convolutional Neural Networks (CNN) and Deep Neural Networks (DNN)). In the highly imbalanced binary setting of the CICIoMT2024 dataset, the combination of RandomUnderSampler and SMOTE under the balanced-training/original-testing scenario produced the strongest improvement in the binary CICIoMT2024 setting, increasing the F1-Score from the unbalanced baseline to 99.87% for Random Forest and 99.86% for XGBoost across repeated runs. However, the benefit of balancing was not universal. In datasets with stronger class separability, such as ECU-IoHT, and in several multi-class settings, the effect of balancing was limited or, in some cases, inferior to the unbalanced baseline. These findings indicate that balancing is most effective under specific conditions, particularly in highly imbalanced binary tasks, and should be validated using class-sensitive metrics rather than overall performance alone.