Severity Prediction of Hypothyroidism Using Cost-Sensitive Classification and Randomized Search Optimization

Abstract Objective: To perform risk stratification of hypothyroidism patients into different severity levels such as low, moderate and severe using optimized machine learning techniques with a focus on handling class imbalance and improving minority class prediction. Method: This study was conducted using the hypothyroid dataset from Kaggle, consisting of 31 parameters for 3771 patients. The Stratification of Risk severity in the prediction of hypothyroidism was accomplished using machine learning algorithms like Decision Tree, Random Forest and K-nearest neighbor with basic preprocessing steps such as data pre-processing, handling imbalance data, relevant feature selection, and model optimization. The imbalanced dataset was dealt by using SMOTE-ENN to handle minority risk categories. Additionally, a Cost-Sensitive Classifier (CSC) was implemented to reduce critical misclassification errors for important classes like high-risk patients as well as penalizing for the serious mistake. Chi square feature selection was used to identify the most relevant features, improving model efficiency and reduce overfitting. Findings: In Medical field, recall is most important than precision, because missing true patients (false negative) can be clinically dangerous. The proposed model achieve accuracy 98.81%, precision 88.74%, recall 99.2% and F1 score 93.32%. To enhance the risk stratification process, a combined two-layer approach was implemented, that is cost-sensitive classifier and cost sensitive probability were integrated with randomized search CV. While penalizing the misclassification for high-risk patients considerably strengthened the model’s ability to correctly identify high-risk patients. Cost sensitive probability converts raw prediction into risk aware probability, it integrates cost sensitive learning with probability adjustment, ensuring that both model training and final prediction are aligned with clinical priorities. The proposed method successfully classifies hypothyroid patients into low, moderate, and high-risk categories with improved sensitivity towards minority classes. Novelty: This research introduces an innovative hybrid framework that integrates probability-based cost-sensitive learning with Randomized Search optimization to stratify hypothyroidism risk. The primary contribution is the use of predicted probabilities to adjust misclassification costs, thereby giving precedence to clinically significant high-risk cases. Keywords: Hypothyroidism, Risk Stratification, Cost-Sensitive Classifier, Randomized Search CV, SMOTE-ENN, Performance Metrics