A data analysis method based on iterative preprocessing and ensemble models for classifying the technical condition of power transformers

Data preprocessing is a crucial stage in solving problems related to information transformation and analysis. In data processing and analysis systems that receive input from various sources (especially in cases where data collection processes have a low degree of automation and monitoring systems do not fully cover the equipment) datasets often contain a significant number of missing values, as well as potential outliers and format errors. Simply removing records or features with a high proportion of missing values can lead to an excessive data loss. In tasks related to the assessment of the technical state of equipment, an additional factor preventing the removal of a large number of records is a class imbalance. Specifically, the number of records corresponding to an unsatisfactory technical condition is one to two orders of magnitude smaller than those representing satisfactory and good technical state. Therefore, it is crucial to develop preprocessing methods that retain as much of the original data as possible. As an example, we consider the assessment of the technical state of power oil-filled transformers. The existence of numerous diagnostic methods, each based on physical processes of different nature, necessitates an aggregated data analysis approach. To address this, an iterative feature selection and missing value imputation algorithm was proposed as a preprocessing method. This algorithm increased the data sets completeness from 75 % to 94 %, thereby reducing the number of missing values that required synthetic imputation by a factor of 2.5. The processed dataset was then used to train machine learning models designed to classify the transformer technical state. To account for both multi-class classification and dataset imbalance, a two-stage classification approach was implemented. The final F1-score of the classification model reached 83%, demonstrating the effectiveness of the preprocessing and modeling pipeline.