Abstract The occurrence of concept drift is an important phenomenon in an operational scenario related to machine learning based system which affects the performance of pre-trained models. To maintain the integrity and confidence in predictions, an effective drift detection and adaptation mechanism is a vital component in these operational systems. Apart from different supervised, semi-supervised and unsupervised drift detection techniques, recently the research trend has been focused more on deep learning-based methods specifically based on autoencoders. A hallmark of autoencoders is their versatility. Different types and implementations exist, each specializing in handling specific tasks. In case of drift detection, the focus is to learn the data distribution using autoencoder and to measure the deviation of the newly arriving data in terms of reconstruction loss. While standard or vanilla autoencoders are the most used; other variations also exist and can be evaluated for better drift detection performance. Apart from the type of the autoencoder, another important consideration is the use of right set of parameters and hyperparameters for an autoencoder based drift detection mechanism. In this paper, we provide a framework to optimize the performance of autoencoder based drift detection methods. We provide a theoretical and an empirical evaluation of other applicable types of autoencoders including denoising, variational and standard, followed by a detailed mechanism for selecting the best hyperparameters using grid search and selection of drift detection method’s specific parameters using both grid search and Bayesian Optimization (BO). For experimentation, we have used AEDDM (Autoencoder-based Drift Detection Method) as the base drift detection method to produce Opt-AEDDM- the optimized version of AEDDM. Detailed experiments on 4 synthetic (Hyperplane, Gaussian, VD and RBF) and two real world datasets (NOAA and Forest Covertype) prove the applicability and effectiveness of the proposed framework in finding the best autoencoder with best set of hyperparameters and parameters with improved drift detection performance.
Ali et al. (Fri,) studied this question.