A controlled sensitivity analysis of Bayesian optimization settings for CNN-based short-term load forecasting

This study investigates the influence of Bayesian Optimization (BO) algorithmic settings on the hyperparameter optimization and forecasting performance of Convolutional Neural Network (CNN) models for short-term electricity load forecasting. Rather than proposing a new forecasting architecture or conducting a broad comparison among optimization algorithms, this work focuses on a controlled sensitivity analysis of BO internal configurations. The objective is to isolate how different BO settings affect CNN tuning outcomes under the same forecasting model, input window, search space, data-processing procedure, and evaluation protocol. The analysis focuses on two key BO settings: the acquisition function and the number of optimization iterations. Four acquisition functions, namely Expected Improvement, Probability of Improvement, Upper Confidence Bound, and Thompson Sampling, were evaluated under four iteration budgets of 5, 10, 15, and 20 trials. These settings form 16 fixed BO configurations. For each BO configuration, CNN hyperparameters were optimized within the same predefined search space, while the overall CNN topology and the remaining experimental conditions were kept consistent. Specifically, the number and order of layers, kernel size, activation function, padding scheme, dropout rate, optimizer, learning rate, and output structure remained unchanged. In contrast, the number of filters in the four Conv1D layers, batch size, and number of training epochs were optimized within predefined candidate sets. This design helps ensure that the observed differences mainly reflect BO-setting effects rather than changes in the overall forecasting framework or data treatment. Mean Absolute Percentage Error (MAPE) was used as the sole forecasting error metric for all BO trials and repeated runs. This choice was made because the purpose of the study is to compare the relative influence of BO configurations under a single, consistent, and interpretable evaluation criterion, rather than to conduct a broad multi-metric forecasting benchmark. MAPE expresses forecasting error as a percentage, making it intuitive for short-term load forecasting and suitable for comparing regional electricity load datasets with different demand magnitudes. Using the same MAPE-based criterion for BO trial evaluation, configuration selection, and final analysis also avoids inconsistencies that may arise when different metrics emphasize different aspects of forecasting error. The results show that the choice of acquisition function affects both forecasting accuracy and optimization stability. Expected Improvement generally provides relatively stable performance across the tested datasets, while Probability of Improvement can achieve competitive results in some cases. In addition, increasing the iteration budget does not guarantee uniformly lower MAPE across all datasets, and smaller iteration budgets can still provide competitive MAPE values in some dataset-specific settings while requiring fewer CNN training trials. The runtime analysis further shows that larger iteration budgets generally require higher total runtime because they involve more CNN training trials. These findings suggest that BO internal settings should be selected carefully when optimizing CNN models for short-term load forecasting, especially when computational resources are limited and stable optimization behavior is required. The conclusions of this study should therefore be interpreted as MAPE-based findings and runtime observations within the controlled BO-configuration analysis considered in this work.