This paper explores the effectiveness of dropout and batch normalization as regularization strategies in deep learning, specifically for multilayered neural networks used in image classification. Our study conducts empirical experiments to assess the individual and combined impacts of these strategies on training efficiency and model accuracy, considering various hyper-parameter settings. The results indicate that using both dropout and batch normalization tends to slow down the training process but improves the overall robustness and accuracy of the model. Additionally, we compare the SGD and RMSProp optimizers in this context, finding that SGD generally enhances classification performance, though it extends training time. The study also observes a longer testing duration with the application of both regularization techniques. These findings offer practical insights into the optimal use of dropout and batch normalization in neural network training, addressing the existing gap in comprehensive studies on their application in relation to different data inputs, network configurations, and learning efficiency. Codes are available111https://github.com/adsf0427/COMP9501.
Junyi Li (Thu,) studied this question.