Metaheuristics Approach for Hyperparameter Tuning of Convolutional Neural Network

Deep learning is an artificial intelligence technique that has been used for various tasks. Deep learning performance is determined by its hyperparameter, architecture, and training (connection weight and bias). Finding the right combination of these aspects is very challenging. Convolution neural networks (CNN) is a deep learning method that is commonly used for image classification. It has many hyperparameters; therefore, tuning its hyperparameter is difficult. In this research, a metaheuristic approach is proposed to optimize the hyperparameter of convolution neural networks. Three metaheuristic methods are used in this research: ant colony optimization (ACO), genetic algorithm (GA), and Harmony Search (HS). The metaheuristics methods are used to find the best combination of 8 hyperparameters with 8 options each which creates 1.6. 107 of solution space. The solution space is too large to explore using manual tuning. The Metaheuristics method will bring benefits in terms of finding solutions in the search space more effectively and efficiently. The performance of the metaheuristic methods is evaluated using MNIST datasets. The experiment results show that the accuracy of ACO, GA and HS are 99,7%, 97.7% and 89,9% respectively. The computational times for the ACO, GA and HS algorithms are 27.9 s, 22.3 s, and 56.4 s, respectively. It shows that ACO performs the best among the three algorithms in terms of accuracy, however, its computational time is slightly longer than GA. The results of the experiment reveal that the metaheuristic approach is promising for the hyperparameter tuning of CNN. Future research can be directed toward solving larger problems or improving the metaheuristics operator to improve its performance.