Improving the Performance of Constructed Neural Networks with a Pre-Train Phase

A multitude of problems in the contemporary literature are addressed using machine learning models, the most widespread of which are artificial neural networks. Furthermore, in recent years, evolutionary techniques have emerged that identify both the architecture of artificial neural networks and their corresponding parameters. Among these techniques, one can also identify the artificial neural networks being constructed, in which the structure and parameters of the neural network are effectively identified using Grammatical Evolution. In this work, a pre-training stage is introduced in which an artificial neural network with a fixed number of parameters is trained using some optimization technique such as the genetic algorithms used here. The final result of this additional phase is a trained artificial neural network, which is introduced into the genetic population used by Grammatical Evolution in the second phase. In this way, finding the overall minimum of the error function will be significantly accelerated, making the second phase method more efficient. The current work was applied to many classification and regression problems found in the related literature, and it was compared against other methods used for neural network training as well as against the original method used to construct neural networks.