Genetic deep neural networks using different activation functions for financial data mining

A Deep Neural Network (DNN) using the same activation function for all hidden neurons has an optimization limitation due to its single mathematical functionality. To solve it, a new DNN with different activation functions is designed to globally optimize both parameters (weights and biases) and function selections. In addition, a novel Genetic Deep Neural Network (GDNN) with different activation functions uses genetic algorithms to optimize the parameters and selects the best activation function combination for different neurons among many activation function combinations through sufficient simulations. Two sample financial data sets ("Dow Jones Industrial Average" and "30-Year Treasury Constant Maturity Rate" were used for performance analysis. Simulation results indicate that a GDNN using different activation functions can perform better than one using a single activation function. Future works include (1) developing more effective DNNs using different activation functions by using both cloud and GPU computing, (2) creating more effective DNNs by using new training optimization methods different from genetic algorithms, (3) using big data to further test the performance of the new GDNN, and (4) expanding its big data mining application areas (i.e. health and biomedical informatics, computer vision, social networks, security, etc.).