The Cramming, Softening and Integrating Learning Algorithm with Parametric ReLU Activation Function for Binary Input/Output Problems

Rare Artificial Neural Networks studies address simultaneously the challenges of (1) systematically adjusting the amount of used hidden layer nodes within the learning process, (2) adopting Parametric ReLU activation function instead of tanh function for fast learning, and (3) guaranteeing learning all training data. This study will address these challenges through deriving the CSI (Cramming, Softening and Integrating) learning algorithm for the single-hidden layer feed-forward neural networks with the binary input/output and making the technical justification. To further verify the proposed learning algorithm, this study conducts an empirical experiment using SPECT heart diagnosis data set from UCI Machine Learning repository. The learning algorithm is implemented via the advanced TensorFlow and GPU.