The Combined Statistical Stepwise and Iterative Neural Network Pruning Algorithm

Abstract In this paper, we present a new pruning algorithm formed by combining the Statistical Stepwise Method (SSM) 1 with the Iterative Pruning (IP) 4 algorithms. This proposed algorithm (SSIP) is used to simultaneously remove unnecessary neurons or weight connections from a given feed-forward neural network (NN) in order to “optimize” its structure. Some modifications to the previous pruning algorithms published in 1 and 4 are also reported. Two versions of the combined SSIP are considered: In the fast version, SSIPl, the modified IP is fast applied to the given neural network in order to prune insignificant units, and then the modified SSM is applied to the pruned network to remove unnecessary links. In the second version, SSIP2, the above procedure is applied to each layer in turn, working from the input layer to the output layer. The performances of the algorithms are compared using two real world applications, brain disease detection and texture classification, and the superiority of the SS...