Machine Learning with Resilient Propagation in Quaternionic Domain

In the last few years there have been a growing number of studies concerning the introduction of quaternions into neural networks, which demand a faster learning technique with superior performance. In this paper, we propose a fast, but novel quaternionic resilient propagation (-RPROP) algorithm for high dimensional problems. It achieves significantly faster learning over quaternionic domain back propagation (-BP) algorithm. The slow convergence and stability of weight update around the local minima are the main drawbacks of -BP. The gradient descent based -BP algorithm takes the value of partial derivative (error gradient) and scales the weight updates through a learning rate while -RPROP does not takes the value of partial derivatives, but it considers only the sign of partial derivatives that indicates the direction for each component of quaternionic weight update. The main aim of -RPROP is to eliminate the value which is a little increased by constant increasing factor in order to accelerate convergence in shallow regions. -RPROP computes an individual delta for each connection of the network, which determines the size of weight update. Therefore, the faster convergence and higher accuracy are the main key features of proposed algorithm. The intelligent behavior of the proposed learning approach is demonstrated through a wide spectrum of prediction problems with different statistical performance evaluation metrics. In order to illustrate the learning and generalization of 3D motion as its inherent behavior, a solid set of experiments is presented where the training is performed through input-output mapping over a line and the generalization ability is verified over various non-linear geometrical objects. The slow convergence problem of back-propagation algorithm has been well combated by -RPROP. It has always demonstrated drastic reduction in the training cycles.