Information backpropagation for blind separation of sources in nonlinear mixture

The linear mixture model is assumed in most of the papers devoted to independent component analysis. A more realistic model for mixture should be nonlinear. In this paper, a two layer perceptron is used as a de-mixing system to extract sources in nonlinear mixture. The learning algorithms for the de-mixing system are derived by two approaches: maximum entropy and minimum mutual information. The algorithms derived from the two approaches have a common structure. The new learning equations for the hidden layer are different from our previous learning equations for the output layer. The natural gradient descent method is applied in maximizing entropy and minimizing mutual information. The information (entropy or mutual information) backpropagation method is proposed to derive the learning equations for the hidden layer.