The natural gradient descent method is applied to train an n-m-1 multilayer perceptron. Based on an efficient scheme to represent the Fisher information matrix for an n-m-1 stochastic multilayer perceptron, a new algorithm is proposed to calculate the natural gradient without inverting the Fisher information matrix explicitly. When the input dimension n is much larger than the number of hidden neurons m, the time complexity of computing the natural gradient is O(n).
No takes yet. Share an insight, caveat, or question.
Yang et al. (1998) studied this question.
Synapse has enriched 3 closely related papers on similar clinical questions. Consider them for comparative context: