Entropy has found significant applications in numerous signal processing problems including independent components analysis and blind deconvolution. In general, entropy estimators require O(N/sup 2/) operations, N being the number of samples. For practical online entropy manipulation, it is desirable to determine a stochastic gradient for entropy, which has O(N) complexity. In this paper, we propose a stochastic Shannon's entropy estimator. We determine the corresponding stochastic gradient and investigate its performance. The proposed stochastic gradient for Shannon's entropy can be used in online adaptation problems where the optimization of an entropy-based cost function is necessary.
No takes yet. Share an insight, caveat, or question.
Erdoğmuş et al. (2003) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: