What question did this study set out to answer?

This research aims to understand how convolutional neural networks handle information during learning in image classification tasks.

January 21, 2026EntropyOpen Access

Uncovering Neural Learning Dynamics Through Latent Mutual Information

View Full Paper

Ask AI

Bookmark

This research aims to understand how convolutional neural networks handle information during learning in image classification tasks.
Track mutual information between inputs, intermediate representations, and labels in various neural network architectures.
Analyze changes in label-relevant and input mutual information across different layers.
Implement inference-time knockouts, shuffles, and perturbations to evaluate functional necessity of high-MI channels.
Introduce a dependence-aware regularizer based on the Hilbert–Schmidt Independence Criterion.
Label-relevant mutual information increases with layer depth across different architectures.
Input mutual information varies significantly depending on architecture and activation types.
High-MI channels show functional necessity for accuracy in information processing.
The proposed regularizer leads to small accuracy improvements and faster convergence during training.

No takes yet. Share an insight, caveat, or question.

Uncovers how convolutional neural networks reorganize information, suggesting new training strategies.

Issitt et al. (2026) studied this question.

View Full Paper

Ask AI

Bookmark