What question did this study set out to answer?

This work aims to define and measure uncertainty accumulation along computational decision trajectories in neural networks.

May 22, 2026Open Access

ENTRO-PATH: A Quantitative Information-Theoretic Framework for Entropy Propagation Across Computational Decision Trajectories

Key Points

This work aims to define and measure uncertainty accumulation along computational decision trajectories in neural networks.
Defined irreducible path entropy (H_path) and local path entropy using Shannon entropy at each layer.
Developed observability index and entropic leakage measurements across various neural network architectures.
Applied k-nearest-neighbour entropy estimation and mutual information probing for numerical examples.
Demonstrated the capacity to measure uncertainty transformation throughout neural network layers.
Provided metrics that can summarize inferential transparency for feedforward, convolutional, and attention-based architectures.
Illustrated the functionality of the proposed framework through examples involving MLP, CNN, and Transformer configurations.

Abstract

This work introduces Irreducible Path Entropy (Hₚath), a formally defined, layer-integrated information-theoretic metric quantifying how much uncertainty accumulates, transforms, and becomes unrecoverable along the inference trajectory of an artificial neural network. Local path entropy is defined as the Shannon entropy of the conditional activation distribution at each layer. Cumulative path entropy sums these quantities across all layers. Reducibility conditions classify each layer relative to a tolerance threshold δ*. The composite Observability Index Ω ∈ 0, 1 summarises inferential transparency across the full network. Entropic leakage Δ (L) measures uncertainty introduced across computation not explained by retained input information. The framework is grounded exclusively in information theory and systems-level analysis, without semantic, cognitive, or anthropomorphic assumptions, and is applicable to feedforward, convolutional, and attention-based architectures. Numerical illustrations are provided for MLP (2L, 6L, 12L), CNN (8L), and Transformer (12L, 24L) configurations. All constructs are experimentally estimable via k-nearest-neighbour entropy estimation and mutual information probing without access to full weight matrices. EntropyLab Independent Research Series · Ronin Institute / Rite of Renaissance · May 2026OSF Preregistration: https: //osf. io/7wp9hLanding page: https: //entropath. netlify. app/Internet Archive: https: //archive. org/details/osf-registrations-7wp9h-v1

Read Full Paperexternally

Ask AI

Helpful

Bookmark

View Full Paper