Entropy of the Mixture of Sources and Entropy Dimension

Suppose that we are given two sourcesS₁,S₂which both send us information from the data spaceX. We assume that we lossy-code information coming fromS₁andS₂with the same maximal error but with different alphabetsP₁andP₂, respectively. Consider a new sourceSwhich sends a signal produced by sourceS₁with probabilitya₁and by sourceS₂with probabilitya₂=1-a₁. We provide a simple greedy algorithm which constructs a coding alphabetPwhich encodes data fromSwith the same value of maximal error as single sources, such that the entropyh(S; P)satisfies:h(S; P)≤ a₁h(S₁; P₁)+a₂h(S₂; P₂)+1.In the proof of the aforementioned formula, the basic role is played by a new equivalent definition of entropy based on measures instead of partitions. As a consequence, we decompose the entropy dimension of the mixture of sources in terms of the convex combination of the entropy dimensions of the single sources. In the case of probability measures inN, this allows us to link the upper local dimension at point with the upper entropy dimension of a measure by an improved version of Young estimation.