Intrinsic Markov chains

Introduction. The object of this paper is to analyse the structure of stochastic processes with finitely many states which behave qualitatively like Markov chains, in that the 'possible' sequences of the processes are determined by a chain rule. Such processes are called intrinsically Markovian. In §1 we establish some necessary and sufficient conditions in order that a stochastic process be intrinsically Markovian. In §2 we investigate an equivalence relation compatibility (weaker than probability equivalence-equivalence) between stochastic processes with finitely many states. Within the compatibility class of an intrinsic Markov chain there are stationary Markov chains and processes which we term piecewise linear. These latter processes are in turn equivalent to stationary Markov chains. In §3 we define the absolute entropy of a stochastic process with finitely many states and show that this is an invariant of compatibility which dominates all the (probability) entropies of stationary processes within the compatibility class of an intrinsic Markov chain. However, there is a unique stationary probability whose entropy is equal to the absolute entropy. This probability makes the process a Markov chain. Moreover, this Markov chain is equivalent to a process which is not only piecewise linear, but uniformly piecewise linear. This result leads to the conclusion that for every positive number between zero and the absolute entropy, there is a compatible stationary Markov chain (equivalent to a piecewise linear process) with this number as its entropy. We also outline a simple procedure for determining the absolute entropy and the chain which has this maximal entropy. An incidental result states that a process which behaves information theoretically like a Markov chain must be a Markov chain. These results appear in §3 and §4. Definitions. 1. A nonatomic stochastic process with a finite number of states (n. p. f. ) is a system (X, 3t, m, T) where: (i) For some integer s = 2, Xcz x = x0, x1, ---: xie (0A, --, s-l).