Noncooperative Stochastic Games

We introduce a sequential competitive decision process that is a generalization of noncooperative finite games and of two-person zero-sum stochastic games (hence, of Markovian decision processes). We prove the existence of equilibrium points under criteria of discounted gain and of average gain. Two person zero-sum stochastic games and noncooperative finite games were introduced in elegant papers by Shapley 22 and Nash 16, 17. Shapley's work prompted a series of papers 1, 4, 5, 10, 11, 12, 14, 18, 26 concerned with the existence of minimax solutions and algorithms for their computation. Even for the two-person zero-sum case, no finite algorithm yet exists. Nash's papers led to a sizeable literature in both mathematics and economics. Mills' 15 work, for example, is related to our characterization of equilibrium points in Section 4. Noncooperative stochastic games may yield fruitful models for several phenomena in the social sciences. Theories of economic markets, for example, have increasingly sought to encompass sequential economic decision processes. Some recent research in social psychology has taken an analogous direction 19, 25. I became aware of recent work by Rogers 20 shortly after completing this paper. His results and ours nearly coincide with our Theorem 2 being slightly stronger than the comparable results in his paper. The basic difference between the papers is that Rogers relies on the Kakutani fixed point theorem whereas we use Brouwer's theorem. Our arguments are somewhat simpler as a consequence.