An ε-Optimal Control of a Finite Markov Chain with an Average Reward Criterion

Previous article Next article An ε-Optimal Control of a Finite Markov Chain with an Average Reward CriterionE. A. FainbergE. A. Fainberghttps://doi.org/10.1137/1125006PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAbout[1] A. N. Shiryaev, Some new results in the theory of controlled random processesTrans. Fourth Prague Conf. on Information Theory, Statistical Decision Functions, Random Processes (Prague, 1965), Academia, Prague, 1967, 131–203, (In Russian.) 37:1186 Google Scholar[2] O. V. Viskov and , A. N. Shiryaev, On controls which reduce to optimal stationary regimes, Trudy Mat. Inst. Steklov., 71 (1964), 35–45, Akad. Nauk SSSR, (In Russian.) 30:3792 Google Scholar[3] David Blackwell, Discrete dynamic programming, Ann. Math. Statist., 33 (1962), 719–726 26:7449 0133.12906 CrossrefGoogle Scholar[4] Cyrus Derman, On sequential decisions and Markov chains, Management Sci., 9 (1962/1963), 16–24 29:6930 CrossrefGoogle Scholar[5] E. B. Dynkin and , A. A. Yushkevich, Controlled Markov Processes and Their Applications, Nauka, Moscow, 1975, (In Russian.) Google Scholar[6] Cyrus Derman, Finite state Markovian decision processes, Mathematics in Science and Engineering, Vol. 67, Academic Press, New York, 1970xiii+159 42:2588 0262.90001 Google Scholar[7] R. Ya. Chitashvili, A controlled finite Markov chain with an arbitrary set of decisions, Theory Prob. Appl., 20 (1975), 839–847 0371.60084 LinkGoogle Scholar[8] E. A. Fainberg, On controlled finite state Markov processes with compact control sets, Theory Prob. Appl., 20 (1975), 856–862 0372.60093 LinkGoogle Scholar[9] John Bather, Optimal decision procedures for finite Markov chains. I. Examples, Advances in Appl. Probability, 5 (1973), 328–339 51:5028 0262.90063 CrossrefGoogle Scholar[10] Cyrus Derman, Denumerable state Markovian decision processes—average cost criterion, Ann. Math. Statist., 37 (1966), 1545–1553 34:6954 0144.43102 CrossrefGoogle Scholar[11] Ralph E. Strauch, Negative dynamic programming, Ann. Math. Statist., 37 (1966), 871–890 33:2456 0144.43201 CrossrefGoogle Scholar[12] I. I. Gikhman and , A. V. Skorohod, Controlled stochastic processes, Springer-Verlag, New York, 1979vii+237 80h:60081 CrossrefGoogle Scholar[13] L. G. Gubenko and , E. S. Shtatland, On discrete time Markov decision processes, Teor. Verojatnost. i Mat. Statist., (1972), 51–64, 163, in Probability Theory and Mathematical Statistics, Kiev, (In Russian.) 48:13275 Google Scholar[14] E. A. Fainberg, Finite controllable Markov chains, Uspehi Mat. Nauk, 32 (1977), 181–182, (In Russian.) 58:15198 Google Scholar[15] Sheldon M. Ross, Arbitrary state Markovian decision processes, Ann. Math. Statist., 39 (1968), 2118–2122 39:3644 0179.24704 CrossrefGoogle Scholar[16] A. Hordijk, Dynamic programming and Markov potential theory, Mathematisch Centrum, Amsterdam, 1974vi+134 pp. (loose addendum) 55:5216 0284.49012 Google Scholar[17] John Bather, Optimal decision procedures for finite Markov chains. II. Communicating systems, Advances in Appl. Probability, 5 (1973), 521–540 54:14791 0275.90049 CrossrefGoogle Scholar[18] Sheldon M. Ross, On the nonexistence of ε-optimal randomized stationary policies in average cost Markov decision models, Ann. Math. Statist., 42 (1971), 1767–1768 50:12273 0238.62094 CrossrefGoogle Scholar[19] J. G. Kemeny and , J. L. Snell, Finite Markov chains, Van Nostrand, New York, 1965 Google Scholar[20] Ashok Maitra, A note on undiscounted dynamic programming, Ann. Math. Statist., 37 (1966), 1042–1044 33:7167 0146.41501 CrossrefGoogle Scholar[21] Ronald A. Howard, Dynamic programming and Markov processes, The Technology Press of M.I.T., Cambridge, Mass., 1960viii+136 22:9287 0091.16001 Google Scholar Previous article Next article FiguresRelatedReferencesCited ByDetails LP based upper and lower bounds for Cesàro and Abel limits of the optimal values in problems of control of stochastic discrete time systemsJournal of Mathematical Analysis and Applications, Vol. 512, No. 1 | 1 Aug 2022 Cross Ref On Structural Properties of Optimal Average Cost functions in Markov Decision Processes with Borel Spaces and Universally Measurable PoliciesJournal of Mathematical Analysis and Applications | 1 Dec 2021 Cross Ref A Characterization of the Optimal Certainty Equivalent of the Average Cost via the Arrow-Pratt Sensitivity FunctionMathematics of Operations Research, Vol. 41, No. 1 | 1 Feb 2016 Cross Ref Examples concerning Abel and Cesàro limitsJournal of Mathematical Analysis and Applications, Vol. 420, No. 2 | 1 Dec 2014 Cross Ref Average Cost Markov Decision Processes with Weakly Continuous Transition ProbabilitiesMathematics of Operations Research, Vol. 37, No. 4 | 1 Nov 2012 Cross Ref On Near Optimality of the Set of Finite-State Controllers for Average Cost POMDPMathematics of Operations Research, Vol. 33, No. 1 | 1 Feb 2008 Cross Ref IntroductionHandbook of Markov Decision Processes | 1 Jan 2003 Cross Ref Average Reward Optimization Theory for Denumerable State SpacesHandbook of Markov Decision Processes | 1 Jan 2002 Cross Ref Finite state Markov decision models with average reward criteriaStochastic Processes and their Applications, Vol. 49, No. 1 | 1 Jan 1994 Cross Ref Discrete-Time Controlled Markov Processes with Average Cost Criterion: A SurveySIAM Journal on Control and Optimization, Vol. 31, No. 2 | 14 July 2006AbstractPDF (7400 KB)An expected average reward criterionStochastic Processes and their Applications, Vol. 26 | 1 Jan 1987 Cross Ref An Average Reward Criterion by MandlDGOR | 1 Jan 1987 Cross Ref Controlled Markov Processes with Arbitrary Numerical CriteriaTheory of Probability & Its Applications, Vol. 27, No. 3 | 17 July 2006AbstractPDF (1950 KB) Volume 25, Issue 1| 1980Theory of Probability & Its Applications1-219 History Submitted:24 August 1978Published online:28 July 2006 InformationCopyright © 1980 © Society for Industrial and Applied MathematicsPDF Download Article & Publication DataArticle DOI:10.1137/1125006Article page range:pp. 70-81ISSN (print):0040-585XISSN (online):1095-7219Publisher:Society for Industrial and Applied Mathematics