Semi-Markov and Jump Markov Controlled Models: Average Cost Criterion

Previous article Next article Semi-Markov and Jump Markov Controlled Models: Average Cost CriterionM. Yu. KitayevM. Yu. Kitayevhttps://doi.org/10.1137/1130036PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAbout[1] E. B. Dynkin and , A. A. Yushkevich, Controlled Markov processes, Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], Vol. 235, Springer-Verlag, Berlin, 1979xvii+289 80k:90037 0426.60063 CrossrefGoogle Scholar[2] Cyrus Derman, Denumerable state Markovian decision processes—average cost criterion, Ann. Math. Statist., 37 (1966), 1545–1553 34:6954 0144.43102 CrossrefGoogle Scholar[3] Sheldon Ross, Average cost semi-Markov decision processes, J. Appl. Probability, 7 (1970), 649–656 46:2771 0204.51704 CrossrefGoogle Scholar[4] L. G. Gubenko and , E. S. Shtatland, Controlled and semi-Markov models and some concrete problems of optimization of stochastic systemsthe collection: Controlled Random Processes and Systems, Izd-vo IK AN USSR, Kiev, 1973, 87–119, (In Russian.) Google Scholar[5] Steven Lippman, Semi-Markov decision processes with unbounded rewards, Management Sci., 19 (1972/73), 717–731 49:2109 0259.60044 CrossrefGoogle Scholar[6] John Bather, Optimal stationary policies for denumerable Markov chains in continuous time, Advances in Appl. Probability, 8 (1976), 144–158 52:16650 0342.60045 CrossrefGoogle Scholar[7] Derek Robinson, Markov decision chains with unbounded costs and applications to the control of queues, Advances in Appl. Probability, 8 (1976), 159–176 52:16655 0343.60044 CrossrefGoogle Scholar[8] A. A. Yushkevich, On semi-Markov controlled models with an average reward criterion, Theory Prob. Applications, 26 (1981), 796–803 0499.60094 LinkGoogle Scholar[9] A. A. Yushkevich and , E. A. Faynberg, On homogeneous controlled Markov models with continuous time and finite or countable state set, Theory Prob. Applications, 24 (1980), 156–161 0437.93033 LinkGoogle Scholar[10] Harold Kushner, Introduction to stochastic control, Holt, Rinehart and Winston, Inc., New York, 1971xvii+390 43:5969 0293.93018 Google Scholar[11] Arie Hordijk, A sufficient condition for the existence of an optimal policy with respect to the average cost criterion in Markovian decision processesTransactions of the Sixth Prague Conference on Information Theory, Statistical Decision Functions, Random Processes (Tech. Univ. Prague, Prague, 1971; dedicated to the memory of Antonín Špaček), Academia, Prague, 1973, 263–274 51:5030 0291.90072 Google Scholar[12] A. Federgruen, , A. Hordijk and , H. C. Tijms, Denumerable state semi-Markov decision processes with unbounded costs, average cost criterion, Stochastic Process. Appl., 9 (1979), 223–235 10.1016/0304-4149(79)90034-6 81b:90150 0422.90084 CrossrefGoogle Scholar[13] M. Yu. Kitayev, The existence of optimal homogeneous strategies in controllable semicontinuous semi-Markov models with respect to the average cost criterion, Theory Prob. Applications, 26 (1981), 614–616 LinkGoogle Scholar[14] A. A. Yushkevich, Controlled jump Markov models, Theory Prob. Applications, 25 (1980), 244–266 0458.90078 LinkGoogle Scholar[15] K. R. Parthasarathy, Probability measures on metric spaces, Probability and Mathematical Statistics, No. 3, Academic Press Inc., New York, 1967xi+276 37:2271 0153.19101 CrossrefGoogle Scholar[16] Jean Jacod, Multivariate point processes: predictable projection, Radon-Nikodym derivatives, representation of martingales, Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 31 (1974/75), 235–253 52:1875 0302.60032 CrossrefGoogle Scholar[17] D. Revuz, Markov chains, North-Holland Publishing Co., Amsterdam, 1975x+336 54:3852 0332.60045 Google Scholar[18] D. Blackwell, Positive dynamic programming, Proc. Fifth Berkeley Symp. Math. Statistics and Probab. Vol. I, Univ. of Calif. Press, Berkeley, 1964, 415–418 0189.19804 Google Scholar[19] V. V. Rykov, Controlled mass service systemsItogi Nauki i Tekhniki; Teoriya Veroyatnostey, Matematicheskaya Statistika, Teoreticheskaya Kibernetika, Vol. 12, VINITI, Moscow, 1975, 43–153, (In Russian.) Google Scholar[20] M. A. Faynberg and , E. A. Gaynberg, Control in mass service systems, Zarubezhnaya Radioelektronika, 3 (1975), 3–33, (In Russian.) Google Scholar[21] B. G. Pittel', Optimal control in a mass service system with several flows of demands, Izv. Akad. Nauk SSSR Tehn. Kibernet., (1972), 101–116, (In Russian.) 54:8923 Google Scholar[22] G. P. Klimov, Time-sharing service system. I, Theory Prob. Applications, 19 (1974), 532–551 0378.60102 LinkGoogle Scholar[23] D. Heyman, Optimal operating policies for $M/G/1$ queuing systems, Oper. Res., 16 (1968), 362–382 0164.47704 CrossrefGoogle Scholar Previous article Next article FiguresRelatedReferencesCited byDetails Nonzero-Sum Risk-Sensitive Continuous-Time Stochastic Games with Ergodic Costs7 June 2022 | Applied Mathematics & Optimization, Vol. 86, No. 1 Cross Ref Optimal Stopping Time on Semi-Markov Processes with Finite Horizon21 April 2022 | Journal of Optimization Theory and Applications, Vol. 194, No. 2 Cross Ref Continuous-Time Zero-Sum Games for Markov Decision Processes with Discounted Risk-Sensitive Cost Criterion28 May 2021 | Dynamic Games and Applications, Vol. 12, No. 2 Cross Ref Sufficiency of Markov Policies for Continuous-Time Jump Markov Decision ProcessesMathematics of Operations Research, Vol. 47, No. 2 Cross Ref Risk-sensitive discounted cost criterion for continuous-time Markov decision processes on a general state space11 April 2022 | Mathematical Methods of Operations Research, Vol. 95, No. 2 Cross Ref Ergodic risk-sensitive control of Markov processes on countable state space revisited1 March 2022 | ESAIM: Control, Optimisation and Calculus of Variations Cross Ref Kolmogorov's Equations for Jump Markov Processes and their Applications to Control ProblemsE. A. Feinberg and A. N. Shiryaev3 February 2022 | Theory of Probability & Its Applications, Vol. 66, No. 4AbstractPDF (548 KB)Continuous-time zero-sum games for markov decision processes with discounted risk-sensitive cost criterion on a general state space21 December 2021 | Stochastic Analysis and Applications, Vol. 76 Cross Ref Optimal stopping time on discounted semi-Markov processes19 April 2021 | Frontiers of Mathematics in China, Vol. 90 Cross Ref Continuous-time zero-sum games for Markov chains with risk-sensitive finite-horizon cost criterion5 March 2021 | Stochastic Analysis and Applications Cross Ref Уравнения Колмогорова для скачкообразных марковских процессов и их применения в задачах управления22 October 2021 | Теория вероятностей и ее применения, Vol. 66, No. 4 Cross Ref Finite horizon risk-sensitive continuous-time Markov decision processes with unbounded transition and cost rates10 January 2019 | 4OR, Vol. 17, No. 4 Cross Ref Risk-Sensitive Discounted Continuous-Time Markov Decision Processes with Unbounded RatesXianping Guo and Zhong-Wei Liao26 November 2019 | SIAM Journal on Control and Optimization, Vol. 57, No. 6AbstractPDF (504 KB)Finite horizon continuous-time Markov decision processes with mean and variance criteria29 September 2018 | Discrete Event Dynamic Systems, Vol. 28, No. 4 Cross Ref Continuous-Time Constrained Stochastic Games under the Discounted Cost Criteria24 August 2016 | Applied Mathematics & Optimization, Vol. 77, No. 2 Cross Ref Realizable Strategies in Continuous-Time Markov Decision ProcessesAlexey Piunovskiy20 February 2018 | SIAM Journal on Control and Optimization, Vol. 56, No. 1AbstractPDF (561 KB)Risk-aware semi-Markov decision processes Cross Ref Continuous-Time Markov Decision Processes with Exponential UtilityYi Zhang30 August 2017 | SIAM Journal on Control and Optimization, Vol. 55, No. 4AbstractPDF (369 KB)Structures of Optimal Policies in MDPs with Unbounded Jumps: The State of Our Art11 March 2017 Cross Ref Optimality of Mixed Policies for Average Continuous-Time Markov Decision Processes with ConstraintsMathematics of Operations Research, Vol. 41, No. 4 Cross Ref Stochastic Games for Continuous-Time Jump Processes Under Finite-Horizon Payoff Criterion18 September 2015 | Applied Mathematics & Optimization, Vol. 74, No. 2 Cross Ref Average Optimality for Continuous-Time Markov Decision Processes Under Weak Continuity Conditions4 February 2016 | Journal of Applied Probability, Vol. 51, No. 04 Cross Ref Average Optimality for Continuous-Time Markov Decision Processes Under Weak Continuity Conditions30 January 2018 | Journal of Applied Probability, Vol. 51, No. 4 Cross Ref Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach31 March 2013 | 4OR, Vol. 12, No. 1 Cross Ref On solutions of Kolmogorovʼs equations for nonhomogeneous jump Markov processesJournal of Mathematical Analysis and Applications, Vol. 411, No. 1 Cross Ref First Passage Optimality for Continuous-Time Markov Decision Processes With Varying Discount Factors and History-Dependent PoliciesIEEE Transactions on Automatic Control, Vol. 59, No. 1 Cross Ref Sufficiency of Markov policies for continuous-time Markov decision processes and solutions to Kolmogorov's forward equation for jump Markov processes Cross Ref Absorbing Continuous-Time Markov Decision Processes with Total Cost Criteria22 February 2016 | Advances in Applied Probability, Vol. 45, No. 02 Cross Ref Absorbing Continuous-Time Markov Decision Processes with Total Cost Criteria22 February 2016 | Advances in Applied Probability, Vol. 45, No. 2 Cross Ref The Transformation Method for Continuous-Time Markov Decision Processes2 March 2012 | Journal of Optimization Theory and Applications, Vol. 154, No. 2 Cross Ref Linear Programming and Constrained Average Optimality for General Continuous-Time Markov Decision Processes in History-Dependent PoliciesXianping Guo, Yonghui Huang, and Xinyuan Song3 January 2012 | SIAM Journal on Control and Optimization, Vol. 50, No. 1AbstractPDF (300 KB)Reduction of Discounted Continuous-Time MDPs with Unbounded Jump and Reward Rates to Discrete-Time Total-Reward MDPs12 July 2012 Cross Ref Discounted continuous-time constrained Markov decision processes in Polish spacesThe Annals of Applied Probability, Vol. 21, No. 5 Cross Ref Accuracy of fluid approximations to controlled birth-and-death processes: absorbing case23 December 2010 | Mathematical Methods of Operations Research, Vol. 73, No. 2 Cross Ref Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss RatesMathematics of Operations Research, Vol. 36, No. 1 Cross Ref Discounted Continuous-Time Markov Decision Processes with Unbounded Rates: The Convex Analytic ApproachAlexey Piunovskiy and Yi Zhang4 October 2011 | SIAM Journal on Control and Optimization, Vol. 49, No. 5AbstractPDF (348 KB)A survey of recent results on continuous-time Markov decision processesTOP, Vol. 14, No. 2 Cross Ref On essential information in sequential decision processes15 November 2005 | Mathematical Methods of Operations Research, Vol. 62, No. 3 Cross Ref Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event ApproachMathematics of Operations Research, Vol. 29, No. 3 Cross Ref A Controlled Jump Discounted Model with ConstraintsA. B. Piunovskii17 February 2012 | Theory of Probability & Its Applications, Vol. 42, No. 1AbstractPDF (214 KB)New approach to optimization of discounted stochastic continuous-time discrete-event systems Cross Ref Optimal control of average reward constrained continuous-time finite Markov decision processes Cross Ref On State Space Reduction in Sequential Decision Processes Cross Ref Volume 30, Issue 2| 1986Theory of Probability & Its Applications History Submitted:19 January 1982Published online:17 July 2006 InformationCopyright © 1986 Society for Industrial and Applied MathematicsPDF Download Article & Publication DataArticle DOI:10.1137/1130036Article page range:pp. 272-288ISSN (print):0040-585XISSN (online):1095-7219Publisher:Society for Industrial and Applied Mathematics