Answers are grounded in this paper and cite their sources.
Let X = (Xₜ, Fₜ, Pˣ)ₜ₀ be a Markov process where (Xₜ; t 0) is the trajectory or sample path, Fₜ is the definitive σ-algebra of events generated by (Xₛ; 0 s t), and Pˣ is the probability distribution on sample paths corresponding to an initial state x. The state space is taken as the semi-compact (E, C) where E is a locally compact separable metric space with family of open sets C. A non-negative extended real valued random variable T such that for each t 0, t\ ε Fₜ is called a Markov time or stopping time. This paper studies the problem of choosing a stopping time T which, for a fixed λ 0, maximizes one of the following criteria:{equation*}{1} Θ_T(x) = E^xe-λ T g(X_T);{equation*}{equation*}{2} λ_T(x) = E^x e-λ T g(X_T) - ∫^T_o e-λ s c(X_s) ds, where E^xT < ∞;or{equation*}{equation*}{3} Φ_T(x) = E^x g(X_T) - ∫^T_0 c(X_s) ds/E^xT, where 0 < E^xT < ∞;{equation*} where g and c are non-negative continuous functions defined on the state space of the process. Dynkin [9] studied criterion (1) where λ = 0 under the general assumption that X is a standard process with a possibly random lifetime and under very weak continuity assumptions concerning the return function g. He showed that criterion (2) can often be transformed into criterion (1), and thus his approach is applicable in this case as well. This paper studies optimal stopping in a Markov process having a Feller transition function, a special case in Dynkin's development. We further specialize to exponentially distributed lifetimes which causes the appearance of a discount factor e-λ t, with the natural interpretation that a dollar transaction t time units hence has a present value of e-λ t. Criterion (3) often has the meaning of a long-run time average return and a means of transforming this criterion into criterion (2) is given. Finally, some techniques for implementing Dynkin's approach in a variety of commonly occurring situations are given along with examples of their use.
No takes yet. Share an insight, caveat, or question.
Howard M. Taylor (1968) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: