What type of study is this?

This is a Quantitative Study study.

October 20, 2025Open Access

MDP modeling for multi-stage stochastic programs

Key Points

The modeling approach improves expressiveness in multi-stage stochastic programs and includes decision-dependent uncertainty, enhancing real-world applicability.
New variants of stochastic dual dynamic programming were developed, effectively addressing non-convexities in these complex models.
Examples of increasing complexity showcase the efficacy of incorporating structured Markov decision processes into the modeling framework.
The study signifies a step forward in stochastic optimization, integrating learning techniques into decision-making processes.

Abstract

We study a class of multi-stage stochastic programs, which incorporate modeling features from Markov decision processes (MDPs). This class includes structured MDPs with continuous state and action spaces. We extend policy graphs to include decision-dependent uncertainty for one-step transition probabilities as well as a limited form of statistical learning. We focus on the expressiveness of our modeling approach, illustrating ideas with a series of examples of increasing complexity. As a solution method, we develop new variants of stochastic dual dynamic programming, including approximations to handle non-convexities.

Read Full Paperexternally

Bookmark

View Full Paper

Cite This Study

Morton et al. (Fri,) studied this question.

synapsesocial.com/papers/68f6196ee0bbbc94fac364af https://doi.org/https://doi.org/10.48550/arxiv.2509.22981

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

Bookmark

View Full Paper