This note presents a (new) basic formula for sample-path-based estimates for performance gradients for Markov systems (called policy gradients in reinforcement learning literature). With this basic formula, many policy-gradient algorithms, including those that have previously appeared in the literature, can be easily developed. The formula follows naturally from a sensitivity equation in perturbation analysis. New research direction is discussed.
No takes yet. Share an insight, caveat, or question.
Xi‐Ren Cao (2005) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: