What question did this study set out to answer?

The aim is to explore off-policy evaluation methods in reinforcement learning and analyze their efficiency and statistical properties.

August 3, 2026Statistical Science13 citationsOpen Access

A Review of Off-Policy Evaluation in Reinforcement Learning

MUMasatoshi Uehara CSChengchun Shi NKNathan Kallus

Key Points

The aim is to explore off-policy evaluation methods in reinforcement learning and analyze their efficiency and statistical properties.
Reviewed various off-policy evaluation methods in the literature.
Discussed the efficiency bounds and statistical properties of these methods.
Highlighted emerging research directions in off-policy evaluation.
Outlined the state-of-the-art methods in off-policy evaluation.
Described the efficiency bounds related to these evaluation methods.
Identified ongoing research avenues linked to off-policy evaluation.

Abstract

Reinforcement learning (RL) is one of the most vibrant research frontiers in machine learning and has been recently applied to solve a number of challenging problems. In this paper, we primarily focus on off-policy evaluation (OPE), one of the most fundamental topics in RL. In recent years, a number of OPE methods have been developed in the statistics and computer science literature. We provide a discussion on the efficiency bound of OPE, some of the existing state-of-the-art OPE methods, their statistical properties and some other related research directions that are currently actively explored.

问 AI

Bookmark

View Full Paper