What type of study is this?

This is a Quantitative Study study.

October 10, 2025Open Access

Trajectory Data Suffices for Statistically Efficient Policy Evaluation in Finite-Horizon Offline RL with Linear q^π-Realizability and Concentrability

Key Points

Statistically efficient policy evaluation is achievable using trajectory data in offline reinforcement learning.
The approach builds on the concepts of concentrability and q^π-realizability to enhance learning efficiency.
Improved sample complexity ensures that fewer data points may be needed for accurate policy evaluations.
This work advances prior research by optimizing existing algorithms for better performance in finite-horizon settings.

Abstract

We study finite-horizon offline reinforcement learning (RL) with function approximation for both policy evaluation and policy optimization. Prior work established that statistically efficient learning is impossible for either of these problems when the only assumptions are that the data has good coverage (concentrability) and the state-action value function of every policy is linearly realizable (q^π-realizability) (Foster et al. , 2021). Recently, Tkachuk et al. (2024) gave a statistically efficient learner for policy optimization, if in addition the data is assumed to be given as trajectories. In this work we present a statistically efficient learner for policy evaluation under the same assumptions. Further, we show that the sample complexity of the learner used by Tkachuk et al. (2024) for policy optimization can be improved by a tighter analysis.

Read Full Paperexternally

Ask AI

Helpful

Bookmark

View Full Paper