June 11, 2024Open Access

Integrating Domain Knowledge for handling Limited Data in Offline RL

Puntos clave

Performance increases by at least 27% using a new regularization technique in offline reinforcement learning, with a clear edge in constrained data situations.
Key evidence shows that offline RL algorithms struggle in regions with limited data, where few observations occur, leading to suboptimal actions.
The proposed method employs a domain knowledge-based regularization technique to refine actions for rare observations in the state space effectively and efficiently in evaluations across various datasets. Though performance improves significantly, further exploration may be needed to validate these findings across diverse scenarios beyond the tested standard datasets.

Resumen

With the ability to learn from static datasets, Offline Reinforcement Learning (RL) emerges as a compelling avenue for real-world applications. However, state-of-the-art offline RL algorithms perform sub-optimally when confronted with limited data confined to specific regions within the state space. The performance degradation is attributed to the inability of offline RL algorithms to learn appropriate actions for rare or unseen observations. This paper proposes a novel domain knowledge-based regularization technique and adaptively refines the initial domain knowledge to considerably boost performance in limited data with partially omitted states. The key insight is that the regularization term mitigates erroneous actions for sparse samples and unobserved states covered by domain knowledge. Empirical evaluations on standard discrete environment datasets demonstrate a substantial average performance increase of at least 27% compared to existing offline RL algorithms operating on limited data.

Leer artículo completoexternamente

Me gusta

Guardar

Ver artículo completo