In complex task environments in both nature and human society, structuralviolations of expectation (VoE) occur with non-trivial frequency. Agents that are designed to operate in such environments must be capable of open-world learning (OWL), defined as the ability to detect and accommodate out-of-distribution inputs, as well as more complex structural VoEs, without requiring extensive and offline re-training. Until recently, OWL research was relatively constrained and limited to areas such as anomaly detection and concept drift. More recently, agent-based OWL research has witnessed much interest from across the community. To support this research, not just for developing OWL algorithms, but also evaluating them, there is a need for multi-agent environments where structural VoEs can be generated, and controlled experiments can be run with relative ease. To address this need, we propose a resource called PokerOWL, a platform that is supported on the Gymnasium infrastructure, which is extensively used in the reinforcement learning and AI game-playing communities. PokerOWL supports both a rich VoE generator and a graphical interface for facilitating development and evaluation of OWL methods. Using an extensive set of experiments and a Poker-playing agent based on Deep Q-Networks, we use PokerOWL to demonstrate how even state-of-the-art agents can struggle to generalize to novel situations without additional OWL capabilities.
Chiu et al. (Sun,) studied this question.