Answers are grounded in this paper and cite their sources.
This paper studies how to train a reinforcement learning agent for Settlers of Catan, a board game that is difficult to learn because of dice randomness, hidden opponent cards, a large action space, and sparse win/loss rewards at the end of a long game. We apply Proximal Policy Optimization (PPO) and compare three ways of training the same agent: Pure-PPO (learning from scratch with no external help), Expert- Guided PPO (EG-PPO) adding an online reward bonus whenever the agent picks a move favored by a heuristic expert), and BC-PPO (first copying an expert via Behavioral Cloning and then fine-tuning freely with reinforcement learning). Pure-PPO never exceeds a 10% win rate against a strong heuristic baseline regardless of architecture or reward shaping. EGPPO lifts this ceiling to 31.2%, and BC-PPO reaches 69.6%, more than doubling the best reward-shaping result. We also report a counter-intuitive teacher-quality paradox: cloning a stronger multi-ply Alpha-Beta search expert yields a worse final agent than cloning a simpler 1-ply heuristic, because the reactive network cannot observe the search tree that motivated the stronger teacher’s moves.
No takes yet. Share an insight, caveat, or question.
Sorin-Nicolae TURCULET (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: