Transfer Learning in Multi-Armed Bandit: A Causal Approach

We leverage causal inference tools to support a principled and more robust transfer of knowledge in reinforcement learning (RL) settings. In particular, we tackle the problem of transferring knowledge across bandit agents in settings where causal effects cannot be identified by Pearl's {do-calculus} nor standard off-policy learning techniques. Our new identification strategy combines two steps -- first, deriving bounds over the arm's distribution based on structural knowledge; second, incorporating these bounds in a novel bandit algorithm, B-kl-UCB. Simulations demonstrate that our strategy is consistently more efficient than the current (non-causal) state-of-the-art methods.