Putting the Human Back in the Loop: A Review of Interactive Bayesian Optimization

Problems in science and engineering are often cast as optimization tasks, where the parameters of some system must be decided in order to maximize or minimize a given objective or set of objectives. Bayesian optimization offers a flexible tool for this task, thanks to its sample-efficiency and its ability to handle opaque-box functions. This paper reviews research on human-in-the-loop Bayesian optimization , a natural development of Bayesian optimization that is rapidly gaining traction. While researchers have looked into ways to include humans in the Bayesian optimization process, there is little understanding of how this is best done and what kinds of benefits are achievable. We review the growing body of literature in this space and identify five main areas of interaction with different properties and advantages: (1) initializing; (2) evaluating explicitly; (3) evaluating implicitly; (4) steering; and (5) reviewing. The review suggests that appropriate interaction techniques can increase the efficacy and efficiency of a system and promote user engagement. We synthesize practical guidance to design human involvement via such techniques. We conclude that the scope and usefulness of Bayesian optimization can be greatly extended by involving humans in the optimization process.