Accelerated controller tuning using human feedback and Multi-Task Preferential Bayesian Optimization

Controller design often requires balancing multiple conflicting objectives, making it hard to define a single objective function or trade-offs between multiple objectives to achieve the desired performance. Users often find it easier to express preferences through pairwise comparisons, such as ’I prefer A over B’. Preferential Bayesian Optimization (PBO) uses this feedback to optimize a utility function that reflects the user’s preferences towards different criteria. However, when a new task is presented, PBO typically ignores previous comparison data, learning the preferences for new tasks from scratch. We introduce Multi-Task PBO (MTPBO), which leverages preferential data from related tasks to speed up optimization for new tasks. Applied to both synthetic case studies and a humanin-the-loop controller tuning problem, MTPBO demonstrates superior performance, finding more preferred responses with a lower experimental budget.