A Framework for Controllable Multi-objective Learning with Annealed Stein Variational Hypernetworks

Pareto Set Learning (PSL) is an efficient approach for approximating the complete set of optimal solutions in Multi-objective Learning (MOL). By learning a set of solutions that map to a dense Pareto front in objective space, PSL aims to recover the underlying Pareto set. However, most existing methods focus heavily on convergence toward optimality, often neglecting solution diversity. To explicitly balance convergence and diversity, we propose Stein Variational Hypernetwork for MOL (SVH-MOL), a novel framework that incorporates Stein variational updates into Pareto set learning. SVH-MOL updates solutions via two complementary components: (i) a driving term that guides particles toward the Pareto set, and (ii) a repulsive term that encourages diversity among solutions. These two terms inherently compete, making stable and effective learning challenging. To address this issue, we introduce an annealing schedule that adaptively controls the relative strength of each term during training. Extensive experiments on synthetic multi-objective benchmarks and real-world multi-task learning problems demonstrate that SVH-MOL achieves a better trade-off between convergence and diversity than existing methods. Our implementation is released at https://github.com/nguyenduc810/SVH-MOL-official .