Adaptive decision-making in non-stationary multi-armed bandit problems remains a central challenge. Traditional strategies such as Upper Confidence Bound (UCB) and Thompson Sampling (TS) often underperform when reward structures shift, as they rely on fixed decision rules and fail to incorporate distributional feedback. This study introduces Adaptive Regret-Matched Fusion UCB-TS (ARF-UCB-TS), a novel dynamic strategy fusion algorithm designed to overcome these limitations by integrating exploration and exploitation mechanisms adaptively. The proposed method employs a signal-based weighting mechanism to fuse UCB and TS scores, enhanced by a nonlinear temperature control function and momentum-aware adjustment for smooth responsiveness. The algorithm dynamically adjusts its reliance on UCB or TS based on evolving reward signals, enabling it to adapt to abrupt or gradual changes in reward distributions. The framework is evaluated under four representative scenarios: stationary rewards, incorrect prior initialization, mid-term mutation, and periodic reward fluctuation. Across all settings, ARF-UCB-TS demonstrates superior performance in minimizing cumulative regret and achieving stable reward rates. These findings underscore its robustness and flexibility in complex decision environments.
Xinwei Deng (Wed,) studied this question.