On-line Adaptation for Self-Adaptive Systems

Modern software systems such as web services deal with dynamic change and uncertainty in the environments in which they operate. Manual or fixed configuration approaches struggle to provide better performance factors as workloads, reward structures, and even essential system requirements change unpredictably. Self-adaptive systems (SASs) offer a promising solution by allowing software to autonomously adjust its behavior or architecture during run-time in response to environmental changes and internal feedback. This thesis addresses the essential need for intelligent, self-adaptive systems capable of learning, optimising, and restructuring themselves dynamically in real-time. In this direction we construct a collection of modern, advanced Bayesian and bandit-based models bridging theory and practice in adaptive decision-making. First, we introduce Dynamic Bayesian Optimisation for Multi-Armed Bandits Algorithms (DBO-MAB), enabling single agent systems to automatically tune their exploration-exploitation balances and adapt hyperparameters using efficient, incremental Bayesian learning. Through dynamic range adjustment and probabilistic search, DBO-MAB consistently outperforms classical algorithms across diverse and shifting environments. Building on this foundation, we propose the DAMAS (Dynamic Adaptation through Multi-Agent Systems) framework, which empowers a group of cooperative agents to quickly specialize and coordinate in non-stationary environment settings. DAMAS is based on Bayesian inference to select among predefined of agents, each being specific to a particular operational environment, resulting in fast adaptation in real-world benchmarks. Finally, to more generalized agent sets for scalability, we present the Likelihood-Adaptive Multi-Agent System (LA-MAS), which autonomously spawns new agents in response to detected changes in the environment, weights them by statistical likelihood, and retains memory for rapid adaptation to recurring environment situations. By coordinating decentralized agent learning, integrating changepoint detection, and supporting lifelong learning, LA-MAS sets a new standard for scalable, robust adaptation. This thesis demonstrates its contributions through theoretical analysis, synthetic experiments, and adaptive real-web server configurations selection scenarios. The results show that integrating Bayesian RL into architectural adaptation leads to significant improvements in system robustness, flexibility, and long-term performance in uncertain and dynamic environments. The proposed framework provides useful insights into the design of scalable, adaptable systems, and has immediate implications to the autonomic computing and self-adaptive systems community.