Large Language Models (LLMs) have achieved strong performance on a wide range of software engineering tasks, yet adapting these models to new tasks and programming languages remains computationally expensive. Parameter-Efficient Fine-Tuning (PEFT) methods address this challenge by updating only a small subset of parameters or introducing lightweight adaptation modules. However, there is limited empirical understanding of how different PEFT methods behave across model families, adaptation settings, and programming language conditions, particularly for low-resource and unseen languages. This thesis investigates the effectiveness of PEFT methods for software engineering through two complementary empirical studies. Study~I examines PEFT in a monolingual adaptation setting, evaluating Compacter, LoRA, and IA³ on general-purpose and code-specific LLMs for code summarization and code generation, with a central focus on knowledge transfer to R, an unseen and low-resource programming language. The results show that LoRA consistently delivers the strongest overall performance, while Compacter achieves competitive accuracy with substantially lower computational and storage requirements. Study~II investigates PEFT in a multilingual adaptation setting, focusing on cross-language knowledge transfer for code translation on modern decoder-only Code-LLMs, including CodeLlama, DeepSeek-Coder, and Qwen2. 5-Coder. Fusion-based approaches are compared against LoRA and Compacter. The results indicate that LoRA remains a consistently strong baseline, while integrating Compacter within fusion architectures substantially improves performance in multilingual code translation. Overall, this thesis demonstrates that PEFT methods are effective mechanisms for facilitating knowledge transfer across monolingual and multilingual adaptation settings. Effective PEFT selection depends on the interaction between task characteristics, model capability, programming language properties, and resource constraints. These findings provide practical guidance for efficient and context-aware model adaptation in software engineering.
Amirreza Esmaeili (Thu,) studied this question.