Adversarial vulnerability following LLM low-resource language fine-tuning: Short report

We briefly report how fine-tuning a multilingual LLM with a low-resource language resulted in an increased vulnerability to adversarial attacks. We fine-tuned GPT-3.5 Turbo (gpt-3.5-0125) with 560 input-output pairs (~274k tokens) of Krama Javanese, a high register of Javanese (a low-resource language). We report brief qualitative and quantitative observations that 1. The fine-tuned model is more compliant towards adversarial prompts, 2. Unsuccessful prompts can be successful when concatenated with an elaboration string, e.g., step-by-step prompting or by specifying details, 3. The model can be prompted in the fine-tuned language to respond in English, thus providing a way to produce harmful responses in a different language. The fine-tuned model sees a 45.1% increase of GPT-4-rated sum of harmfulness for Krama Javanese responses and a 13.8% increase for English responses. Notably, all of these vulnerabilities can be reached very effectively with the benign nature and our small dataset size. Our work contributes knowledge in the intersection of AI safety and multilingual models, indicating that fine-tuning an LLM on a low-resource language should include additional data examples for retaining safety guardrails.