Cracks in Collaboration: Threat Models and Attacks on Multi-LLM Collaborative Systems

Multi-LLM collaborative systems have attracted significant attention as a promising solution for complex tasks, enabling multiple large language models (LLMs) with different domains to work together toward a common goal. Different collaborative structures (e.g., Centralized, Horizontal, and Joint Interaction) and communication methods (e.g., direct, summary, and vote) give the system with enhanced flexibility and reasoning capability. However, these same mechanisms also introduce potential security and privacy risks, such as the generation of incorrect responses and the leakage of sensitive information. Based on the above unique characteristics of collaborative systems, we propose three attack methods (named Decision Poisoning Attack, Indirect Echoleak Attack and Information Collision Attack) that exploit the interactions between LLMs to achieve different objectives like system manipulation and privacy leakage. Extensive experiments demonstrate the effectiveness of the proposed attack on three structures and three communication methods, highlighting the security vulnerabilities and potential risks in Multi-LLM collaborative systems. We further discuss possible defense methods that can mitigate the attack performance. Our work show that (1) the key factor for a successful attack on collaborative systems is ensuring the malicious instruction persists and propagates throughout the inter-LLMs communication. (2) both the system architecture and the communication method can affect attack effectiveness, offering valuable insights for the design of more secure Multi LLM collaborative systems in the future.