A Survey on Security and Privacy of Multimodal LLMs - Connected Healthcare Perspective

Recently, large language and vision language models have shown tremendous success and industry acceptance for healthcare applications. While the firstgeneration models mostly showed advancement in language models, the second generation now offers multi-modal inputs such as audio, image/video, sensory data, and depth, which makes these more suitable toward healthcare applications. However, security and privacy of these multimodal large AI models is largely ignored due to lack of regulatory and compliance applied on these AI models. The healthcare industry requires robust security and privacy of AI models, privacypreserving, and regulation-compliant large language models as it is applied on end users. In this paper, we survey different multimodal large language models and their security and privacy concerns that need to be addressed before these language models can be democratized and accepted by the medical industry. The survey covers different security and privacy threats and vulnerabilities that have been raised by researchers and government bodies and defensive actions such as federated learning, differential privacy, and monitoring LLM processes that have been suggested as remedial actions.