The Metaverse and Conversational AI as a Threat Vector for Targeted Influence

Over the last 18 months, two human-computer interaction (HCI) technologies have rapidly come to mainstream markets, funded by massive investments from major corporations. The first area of advancement has been virtual and augmented worlds, now commonly called “The Metaverse.” The second area of advancement has been the foundational AI models that allow users to freely interact with computers through natural dialog. Commonly referred to as “Conversational AI,” this technology has advanced rapidly with the deployment of Large Language Models (LLMs). When combined, these two disciplines will enable users to hold conversations with realistic virtual agents. While this will unleash many positive applications, there is significant danger of abuse. Most significant is the potential deployment of real-time interactive experiences that are designed to persuade, coerce, or manipulate users as a form of AI-powered targeted influence. This issue has largely been overlooked by policymakers who have focused instead on traditional privacy, bias and surveillance risks. It is increasingly important for policymakers to appreciate that interactive influence campaigns can be deployed through AI-powered Virtual Spokespeople (VSPs) that look, speak, and act like authentic users but are designed to push the interests of third parties. Because this “AI Manipulation Problem” is unique to real-time interactive environments, it is presented in this paper in the context of Control Theory to help policymakers appreciate that regulations are likely needed to protect against closed-loop forms of influence, especially when Conversational AI is deployed.