Hybrid Avatar-Agent Technology – A Conceptual Step Towards Mediated “Social” Virtual Reality and its Respective Challenges

Abstract Driven by large industry investments, developments of Virtual Reality (VR) technologies including unobtrusive sensors, actuators and novel display devices are rapidly progressing. Realism and interactivity have been postulated as crucial aspects of immersive VR since the naissance of the concept. However, today’s VR still falls short from creating real life-like experiences in many regards. This holds particularly true when introducing the “social dimension” into the virtual worlds. Apparently, creating convincing virtual selves and virtual others and conveying meaningful and appropriate social behavior still is an open challenge for future VR. This challenge implies both, technical aspects, such as the real-time capacities of the systems, but also psychological aspects, such as the dynamics of human communication. Our knowledge of VR systems is still fragmented with regard to social cognition, although the social dimension is crucial when aiming at autonomous agents with a certain social background intelligence. It can be questioned though whether a perfect copy of real life interactions is a realistic or even meaningful goal of social VR development at this stage. Taking into consideration the specific strengths and weaknesses of humans and machines, we propose a conceptual turn in social VR which focuses on what we call “hybrid avatar-agent systems”. Such systems are required to generate i) avatar mediated interactions between real humans, taking advantage of their social intuitions and flexible communicative skills and ii) an artificial social intelligence (AIS) which monitors, and potentially moderates or transforms ongoing virtual interactions based on social signals, such as performing adaptive manipulations of behavior in intercultural conversations. The current article sketches a respective base architecture and discusses necessary research prospects and challenges as a starting point for future research and development.