This document aims to establish the technological base for INDUX-R by presenting and analysing the State-of-the-Art in the scientific fields that comprise an eXtended Reality system relevant to the needs of the project, harnessing the expertise of INDUX-R partners. During this evaluation of existing methodologies, several factors were considered such as capacity for innovation and interoperability with other technologies. Effort was put towards pinpointing technological constraints, comprehending the current gaps impeding the advancement and adoption of eXtended Reality applications and highlighting potential breakthroughs. Additionally, relevant standards and open-source tools were identified, that can facilitate the development of the INDUX-R ecosystem. In this deliverable, the foundations for the creation of the eXtened Reality world and for the animation of humans and virtual objects in it are set. An overview of the best practises for 3D reconstruction and neural rendering is provided, as well as techniques for human motion animation, be it body or facial. The deliverable continues with presenting methods for spatial and immersive audio. These methods have a pivotal role in enhancing the immersive experience of the users via making the eXtened Reality world more realistic. A State-of-the-Art analysis for egocentric vision techniques including methods for Object Detection, Visual Object Tracking and Instance Segmentation is carried out. These methods are essential for user interaction. Additionally, 2D/3D human and object pose estimation are paramount for localization and are also investigated thoroughly. User interaction in immersive systems through gesture-based interaction and haptic feedback integration in eXtened Reality devices is thoroughly assessed. Additionally, context aware visualization, taking into consideration the users emotional state, is investigated. The deliverable progresses to the Internet of Things infrastructure, essential for eXtened Reality systems. Starting with Display Systems and Optics, the user’s comfort, the optical sensor specifications and display devices for XR are dissected. Lastly the eXtened Reality media streaming requirements in packet loss and latency for the different use cases are defined, setting the ground for the potential functions and analytics of 5G infrastructure in eXtened Reality systems. The middleware solution which will be utilized as the intermediate layer between the operating system and the various applications is scrutinized, as well as an overview of the various communication, data exchange, security and networking protocols.
Iason Karakostas (Thu,) studied this question.