Seamless Avatar Expression Transfer: Merging Camera and Smart Eyewear Embedded with Photo-Reflective Sensor Technologies

Virtual Reality (VR) offers new ways to interact that are different from usual communication in the real world. In VR, avatars are key displays for showing users' feelings and personalities non-verbally. Addressing the need for more convenient and continuous expression representation, we introduce a novel method for facial expression transfer in VR. This method integrates camera systems with photo-reflective sensors embedded in eyewear, overcoming the limitations of traditional camera-based tracking. By offering smoother tracking and reducing manual calibration needs, this approach highlights the potential of multimodal technology to enhance non-verbal commu-nication in virtual environments. Building on this, we demonstrated the example implementation of smile transfer and discussed future direction.