Application of Virtual Avatar using Motion Capture in Immersive Virtual Environment

AbstractThis study proposes a method for providing various experiential environments and experiences by applying virtual avatars based on motion capture to produce immersive virtual environment. It aims to increase the immersion of virtual environment and provide new experiences, whereby both non-immersive users (using PCs and mobile devices) and immersive users (using virtual reality (VR)) can participate and perform various experiences. To this end, an integrated development environment based on the Unity 3D engine is built. In this environment, immersive content allowing for multi-user participation on cross-platforms and virtual avatars using motion capture can work together flexibly. Further, it defines the real-time streaming processes of movements in a real space as calculated from actors (skeleton data) and motion capture cameras for avatars in a virtual environment. Immersive content that users in the various participating environments can experience together is created and virtual avatars based on actor are applied according to the motion capture. In addition, through survey experiments, the experiential environment that interacts with virtual avatars based on human actor using motion capture and non-player character (NPC) avatars operating according to a predetermined pattern within the produced virtual environment is compared and analyzed. This virtual avatar application method using motion capture provides a high-immersion, and an interesting and positive experience owing to the irregular and unpredictable variety of experiences.Keywords: Virtual avatarmotion capturenon-immersive and immersive usersinteractionimmersive virtual environment Disclosure statementNo potential conflict of interest was reported by the author(s).Additional informationFundingThis research was financially supported by Hansung University for Jinmo Kim.Notes on contributorsMyeongseok ParkMyeongseok Park is currently a master's degree student of Department of Computer Engineering, Hansung University in South Korea. He received B.E. degree in Division of Computer Engineering from Hansung University, South Korea, in 2022. His research interests include virtual reality, augmented reality, HCI and computer graphics application.Yunsik ChoYunsik Cho is currently a Ph.D. student of the Department of Information and Computer Engineering, Hansung University in South Korea. He received B.E. and M.S. degrees from Hansung University in 2021 and 2022, respectively. His research interests include virtual reality, augmented reality, HCI and computer graphics application.Giri NaGiri Na is currently a master's degree student of Department of Computer Engineering, Hansung University in South Korea. She received B.E. degree in Division of Computer Engineering from Hansung University, South Korea, in 2023. His research interests include virtual reality, augmented reality, HCI and computer graphics application.Jinmo KimJinmo Kim is an associate professor of Division of Computer Engineering, Hansung University. He received B.E. degree in Multimedia Engineering, Dongguk University in 2006 and his M.E. and Ph.D. degrees in Multimedia, Dongguk University in 2008 and 2012, respectively. His primary research interests include VR/AR, computer graphics, and related applications.