This paper proposes a novel framework for designing immersive creative systems in Virtual and Mixed Reality (VR/MR) by drawing upon the rich, centuries-old domain of music. It argues that the principles underlying musical creation provide a powerful blueprint for interaction design. By analyzing the cognitive processes of composition and performance, this work synthesizes a design framework organized around four key pillars: (1) the Visualization of the Sonic Space, which translates abstract musical ideas into navigable environments; (2) the design of Embodied Instruments as gestural interfaces; (3) the implementation of Structured Freedom through guiding constraints; and (4) a paradigm shift towards The Composer as World-Builder, creating environments for interpretation. The paper then demonstrates the framework’s practical application by outlining its use in designing real-world immersive experiences. The goal is to guide development away from simple virtual tools and towards interpretive, collaborative environments that amplify human creative potential.
Rennó-Costa et al. (Mon,) studied this question.