Everyday, Augmented Reality (AR) headsets are becoming smaller in size and more fashion-forward, where the headsets will see all-day wear and prolonged use throughout the day. AR headsets are equipped with an array of ‘always-on’ sensors, such as front-facing depth and RGB cameras, microphones, and eye-tracking that continuously collect data, enabling functionalities such as augmenting the user’s view of the world. The ‘always-on’ sensors lead users to provide AR headsets with more data about themselves, their surroundings, and those around them than they might initially have been aware of or even have consented to be collected. Despite the scale and continuity of the data collection, the data-access controls available on current AR headsets remain limited and largely require users to manually configure permissions. However, as is well known from smartphone usage, users rarely revisit or manage such permission settings; the same would be likely true for AR. Thus, most AR applications would be expected to retain broad, long-term access to sensitive sensor data without the user’s continued awareness or oversight. This publication-based thesis examines these challenges across the lifecycle of everyday AR use and contributes three core advances: (1) mapping privacy challenges in AR and user difficulties with current permission models; (2) introducing new approaches for data-access control tailored to AR’s continuous sensing; and (3) developing mechanisms to support user awareness of active applications and real-time data collection. First, through expert focus groups, the thesis identifies emerging XR privacy risks, including the inadequacy of binary permission prompts and the increased quantity, quality, and scope of data captured by head-worn devices. Complementary user studies show that current smartphone-derived permission systems do not support informed decision-making in AR, leading users uncertain in understanding what data AR applications collect, when sensing occurs, and why access is needed. Second, the thesis introduces a novel fine-grained, user-experience-based permission system that allows users to grant applications different levels of data fidelity, each paired with a preview of expected functionality. Evaluations show that the novel permission system improves users’ understanding of application data access, increases trust, and supports more privacy-aware choices compared to state-of-the-art mobile and AR permission controls. Third, the thesis identifies seven sensitive context archetypes that motivate when users wish to restrict AR sensing, providing a foundation for context-aware data-access control. Finally, the thesis develops and evaluates interface concepts that communicate augmentation origins and real-time data collection, significantly improving users’ ability to detect over-privileged or malicious applications. Collectively, the thesis provides empirical foundations, design concepts, and practical mechanisms that support more private, informed, and trustworthy uses of everyday AR headsets.
Melvin Abraham (Thu,) studied this question.