We put forward an augmented reality (AR) system using artificial intelligence (AI) to guide users through real-world procedures. The system uses the cameras of the headset to recognize objects and their positions in real time. It constructs a model to predict: (i) the task being completed, (ii) the step of the task they are on, and (iii) whether they have made any errors. By updating this model in real-time as the user completes the task, the system automatically updates the instructions and cues provided to the user. This system represents a step towards ubiquitous task guidance via AR.
No takes yet. Share an insight, caveat, or question.
Rheault et al. (2024) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: