Vision-based robust calibration for optical see-through head-mounted displays

We propose Vision-based Robust Calibration (ViRC) method for OSTHMDs equipped with a camera. In the ViRC method, calibration parameters are decomposed into off-line parameters that remain constant relative to the positional relationship between the camera and the virtual screen, and on-line parameters related to the user's eye. Calculating the off-line parameters beforehand reduces the number of unknown parameters in the on-line phase, giving robust protection against the user's misalignments during calibration. In the off-line phase, the approximate position of the user's eye is calculated using the PnP algorithm. In the online phase, the actual position of the user's eye is estimated from the approximate one by non-linear minimization. In our experiments, we show that the ViRC method can decrease reprojection error by as much as 83% compared with the conventional method based on the DLT algorithm.