A Keystroke and Pointer Control Input Interface for Wearable Computers

The widespread adoption of mobile electronic devices and the advent of wearable computing have encouraged the development of compact alternatives to the keyboard and mouse. These include one-handed keyboards, digitizing tablets, and glove-based devices. This paper describes a combination pointer position and non-chorded keystroke input device that relies on miniature wrist-worn wireless video cameras that track finger position. A hidden Markov model is used to correlate finger movements to keystrokes during a brief training phase, after which the user can type in the air or above a flat surface as if typing on a standard keyboard. Language statistics are used to help disambiguate keystrokes, allowing the assignment of multiple unique keys to each finger and obviating chorded input. In addition, the system can be trained to recognize certain finger positions for switching between input modes; for example, from typing mode to pointer movement mode. In the latter mode of operation, the position of the mouse pointer is controlled by hand movement. The camera motion is estimated by tracking environmental features and is used to control pointer position. This allows fast switching between keystroke mode and pointer control mode.