Autonomous off‐road navigation with end‐to‐end learning for the LAGR program

Abstract We describe a fully integrated real‐time system for autonomous off‐road navigation that uses end‐to‐end learning from onboard proprioceptive sensors, operator input, and stereo cameras to adapt to local terrain and extend terrain classification into the far field to avoid myopic behavior. The system consists of two learning algorithms: a short‐range, geometry‐based local terrain classifier that learns from very few proprioceptive examples and is robust in many off‐road environments; and a long‐range, image‐based classifier that learns from geometry‐based classification and continuously generalizes geometry to appearance, making it effective even in complex terrain and varying lighting conditions. In addition to presenting the learning algorithms, we describe the system architecture and results from the Learning Applied to Ground Robots (LAGR) program's field tests. © 2008 Wiley Periodicals, Inc.