Evaluating and Optimizing Chaotically Behaving Mobile Robots with a Deterministic Simulation

Testing and comparing prototypes is essential during the development of mobile autonomous robots. These robots often behave chaotically, i.e. their behavior is sensitive to small perturbations and their performance is unpredictable. An evaluation and comparison of solutions cannot be based on a few test runs. We present an approach that uses a digital simulation to acquire the data necessary for statistical comparison. The simulation can be sped up and automated and thus allows the developers to compare solutions more quickly than physical tests can. While it is not possible to replicate a test setup in the physical world with sufficient accuracy, the simulation is deterministic and can be used to investigate the dependency of a specific perturbation. We illustrate this approach using three solutions to an example task where the robot needs to remove objects from an area in the shortest time possible. When executing each solution from 99 randomly chosen starting conditions an average performance difference of as little as 2% was distinguished with statistical significance. Differences in performance under an environmental perturbation such as a slight misaligned of the robot’s starting orientation or a hardware perturbation of imprecisely controlled motor speeds are also found to be significant for some solutions showing how the approach can be used to test robustness of solutions to perturbations. Robot performance is dependent on the interplay between the algorithm that controls the robot, the robot hardware and its environment. By testing different algorithms and subjecting them to perturbations of the environment and the robot’s hardware, it is possible to optimize the algorithm-hardware combination for the least restrictive hardware requirements necessary while still ensuring the desired performance. In this paper we offer a quantitative approach for iterative development of the algorithm-hardware combination of mobile autonomous robot solutions.