Autonomous robot navigation in off-road environments requires a comprehensive understanding of the terrain geometry and traversability. The degraded perceptual conditions and sparse geometric information at longer ranges make the problem challenging especially when driving at high speeds. Furthermore, the sensing-to-mapping latency and the look-ahead map range can limit the maximum speed of the vehicle. Building on top of the recent work RoadRunner, in this work, we address the challenge of long-range (±100 m) traversability estimation. Our RoadRunner (M&M) is an end-to-end learning-based framework that directly predicts the traversability and elevation maps at multiple ranges (±50 m, ±100 m) and resolutions (0.2 m, 0.8 m) taking as input multiple images and a LiDAR voxel map. Our method is trained in a self-supervised manner by leveraging the dense supervision signal generated by fusing predictions from an existing traversability estimation stack (X-Racer) in hindsight and satellite Digital Elevation Maps (DEM). RoadRunner M&M achieves a significant improvement of up to 50% for elevation mapping and 30% for traversability estimation over RoadRunner, and is able to predict in 30% more regions compared to X-Racer while achieving real-time performance. Experiments on various out-of-distribution datasets also demonstrate that our data-driven approach starts to generalize to novel unstructured environments. We integrate our proposed framework in closed-loop with the path planner to demonstrate autonomous high-speed off-road robotic navigation in challenging real-world environments.
RoadRunner M&M takes as input four RGB images and a LiDAR voxel map to predict traversability (risk) and elevation maps at multiple ranges: high resolution Micro range (±50 m) and low resolution Short range (±100 m). In the above example, the vehicle is traversing through a dense forest environment. In the zoomed-in version of the Micro range risk map, the risk associated with the trees (a, b) can be clearly visualized.
Overview of the RoadRunner M&M network architecture. The network takes as an input four RGB images which are encoded using the Lift-Splat method. PointPillars encoding is used for the input voxel map. Additonally, a raw elevation map is extracted from the voxel map using the min Z values. These multi-modal features are stacked and passed through a hierarchical decoder which predicts the maps at different ranges and resolutions.
Our network is trained on a dataset consisting of sequences from a dry hilly grassland environment at Paso Robles, CA, USA. Here we show some qualitative results on the test sequences from a similar environment.
We deploy our proposed approach zero-shot on out-of-distribution test datasets including a dense forest, canyon, beach and desert environments to evaluate its generalization performance. These environments are markedly different from the training (Paso Robles) dataset. Overall, RoadRunner M&M predicts accurate and consistent elevation maps and is able to associate the corresponding traversability risks even at longer ranges. However, occasionally, it struggles to assign risk to certain unseen objects such as small Joshua trees in the Mojave desert.
Evaluation in a Beach environment (Pendleton Beach, CA, USA). RoadRunner M&M is able to predict the elevation of the rocky and canyon sturctures while correctly associating traversability risks to it.
We perform qualitative evaluations of our predictions with the short range planner. RoadRunnner M&M is able to better predict the risks at longer ranges (resembling the ground truth risk maps) when compared to X-Racer. On providing a goal at a distance of 100 m from the vehicle, RoadRunner M&M planner is able to plan trajectories (orange) around the obstacles while X-Racer planner gives a uniform cost-to-go away from the goal point and thus plans trajectories (pink) straight through the obstacles (since it is yet to detect the obstacles at longer ranges).
We carry out an autonomous mission at Arroyo Seco, Pasadena, CA, using our modified Polaris RZR all-terrain vehicle. The vehicle navigated a 400 m course with five waypoints. The planner stack used Short range maps from RoadRunner M&M, allowing the vehicle to safely complete the course at speeds up to 12 m/s, successfully reaching all waypoints.
We observe an interesting failure case at the San Gabriel Canyon where the network incorrectly predicts higher elevation and risk for the overhead bridge as seen in the above video. This could be likely due to the absence of similar overhanging structures in the training data. Training on a larger, more diverse dataset might mitigate this issue and lead to better performance.
@misc{patel2024roadrunnermmlearning,
title={RoadRunner M&M -- Learning Multi-range Multi-resolution Traversability Maps for Autonomous Off-road Navigation},
author={Manthan Patel and Jonas Frey and Deegan Atha and Patrick Spieler and Marco Hutter and Shehryar Khattak},
year={2024},
eprint={2409.10940},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2409.10940},
}