Learned Perceptive Forward Dynamics Model for Safe and Platform-aware Robotic Navigation

Pascal Roth1, 2, Jonas Frey1, 3, Cesar Cadena1, and Marco Hutter1
1ETH Zurich, 2NVIDIA, 3Max Planck Institute for Intelligent Systems
Paper arXiv Code

Perceptive Forward Dynamics Model to predict future states and failures, enabling safe navigation using an MPPI planner.

Abstract

Ensuring safe navigation in complex environments requires accurate real-time traversability assessment and understanding of environmental interactions relative to the robot's capabilities. Traditional methods, which assume simplified dynamics, often require designing and tuning cost functions to safely guide paths or actions toward the goal. This process is tedious, environment-dependent, and not generalizable.

To overcome these issues, we propose a novel learned perceptive Forward Dynamics Model (FDM) that predicts the robot's future state conditioned on the surrounding geometry and history of proprioceptive measurements, proposing a more scalable, safer, and heuristic-free solution. The FDM is trained on multiple years of simulated navigation experience, including high-risk maneuvers, and real-world interactions to incorporate the full system dynamics beyond rigid body simulation. We integrate our perceptive FDM into a zero-shot Model Predictive Path Integral (MPPI) planning framework, leveraging the learned mapping between actions, future states, and failure probability. This allows for optimizing a simplified cost function, eliminating the need for extensive cost-tuning to ensure safety. On the legged robot ANYmal, the proposed perceptive FDM improves the position estimation by on average 41% over competitive baselines, which translates into a 27% higher navigation success rate in rough simulation environments. Moreover, we demonstrate effective sim-to-real transfer and showcase the benefit of training on synthetic and real data.

Experiments

Dynamics Estimation

Demonstration of environment- and platform-aware state predictions using the presented FDM. Collision-free predictions of our method are displayed in \( \ours{\rule{1.5ex}{1.5ex}} \), in collision ones in \( \collision{\rule{1.5ex}{1.5ex}} \), whereas the actual path is presented in \( \trajectory{\rule{1.5ex}{1.5ex}} \). In the simulation, the same four action sequences are rolled out across multiple environments, showing that the predicted paths by our model are adapted to the environment. In the real-world experiments, a qualitative comparison between constant velocity estimation \( \constantvel{\rule{1.5ex}{1.5ex}} \) and our model's predictions for the same action sequences across multiple scenarios is shown.

Plane

2D

3D

2D-3D

Traversable Stairs and Ramp

Non-Traversable Ramp and Traversable Stairs

Planning Experiments

We integrate our perceptive FDM into a zero-shot Model Predictive Path Integral (MPPI) planning framework, leveraging the learned mapping between actions, future states, and failure probability. This allows for optimizing a simplified reward function (consisting only of collision and pose reward), eliminating the need for extensive cost-tuning to ensure safety.

Pillar
Untraversable Ramp
Untraversable Stairs
Stairs & Wall
Ramp & Wall
Reward
Pose Reward
Risk Reward

Real-World Planning Part 1

Real-World Planning Part 2

BibTeX

If you use our FDM in your research, please cite our paper: 

@inproceedings{roth2025fdm,
  title={Learned Perceptive Forward Dynamics Model for Safe and Platform-aware Robotic Navigation},
  author={Roth, Pascal and Frey, Jonas and Cadena, Cesar and Hutter, Marco},
  booktitle={Robotics: Science and Systems (RSS 2025)},
  year={2025}
}