Watch Your Step: Anticipated Perception for Humanoid Locomotion

Abstract

Perceptive locomotion policies typically observe a dense robot-centric elevation map, although only the terrain beneath and around the next footsteps informs the control decision. Learned encoders compress the whole visual input, leaving the policy to work out which inputs matter while learning the baseline locomotion task. Learning both at once slows training and, without computationally expensive attention mechanisms, makes generalization to new terrain harder. We instead remove redundant inputs from the elevation map before the policy sees it. A lightweight foothold prediction model (FPM) predicts future footholds from proprioception and the velocity command alone, and we sample the elevation map only in small patches around these predicted footholds. We train the prediction model alongside the locomotion policy, periodically retraining it on new policy rollouts as the robot’s gait matures. This allows the perception to anticipate where the policy will step, while the policy learns to leverage terrain information around those steps. With the proposed lightweight preprocessing, the policy reaches a given terrain difficulty in roughly a third of the training iterations required with the complete map. It outperforms policies that observe the complete map where footholds are sparse, including in narrow walled corridors, and on unseen terrain, and it matches an attention-based encoder trained at several times the cost. The policy transfers zero-shot to a Unitree G1 humanoid traversing various obstacles, stairs, gaps, and step-downs beyond the training distribution in both indoor and outdoor environments.

Video

The video will be on YouTube shortly.

Watch on YouTube

Method

The FPM predicts the next two footholds from proprioception and the velocity command. The policy sees the elevation map only in two 7 × 7 patches around them: 2 × 49 heights and the 4 foothold coordinates, 102 terrain inputs instead of the 693 of the full height scan. Every 500 iterations, the FPM is retrained on rollouts of the current policy.

Results

Demo

If you want an example of our results, feel free to play around.

WASD move   QE turn   R reset

predicted footholds patches the policy reads full map (not used)

Q↶ W▲ E↷ A◀ S▼ D▶

Launch

Runs in your browser (about 14 MB).

Citation

Copy@misc{heyrman2026watchstepanticipatedperception,
  title         = {Watch Your Step: Anticipated Perception for Humanoid Locomotion},
  author        = {Matthias Heyrman and Chenhao Li and Ioannis Dadiotis and Marco Hutter},
  year          = {2026},
  eprint        = {2610.29612},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url           = {https://arxiv.org/abs/2610.29612}
}

Acknowledgments

The authors acknowledge the use of generative AI to assist with code generation and language refinement. We thank Tianxu An, Pwint Thu Aung, Dario Hug, Shengzhi Wang, and Fan Yang for their assistance with testing and filming, Victor Klemm for discussions that inspired the assistive force curriculum, and René Zurbrügg for his help in preparing the visuals for this work.