Real-world humanoid locomotion with reinforcement learning.

Humanoid robots that can autonomously operate in diverse environments have the potential to help address labor shortages in factories, assist elderly at home, and colonize new planets. Although classical controllers for humanoid robots have shown impressive results in a number of settings, they are challenging to generalize and adapt to new environments. Here, we present a fully learning-based approach for real-world humanoid locomotion. Our controller is a causal transformer that takes the history of proprioceptive observations and actions as input and predicts the next action. We hypothesized that the observation-action history contains useful information about the world that a powerful transformer model can use to adapt its behavior in context, without updating its weights. We trained our model with large-scale model-free reinforcement learning on an ensemble of randomized environments in simulation and deployed it to the real-world zero-shot. Our controller could walk over various outdoor terrains, was robust to external disturbances, and could adapt in context. Editor's summary: The ability of robots to navigate adaptively and robustly in varying terrain increases their chances of success when deployed in the real world. However, stable locomotion of full-size bipedal humanoid robots creates a challenge from a controls perspective. Radosavovic et al. developed a reinforcement learning approach for controlling locomotion of a humanoid robot, Digit. They trained their model in simulation and subsequently deployed it into the real-world zero-shot and showed the potential for robust locomotion on various indoor and outdoor environments. The robot could exhibit natural and adaptive walking behaviors, including an emergent arm-swing motion, and adapt to external perturbations. —Amos Matsiko [ABSTRACT FROM AUTHOR]