Reinforcement Learning-Controlled Quadruped Robot: From Basic Gaits to Complex Motions

Abstract

Quadrupedal locomotion control via deep reinforcement learning (RL) has demonstrated strong performance on symmetric gaits, yet its ability to learn non-conventional motions requiring asymmetric, temporally precise coordination remains underexplored. The objective of this study is to examine how increasing locomotion complexity affects reward design, learning dynamics, and zero-shot sim-to-sim robustness. This issue is investigated by progressively training three behaviours of increasing complexity, trotting, bounding, and a lateral alternating jumping, using Proximal Policy Optimization (PPO)-based training with curriculum learning and reference-state initialisation introduced for the more complex behaviours. Policies are evaluated via zero-shot sim-to-sim transfer from Isaac Lab to MuJoCo to assess robustness under differing physics solvers and contact models. The three policies converged after approximately 1k, 2k, and 6.5k iterations, with task-specific final mean rewards of 29, 85, and 624, confirming the need for more structured rewards as locomotion complexity increases.