A semantic-conditioned multi-modal behavior cloning framework for terrain-specific behavior isolation in humanoid locomotion

Humanoid locomotion across heterogeneous terrains requires terrain-specific behaviors to be selected without unwanted interference between learned skills. This work investigates whether a semantic terrain label can act as a deterministic condition for selecting pre-trained terrain-specific behaviors while preserving behavioral isolation. Two terrain-specific reference policies were independently trained using Proximal Policy Optimization for flat-ground and stairway locomotion of the Unitree H1 humanoid. Their demonstrations were collected as trajectories and used to train a Unified Multi-modal Behavior Cloning (UMBC) policy with dedicated encoders for proprioception, terrain geometry, and semantic terrain identity. The semantic terrain identity is represented using a learned embedding and explicitly conditions a shared policy head for joint-action prediction. The proposed policy achieved an overall test-set action MAE of 0.016349 and RMSE of 0.025733, with a best validation loss of 0.000367. In simulator-only closed-loop deployment, the UMBC policy recovered $$94.73%$$ and $$93.07%$$ of the reference-policy rewards on flat-ground and stairway terrains, respectively. Counterfactual terrain-label analysis produced non-zero action changes across the independent test-set, with mean L₂ prediction distances of 0.5019 and 0.5187 for flat-to-stairway and stairway-to-flat label changes, respectively. Deployment experiments demonstrated terrain-transition capability and evaluated transition delays and locomotion robustness under external perturbations. Ablation experiments showed that adaptive terrain-dependent weighting slightly improved performance on the more difficult stairway terrain but reduced performance on flat-terrain, while one-hot, no-label, and encoder-free concatenation variants resulted in reduced terrain-specific performance relative to the proposed architecture. These results support semantic terrain conditioning as an explicit mechanism for selecting terrain-specific behaviors within a single policy.