Skip to content

Feature/hetero humanoid - #1

Merged
modanesh merged 11 commits into
modanesh:mainfrom
kaidalisohaib:feature/hetero-humanoid
Sep 4, 2026
Merged

modanesh merged 11 commits into
modanesh:mainfrom
kaidalisohaib:feature/hetero-humanoid

Conversation

@kaidalisohaib

@kaidalisohaib kaidalisohaib commented Aug 8, 2026 •

Copy link
Copy Markdown
Contributor

Description

This PR introduces the HeteroHumanoidEnv, a direct RL environment designed to train a single, unified neural network policy across multiple morphologically distinct humanoid robots simultaneously (Cassie, Digit, G1, and H1).

Key Features & Technical Details

  • Dynamic Unified Action Space: Implemented a canonical joint mapping system (robot_configs.py) that aligns the 12-DOF biped legs across all robots (handling kinematic sign inversions like "bird" vs "humanoid" legs). The environment dynamically calculates the required action dimensions at runtime based on the most complex robot in the configuration (currently scaling to 39-DOF for G1), seamlessly padding the extra DOFs (arms, torso, fingers) for simpler robots.
  • MaskedMLP Architecture: Added a custom MaskedMLPModel that intercepts binary action_masks from the environment observation dictionary. This masks out dummy (padded) dimensions in the forward pass and actively overrides PPO's log_prob and entropy calculations to prevent the optimizer from wasting gradients on non-existent joints.
  • Dynamic Reward Multiplexing: Implemented a scalable reward calculation system (hetero_humanoid_rewards.py) that dynamically routes robot-specific reward functions and penalty weights based on environment IDs (self.get_robot_env_ids()).
  • Per-Robot Environment Overrides: Introduced granular control over environment mechanics, allowing commands, domain randomization (mass, friction, base resets), and push probabilities to be overridden on a strictly per-robot basis, allowing distinct training curricula to coexist in the same environment.
  • Domain Randomization Alignment: Fully implemented and aligned the structural Domain Randomization mechanics (friction, mass, base velocity resets, yaw offsets, and joint pose scales) to perfectly match the parameters of the single-robot manager-based baselines.

File Overview

  • agents/rsl_rl_ppo_cfg.py: RSL-RL PPO algorithm configurations for the flat and rough terrain runners. Integrates the MaskedMLPModel and scales the network hidden dimensions to [512, 256, 128] to accommodate the multi-morphology complexity.
  • __init__.py: Gym environment registry. Maps the Isaac-Velocity-[Flat/Rough]-HeteroHumanoid-v0 IDs to their respective environment and runner configuration entry points.
  • hetero_humanoid_env.py: The core DirectRLEnv implementation. Handles environment resets, parses active humanoids, applies per-robot domain randomization overrides, masks/unpads actions using policy indices, and assembles the unified num_observations tensor.
  • hetero_humanoid_rewards.py: Contains the dynamically scalable reward functions. Extracts robot_env_ids to route logic and penalties exclusively to the robots active in those environments (e.g., universal world-frame velocity tracking and specialized single-stance feet_air_time).
  • hetero_humanoid_velocity_env_cfg.py: The central environment configuration class. Sets dynamic limits (num_actions, num_observations), configures scenes and sensors, and defines the extensive per-robot overrides for commands and domain randomization.
  • masked_mlp_model.py: A custom PyTorch MLPModel subclass. Intercepts the action_mask from the observation dictionary to mathematically zero-out padded/dummy dimensions during forward propagation, log_prob, and entropy calculations.
  • robot_configs.py: A centralized configuration registry defining RobotJointConfig dataclasses for Cassie, Digit, G1, and H1. Defines the canonical joint-slot mapping, kinematic inversion types ("bird" vs "humanoid"), base links, and individual action scales.

How to Run

Train (Flat Terrain):

./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Velocity-Flat-HeteroHumanoid-v0 --num_envs 4096 --headless

Train (Rough Terrain):

./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Velocity-Rough-HeteroHumanoid-v0 --num_envs 4096 --headless

Play / Visualize (Flat Terrain):

./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/play.py --task Isaac-Velocity-Flat-HeteroHumanoid-Play-v0 --num_envs 32

Play / Visualize (Rough Terrain):

./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/play.py --task Isaac-Velocity-Rough-HeteroHumanoid-Play-v0 --num_envs 32

Visuals

hetero_isaaclab_combined

Fixes # (issue)

Type of change

  • New feature (non-breaking change which adds functionality)

Screenshots

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@github-actions github-actions Bot added documentation Improvements or additions to documentation enhancement New feature or request isaac-lab labels Aug 8, 2026
…itecture scaling

- Implemented independent per-robot overrides for DR, pushes, and commands

- Scaled num_actions dynamically based on the largest robot config

- Integrated MaskedMLPModel for continuous action masking

- Removed hidden layer bottlenecks in flat terrain runner config
@kaidalisohaib
kaidalisohaib force-pushed the feature/hetero-humanoid branch from 8725eb2 to 23de576 Compare August 30, 2026 01:29
@modanesh
modanesh merged commit fae8e2d into modanesh:main Sep 4, 2026
4 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request isaac-lab

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants