Repository navigation
Feature/hetero humanoid - #1
Merged
Merged
Conversation
…ty, yaw, joint pose scale)
…itecture scaling - Implemented independent per-robot overrides for DR, pushes, and commands - Scaled num_actions dynamically based on the largest robot config - Integrated MaskedMLPModel for continuous action masking - Removed hidden layer bottlenecks in flat terrain runner config
kaidalisohaib
force-pushed
the
feature/hetero-humanoid
branch
from
August 30, 2026 01:29
8725eb2 to
23de576
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR introduces the
HeteroHumanoidEnv, a direct RL environment designed to train a single, unified neural network policy across multiple morphologically distinct humanoid robots simultaneously (Cassie, Digit, G1, and H1).Key Features & Technical Details
robot_configs.py) that aligns the 12-DOF biped legs across all robots (handling kinematic sign inversions like "bird" vs "humanoid" legs). The environment dynamically calculates the required action dimensions at runtime based on the most complex robot in the configuration (currently scaling to 39-DOF for G1), seamlessly padding the extra DOFs (arms, torso, fingers) for simpler robots.MaskedMLPModelthat intercepts binaryaction_masksfrom the environment observation dictionary. This masks out dummy (padded) dimensions in the forward pass and actively overrides PPO'slog_probandentropycalculations to prevent the optimizer from wasting gradients on non-existent joints.hetero_humanoid_rewards.py) that dynamically routes robot-specific reward functions and penalty weights based on environment IDs (self.get_robot_env_ids()).File Overview
agents/rsl_rl_ppo_cfg.py: RSL-RL PPO algorithm configurations for the flat and rough terrain runners. Integrates theMaskedMLPModeland scales the network hidden dimensions to[512, 256, 128]to accommodate the multi-morphology complexity.__init__.py: Gym environment registry. Maps theIsaac-Velocity-[Flat/Rough]-HeteroHumanoid-v0IDs to their respective environment and runner configuration entry points.hetero_humanoid_env.py: The coreDirectRLEnvimplementation. Handles environment resets, parses active humanoids, applies per-robot domain randomization overrides, masks/unpads actions using policy indices, and assembles the unifiednum_observationstensor.hetero_humanoid_rewards.py: Contains the dynamically scalable reward functions. Extractsrobot_env_idsto route logic and penalties exclusively to the robots active in those environments (e.g., universal world-frame velocity tracking and specialized single-stancefeet_air_time).hetero_humanoid_velocity_env_cfg.py: The central environment configuration class. Sets dynamic limits (num_actions,num_observations), configures scenes and sensors, and defines the extensive per-robot overrides for commands and domain randomization.masked_mlp_model.py: A custom PyTorchMLPModelsubclass. Intercepts theaction_maskfrom the observation dictionary to mathematically zero-out padded/dummy dimensions during forward propagation,log_prob, andentropycalculations.robot_configs.py: A centralized configuration registry definingRobotJointConfigdataclasses for Cassie, Digit, G1, and H1. Defines the canonical joint-slot mapping, kinematic inversion types ("bird" vs "humanoid"), base links, and individual action scales.How to Run
Train (Flat Terrain):
Train (Rough Terrain):
Play / Visualize (Flat Terrain):
Play / Visualize (Rough Terrain):
Visuals
Fixes # (issue)
Type of change
Screenshots
Checklist
pre-commitchecks with./isaaclab.sh --formatconfig/extension.tomlfileCONTRIBUTORS.mdor my name already exists there