Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 33 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,24 +14,25 @@

**Notice:** This repository is a fork of the main Isaac Lab repo built on top of upstream commit [84d0ff05](https://github.com/isaac-sim/IsaacLab/tree/84d0ff05a6). Modifications for heterogeneous training are layered directly on top of the core framework, maintaining a clean Git history.

**Notice:** This project is different from [IsaacLab-HARL](https://github.com/DIRECTLab/IsaacLab-HARL) which focuses on heterogeneous _multi-agent_ learning in Isaac Lab. In this project, we focus on heterogeneous _single-robot_ learning, where each environment contains a single robot but the robots across environments are different. This allows us to train morphology-agnostic policies across multiple quadrupedal robots simultaneously.
**Notice:** This project is different from [IsaacLab-HARL](https://github.com/DIRECTLab/IsaacLab-HARL) which focuses on heterogeneous _multi-agent_ learning in Isaac Lab. In this project, we focus on heterogeneous _single-robot_ learning, where each environment contains a single robot but the robots across environments are different. This allows us to train morphology-agnostic policies across multiple quadrupedal and humanoid/bipedal robots simultaneously.

**Isaac Lab** is a GPU-accelerated, open-source framework designed to unify and simplify robotics research workflows, such as reinforcement learning, imitation learning, and motion planning. Built on NVIDIA Isaac Sim, it combines fast and accurate physics and sensor simulation, making it an ideal choice for sim-to-real transfer in robotics.

## 🤖 Heterogeneous Multi-Robot Training

This fork introduces **Hetero-IsaacLab**, a specialized architecture for training morphology-agnostic locomotion policies across multiple heterogeneous **quadrupedal** robotic environments simultaneously.
This fork introduces **Hetero-IsaacLab**, a specialized architecture for training morphology-agnostic locomotion policies across multiple heterogeneous **quadrupedal** and **humanoid/bipedal** robotic environments simultaneously.

Most physics simulators and RL frameworks assume homogeneity, making it difficult to train universal controllers. This repository bridges that gap, providing concrete advantages:
* **Morphology-Agnostic Feature Learning:** The policy is forced to learn fundamental locomotion principles that transcend specific hardware rather than memorizing robot-specific quirks.
* **Efficient Multi-Platform Deployment:** Training 8 robot types heterogeneously uses the same compute as training 1 robot type, eliminating the need to maintain separate codebases and models.
* **Better Exploration:** Different morphologies explore different regions of the state-action space naturally (e.g., lighter robots discover high-speed gaits, heavier ones excel at stability).
* **Morphology-Agnostic Feature Learning:** The policy is forced to learn fundamental locomotion principles that transcend specific hardware morphologies rather than memorizing robot-specific quirks.
* **Efficient Multi-Platform Deployment:** Training 8 quadruped or 5 humanoid robot types heterogeneously uses the same compute as training 1 robot type, eliminating the need to maintain separate codebases and models.
* **Better Exploration:** Different morphologies explore different regions of the state-action space naturally (e.g., varying leg lengths, weights, and joint actuation limits discover robust, versatile gaits).

### Key Architecture Features
* **Heterogeneous Configuration System:** Custom config classes with dynamic environment assignment and reward filtering.
* **Observation & Action Unification:** Enforces an "ANYmal Joint-Major" format, mapping diverse joint orders (e.g., Spot, Unitree) to a standard policy format.
* **Index Mapping System:** Efficient conversion between global environment IDs and robot-local indices.
* **Comprehensive Domain Randomization:** Handles extreme morphological quirks with flexible reset randomizations (mass, CoM, friction) and interval randomization for external disturbances.
* **Dual-Paradigm Heterogeneous Environments:** Supports both Manager-Based environments and high-throughput Direct RL workflows (`HeteroQuadrupedEnv` and `HeteroHumanoidEnv`).
* **Quadruped Joint Unification:** Enforces an "ANYmal Joint-Major" format, mapping diverse quadruped joint orders (e.g., Spot, Unitree, ANYmal) to a standard policy format.
* **Humanoid Semantic Joint Alignment & Action Masking:** A canonical 50-DOF semantic joint alignment system combined with `MaskedMLPModel` to seamlessly handle morphological variations across humanoids (from 12-DOF biped legs to 30+ DOF full-body humanoids with arms and hands) without gradient pollution or entropy distortions.
* **Index Mapping System:** Efficient zero-copy conversion between global environment IDs and robot-local indices.
* **Comprehensive Domain Randomization:** Handles extreme morphological quirks with per-robot overrides for mass, center of mass, ground friction, velocity command ranges, and external disturbance pushes.

### Training Results and Insights
For a comprehensive breakdown of these experimental results, including the detailed methodology and the complete set of performance plots, the full report is available in [this WandB report](https://wandb.ai/modanesh/Hetero-Isaac/reports/Hetero-IsaacLab-Experiments--VmlldzoxNjQ4NTMzNw).
Expand All @@ -54,6 +55,7 @@ cd Hetero-IsaacLab

### Basic Training

#### 1. Heterogeneous Quadrupeds
To train on a specific subset of quadrupeds, pass the `--quadrupeds` flag from `anymal_d,anymal_c,anymal_b,unitree_a1,unitree_go1,unitree_go2,unitree_b2,spot`:

```bash
Expand All @@ -63,13 +65,32 @@ To train on a specific subset of quadrupeds, pass the `--quadrupeds` flag from `
--quadrupeds anymal_d,anymal_c,anymal_b,unitree_a1,unitree_go1,unitree_go2,unitree_b2,spot
```

To train on heterogeneous **humanoid/biped** robots (`cassie,digit,g1,h1`), pass the `--humanoids` flag:
#### 2. Heterogeneous Humanoids & Bipeds
To train on heterogeneous **humanoid/biped** robots (`cassie,digit,g1,h1,h2`), pass the `--humanoids` flag:

```bash
# Train on all 4 humanoid robots with 4096 environments
# Train on all 5 humanoid robots with 4096 environments
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py \
--task=Isaac-Velocity-Flat-HeteroHumanoid-v0 \
--humanoids cassie,digit,g1,h1
--humanoids cassie,digit,g1,h1,h2
```

#### 3. Single-Robot Locomotion (Unitree H2 Humanoid)
You can also train on the standalone Unitree H2 full-body humanoid (31-DOF) via the Manager-Based RL workflow:

```bash
# Download official Unitree H2 USD assets (if needed)
python scripts/tools/download_h2_assets.py

# Train Unitree H2 on flat terrain
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py \
--task=Isaac-Velocity-Flat-H2-v0 \
--num_envs=4096

# Play and visualize trained H2 policy
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/play.py \
--task=Isaac-Velocity-Flat-H2-Play-v0 \
--num_envs=16
```


Expand Down
4 changes: 2 additions & 2 deletions docs/source/_static/all_robots_unified.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
112 changes: 67 additions & 45 deletions scripts/demos/visualize_all_robots.py
Original file line number Diff line number Diff line change
@@ -1,32 +1,45 @@
# Copyright (c) 2022-2026, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause

import argparse
import sys
import torch
import os
import subprocess

from isaaclab.app import AppLauncher

parser = argparse.ArgumentParser(description="Visualize all 12 robots in one scene.")
parser = argparse.ArgumentParser(description="Visualize all 13 robots in one scene.")
AppLauncher.add_app_launcher_args(parser)
args_cli = parser.parse_args()

app_launcher = AppLauncher(args_cli)
simulation_app = app_launcher.app

import isaaclab.sim as sim_utils
from isaaclab.scene import InteractiveScene, InteractiveSceneCfg
from isaaclab.assets import AssetBaseCfg
from isaaclab.scene import InteractiveScene, InteractiveSceneCfg
from isaaclab.sensors import CameraCfg
from isaaclab.utils import configclass

from isaaclab_assets.robots.agility import DIGIT_V4_CFG

# Quadruped configs
from isaaclab_assets.robots.anymal import ANYMAL_D_CFG, ANYMAL_C_CFG, ANYMAL_B_CFG
from isaaclab_assets.robots.unitree import UNITREE_A1_CFG, UNITREE_GO1_CFG, UNITREE_GO2_CFG, UNITREE_B2_CFG
from isaaclab_assets.robots.spot import SPOT_CFG
from isaaclab_assets.robots.anymal import ANYMAL_B_CFG, ANYMAL_C_CFG, ANYMAL_D_CFG

# Humanoid configs
from isaaclab_assets.robots.cassie import CASSIE_CFG
from isaaclab_assets.robots.agility import DIGIT_V4_CFG
from isaaclab_assets.robots.unitree import G1_MINIMAL_CFG, H1_MINIMAL_CFG
from isaaclab_assets.robots.spot import SPOT_CFG
from isaaclab_assets.robots.unitree import (
G1_MINIMAL_CFG,
H1_MINIMAL_CFG,
H2_MINIMAL_CFG,
UNITREE_A1_CFG,
UNITREE_B2_CFG,
UNITREE_GO1_CFG,
UNITREE_GO2_CFG,
)


@configclass
class UniversalSceneCfg(InteractiveSceneCfg):
Expand All @@ -38,7 +51,7 @@ class UniversalSceneCfg(InteractiveSceneCfg):
prim_path="/World/ground",
spawn=sim_utils.GroundPlaneCfg(),
)

# Light
light = AssetBaseCfg(
prim_path="/World/skyLight",
Expand All @@ -51,7 +64,7 @@ class UniversalSceneCfg(InteractiveSceneCfg):
# Global camera to capture the grid
camera = CameraCfg(
prim_path="/World/camera",
update_period=0.02, # 50 fps
update_period=0.02, # 50 fps
height=720,
width=1280,
data_types=["rgb"],
Expand All @@ -60,13 +73,13 @@ class UniversalSceneCfg(InteractiveSceneCfg):
),
# Position the camera at (18.0, 18.0, 12.0) looking at (2.0, 3.0, 0.0)
offset=CameraCfg.OffsetCfg(
pos=(18.0, 18.0, 12.0),
rot=(0.3420, 0.2027, 0.4679, 0.7893), # w, x, y, z
convention="opengl"
pos=(18.0, 18.0, 12.0),
rot=(0.3420, 0.2027, 0.4679, 0.7893), # w, x, y, z
convention="opengl",
),
)

# 12 Robots
# 13 Robots (8 Quadrupeds + 5 Humanoids)
anymal_d = ANYMAL_D_CFG.replace(prim_path="/World/envs/env_.*/anymal_d")
anymal_c = ANYMAL_C_CFG.replace(prim_path="/World/envs/env_.*/anymal_c")
anymal_b = ANYMAL_B_CFG.replace(prim_path="/World/envs/env_.*/anymal_b")
Expand All @@ -75,41 +88,45 @@ class UniversalSceneCfg(InteractiveSceneCfg):
unitree_go2 = UNITREE_GO2_CFG.replace(prim_path="/World/envs/env_.*/unitree_go2")
unitree_b2 = UNITREE_B2_CFG.replace(prim_path="/World/envs/env_.*/unitree_b2")
spot = SPOT_CFG.replace(prim_path="/World/envs/env_.*/spot")

cassie = CASSIE_CFG.replace(prim_path="/World/envs/env_.*/cassie")
digit = DIGIT_V4_CFG.replace(prim_path="/World/envs/env_.*/digit")
g1 = G1_MINIMAL_CFG.replace(prim_path="/World/envs/env_.*/g1")
h1 = H1_MINIMAL_CFG.replace(prim_path="/World/envs/env_.*/h1")
h2 = H2_MINIMAL_CFG.replace(prim_path="/World/envs/env_.*/h2")


def main():
scene_cfg = UniversalSceneCfg()

# The configs are set up to be placed in a grid in env_0 and then cloned.
# Grid: 4 columns, 3 rows
# x goes 0, 2, 4
# y goes 0, 2, 4, 6

configs = [
scene_cfg.anymal_d, scene_cfg.anymal_c, scene_cfg.anymal_b, scene_cfg.unitree_a1,
scene_cfg.unitree_go1, scene_cfg.unitree_go2, scene_cfg.unitree_b2, scene_cfg.spot,
scene_cfg.cassie, scene_cfg.digit, scene_cfg.g1, scene_cfg.h1
scene_cfg.anymal_d,
scene_cfg.anymal_c,
scene_cfg.anymal_b,
scene_cfg.unitree_a1,
scene_cfg.unitree_go1,
scene_cfg.unitree_go2,
scene_cfg.unitree_b2,
scene_cfg.spot,
scene_cfg.cassie,
scene_cfg.digit,
scene_cfg.g1,
scene_cfg.h1,
scene_cfg.h2,
]

# We will orient them all to face -X so they look at the camera
# Quaternion for 180 degree yaw (Z axis)
import math
yaw_180 = (0.0, 0.0, 0.0, 1.0) # w,x,y,z for 180 yaw is actually (0, 0, 0, 1) or (0, 0, 1, 0)
# wait w=cos(90)=0, z=sin(90)=1 -> (0, 0, 0, 1)


for i, cfg in enumerate(configs):
row = i // 4
col = i % 4
cfg.init_state.pos = (row * 2.0, col * 2.0, 1.0)
cfg.init_state.rot = (0.0, 0.0, 0.0, 1.0)

# Reset simulation first
sim = sim_utils.SimulationContext(sim_utils.SimulationCfg(dt=0.005))
sim.set_camera_view([10, 10, 5], [3, 3, 0])

scene = InteractiveScene(scene_cfg)
sim.reset()
scene.update(0.0)
Expand All @@ -126,48 +143,53 @@ def main():
# Step physics (zero actions applied naturally)
sim.step()
scene.update(sim.cfg.dt)

# Capture camera
if step % 4 == 0: # capture every 4 physics steps (dt=0.005 -> 0.02 render step)
# image is (H, W, 4) from isaaclab camera
img_tensor = scene.sensors["camera"].data.output["rgb"][0].cpu().numpy()

# Convert RGBA to RGB if needed
if img_tensor.shape[2] == 4:
img_tensor = img_tensor[:, :, :3]

# IsaacLab images are usually uint8 already
if img_tensor.dtype != np.uint8:
img_tensor = (img_tensor * 255).astype(np.uint8)

frames.append(img_tensor)

print(f"[INFO] Captured {len(frames)} frames. Generating GIF with ffmpeg...")

# Write frames to tmp dir for ffmpeg
for i, f in enumerate(frames):
# Convert RGB to BGR for cv2 imwrite
f_bgr = cv2.cvtColor(f, cv2.COLOR_RGB2BGR)
cv2.imwrite(f"tmp_frames/frame_{i:04d}.png", f_bgr)

# Generate GIF
out_path = "docs/source/_static/all_robots_unified.gif"
cmd = [
"ffmpeg",
"-framerate", "50",
"-i", "tmp_frames/frame_%04d.png",
"-filter_complex", "split[s0][s1];[s0]palettegen=max_colors=256[p];[s1][p]paletteuse=dither=bayer",
"-framerate",
"50",
"-i",
"tmp_frames/frame_%04d.png",
"-filter_complex",
"split[s0][s1];[s0]palettegen=max_colors=256[p];[s1][p]paletteuse=dither=bayer",
"-y",
out_path
out_path,
]
subprocess.run(cmd, check=True)

# Cleanup
for i in range(len(frames)):
os.remove(f"tmp_frames/frame_{i:04d}.png")
os.rmdir("tmp_frames")

print(f"[SUCCESS] Unified GIF saved to {out_path}")
simulation_app.close()


if __name__ == "__main__":
main()
2 changes: 1 addition & 1 deletion scripts/reinforcement_learning/rsl_rl/cli_args.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ def add_rsl_rl_args(parser: argparse.ArgumentParser):
"--humanoids",
type=str,
default=None,
help="Comma-separated list of humanoids to use, e.g., 'cassie,digit,g1,h1'.",
help="Comma-separated list of humanoids to use, e.g., 'cassie,digit,g1,h1,h2'.",
)


Expand Down
Loading
Loading