Online Sim-to-Real Adaptation via Closed-Loop System Modeling

Yuhao Huang, Samuel A. Moore, and Boyuan Chen
Duke University

OSRAM (Online Sim-to-Real Adaptation via Closed-Loop System Modeling) is a research framework for training robot policies and learned dynamics models, adapting those models from online observations, and deploying the resulting controllers on TRON1 in simulation or on hardware.

This repository combines the training and deployment code in one checkout. The two modules remain independently installable and keep their own dependencies, configuration, and documentation.

Modules

Module	Purpose	Source snapshot
training	Python package for Mjlab policy training, rollout collection, learned-dynamics training, online finetuning, and MPPI controllers. See its README.
deployment	ROS 2 packages, robot descriptions, launch files, policies, and planners for TRON1 simulation and hardware deployment. See its README.

How the modules fit together

Mjlab policy training and simulation
              |
              v
      trajectory collection
              |
              v
learned dynamics training / finetuning
              |
              v
 policy and dynamics checkpoints
              |
              v
 ROS 2 simulation or TRON1 deployment

The training module owns the Python package and the training pipeline. The deployment module owns the ROS 2 runtime and robot assets. Its MPPI nodes import training, so install and activate the Python environment before running those nodes. Moving a newly trained checkpoint into a deployment configuration is currently an explicit step; the modules do not automatically synchronize generated artifacts.

Getting started

Training and dynamics finetuning

Requirements include Linux, Python 3.12 or 3.13, and uv. CUDA 12.8 is required for simulator training and GPU-backed JAX workflows.

cd training
uv sync --frozen --dev
uv run list-envs

On a compatible CUDA workstation, include the CUDA extra:

uv sync --frozen --dev --extra cuda

See the training module guide for policy training, exploration, dataset creation, model training, plotting, and development checks.

ROS 2 deployment

The deployment module targets Ubuntu 24.04 and ROS 2 Jazzy. It also requires the packages supplied by deployment_code_base. Place this repository and that dependency under a ROS 2 workspace's src/ directory, then build the two TRON1 packages:

cd ~/tron1_ws
source /opt/ros/jazzy/setup.bash
rosdep install --from-paths src --ignore-src -r -y
colcon build --symlink-install --packages-select tron1_description tron1_deploy
source install/setup.bash

For MPPI launches, first activate the environment in which the training module is installed and confirm the import succeeds:

python3 -c "import training"

See the deployment module guide for simulation commands, hardware launch arguments, runtime-model locations, and troubleshooting.

Warning

Real-hardware launch files command a physical robot. Validate the selected policy and controller configuration in simulation, provide a working emergency stop, suspend the robot, and clear the work area before enabling actuators.

Repository layout

OSRAM/
├── training/                                  # Python training package
├── deployment/                                # ROS 2 deployment packages
└── README.md

Run build, test, and formatting commands from the relevant module directory; there is no root-level environment that merges their Python and ROS 2 dependencies.

Citation

If you find our paper or codebase helpful, please consider citing:

@misc{huang2026onlinesimtorealadaptationclosedloop,
      title={Online Sim-to-Real Adaptation via Closed-Loop System Modeling},
      author={Yuhao Huang and Samuel A. Moore and Boyuan Chen},
      year={2026},
      eprint={2609.28878},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2609.28878},
}