Skip to content

Add native window single-frame capture - #700

Draft
yuecideng wants to merge 1 commit into
mainfrom
codex/issue-692-window-capture
Draft

yuecideng wants to merge 1 commit into
mainfrom
codex/issue-692-window-capture

Conversation

@yuecideng

Copy link
Copy Markdown
Contributor

Description

This PR adds default native-window single-frame capture support to SimulationManager.

  • Adds WindowCaptureCfg with an enabled-by-default C hotkey, configurable key, output path, and file prefix.
  • Adds SimulationManager.capture_window() and capture_window_image() with owned NumPy image results and PNG export.
  • Uses DexSim Windows.get_picture() when a native frame is available.
  • Falls back to an offscreen camera at the current native-window pose for renderers without CPU swapchain readback, including the DexSim 0.5.0 Hybrid and FastRT paths.
  • Rejects closed windows and empty frames without creating empty output files.
  • Documents the configuration and updates focused manager/config tests.

This is a documented EmbodiChain fallback for DexSim issue #231; no DexSim dependency version change is required.

Fixes #692

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (non-breaking change which improves an existing functionality)
  • Breaking change (existing functionality will not work without user modification)
  • Documentation update

Validation

  • black .
  • python docs/scripts/check_api_docs.py
  • python .agents/skills/project-dev-context/scripts/context.py check
  • python -m pytest -q -c /dev/null --noconftest tests/test_agent_context_map.py tests/test_agent_context_tools.py (40 passed)
  • Manual DexSim 0.5.0 Vulkan smoke checks for hybrid and fast-rt: native get_picture() returned an empty (0,) frame, while the current-pose offscreen fallback returned a non-empty (120, 160, 4) image.
  • Focused EmbodiChain simulation tests were not runnable in the current host because the system Python is 3.13 while the installed DexSim native package is CPython 3.11; changed files were syntax-compiled and the pure-Python capture harness passed.

Screenshots

Not applicable; the change is a simulation API and native-window control.

Checklist

  • I have run the black . command to format the code base.
  • I reviewed affected documentation and agent context, updated them where needed, or explained why no update was needed.
  • Public API changes are reflected in the API docs (python docs/scripts/check_api_docs.py), if applicable
  • I have added tests that prove my fix is effective or that my feature works
  • Dependencies have been updated, if applicable.

Add a configurable default capture hotkey and a public SimulationManager capture API. Use DexSim native readback when available and a current-pose offscreen fallback for renderers without CPU window readback.
@yuecideng yuecideng added enhancement New feature or request rendering Things related to rendering (eg, performace, efficiency, bug) visualization Features or functions for visualization with the simulation dexsim Things related to dexsim labels Sep 27, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dexsim Things related to dexsim enhancement New feature or request rendering Things related to rendering (eg, performace, efficiency, bug) visualization Features or functions for visualization with the simulation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Proposal] Add a default native-window capture event

1 participant