Skip to content

fix(video): preserve immutable VideoToolbox output buffers - #876

Open
jonluca wants to merge 1 commit into
superuser404notfound:mainfrom
jonluca:fix/immutable-videotoolbox-output
Open

jonluca wants to merge 1 commit into
superuser404notfound:mainfrom
jonluca:fix/immutable-videotoolbox-output

Conversation

@jonluca

@jonluca jonluca commented Oct 25, 2026

Copy link
Copy Markdown

Summary

Respect VideoToolbox's ownership of decoded image buffers. The output callback previously changed color and pixel-aspect attachments even when imageBufferModifiable was absent, although the decoder can still retain that buffer as a reference picture.

What changed

  • Forward the callback's modifiability flag to a dedicated output metadata normalizer.
  • Reuse matching buffers; apply metadata in place only when allowed; otherwise copy through a locked transfer session and format-specific pixel buffer pool.
  • Preserve propagated attachments, clear stale pixel aspect ratios and retain the original frame if copying fails.
  • Keep the existing callback, HDR10+ delivery, decode progress and session-recovery paths intact.
  • Add regression coverage and document the ownership contract. Apple's callback contract is the basis for the change.

Test plan

  • Device / OS: Apple M4 Max, macOS 27.0.1, Xcode 27.1.
  • Source media: synthetic BGRA, NV12 and P010 CoreVideo buffers; no external file required. Tests exercise immutable/mutable outputs, no-copy reuse, stale PAR, attachment preservation, pixels, pool format changes and reset.
  • 59 tests across 7 focused output-metadata and HDR10+ suites pass, including both NV12/P010 pixel-preservation cases.
  • AetherEngine builds with -warnings-as-errors.
  • python3 Scripts/check-doc-links.py and git diff --check pass.
  • Hardware playback on iOS/tvOS was not exercised.

Checklist

  • CHANGELOG.md updated
  • Conventional commit
  • Fix lives in the engine
  • No public API change

Copilot AI balanced review requested due to automatic review settings October 25, 2026 16:27

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The attachment-propagation regression needs correction, and the platform-dependent transfer path still warrants human validation on iOS/tvOS.

1 open issue
What changed in this PR

This PR protects VideoToolbox-owned reference buffers while preserving the engine’s color and pixel-aspect metadata normalization.

Changes:

  • Forward callback modifiability flags and normalize metadata in place only when permitted.
  • Copy immutable buffers through a format-keyed pool, preserving propagated attachments and retaining the original on copy failure.
  • Add ownership and pixel-preservation regression tests and document the contract.
File Description
Tests/​AetherEngineTests/​Issue354HardwareDecoderPixelAspectTests.swift Adds ownership, attachment, pixel-copy, and pool-reset tests.
Sources/​AetherEngine/​Decoder/​VideoToolboxOutputMetadataNormalizer.swift Implements metadata matching, owned copies, and transfer fallback.
Sources/​AetherEngine/​Decoder/​HardwareVideoDecoder.swift Passes modifiability flags and integrates normalizer lifecycle.
docs/​architecture.md Documents buffer ownership and normalization behavior.
CHANGELOG.md Records the immutable-buffer fix.

Review effort: Balanced

Comment on lines +117 to +119
func attachment(_ key: CFString) -> CFTypeRef? {
CVBufferCopyAttachment(imageBuffer, key, nil)
}
@jonluca
jonluca force-pushed the fix/immutable-videotoolbox-output branch from a069980 to 9f764cd Compare October 25, 2026 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants