Skip to content

feat(stills): add accurate downscaled preview extraction - #871

Open
jonluca wants to merge 1 commit into
superuser404notfound:mainfrom
jonluca:feat/accurate-frame-previews
Open

jonluca wants to merge 1 commit into
superuser404notfound:mainfrom
jonluca:feat/accurate-frame-previews

Conversation

@jonluca

@jonluca jonluca commented Oct 25, 2026

Copy link
Copy Markdown

Summary

Add FrameExtractor.preview(at:maxWidth:) for frame-accurate stills at a requested display width. This fills the gap between inexpensive keyframe thumbnails and native-resolution snapshots while keeping preview work separate from playback.

What changed

  • Add FrameMode.preview and the public actor method, with the same serial decoder and supersession/shutdown lifecycle as other extraction requests.
  • Decode forward to the target, retain loop filtering and full reconstruction for previews/snapshots, and use bicubic scaling for those accurate modes.
  • Keep a separate two-image preview LRU keyed by exact requested time and width, and yield uncached previews when playback reports starvation.
  • Add aetherctl extract --preview; reject combining it with --snapshot.
  • Preserve the existing bounded local/resident still paths, actual-PTS reporting, and container-origin-aware seek clamping.
  • Add embedded synthetic-media regressions and update API, architecture, CLI docs and changelog.

Test plan

  • Device / OS: Apple M4 Max, macOS 27.0.1, Xcode 27.1.
  • Source media: generated MP4 with MPEG-4 Part 2 video, 32x32, 4 fps, eight frames with only the first a keyframe; no audio, HDR or Dolby Vision. Fixture and regeneration command are in the tests.
  • Focused tests: 14 Swift Testing cases and 26 documentation XCTest cases passed. A request at 0.75 seconds decodes the 0.75-second frame rather than the keyframe at zero; preview pixels match an equally downscaled snapshot. Width changes, cache hits/eviction, early cancellation, starvation and shutdown are covered.
  • Related frame/still/scrub/documentation selection: 326 Swift Testing cases in 138 suites passed, plus 78 XCTest cases. Optional media-dependent tests skip when their fixtures are absent.
  • CLI: extract --at 0.75 --preview --width 16 produced a 16x16 PNG; mutually exclusive mode flags returned exit 64.
  • swift build --target AetherEngine -Xswiftc -warnings-as-errors, the documentation link checker and git diff --check passed. No iOS/tvOS device or HDR/Dolby Vision visual validation was performed for the new mode.

Checklist

  • CHANGELOG.md updated
  • Commit follows Conventional Commits
  • The feature lives in the engine
  • Public API changes are intentional and documented

Copilot AI balanced review requested due to automatic review settings October 25, 2026 16:25

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Successful HDR and Dolby Vision conversions bypass the documented accurate-mode scaling policy.

1 open issue
What changed in this PR

Adds frame-accurate, downscaled previews to the engine’s isolated still-extraction API, bridging keyframe thumbnails and full-resolution snapshots.

Changes:

  • Adds preview extraction with forward decoding, quality-aware scaling, cancellation, and starvation handling.
  • Introduces a separate two-image preview cache keyed by exact time and width.
  • Adds CLI support, synthetic-media regression tests, and documentation.
File Description
Tests/​AetherEngineTests/​FrameDecodeContextHardwareTests.swift Tests preview accuracy, dimensions, and lifecycle.
Tests/​AetherEngineTests/​FrameCacheTests.swift Tests preview cache keys and eviction.
Sources/​AetherEngine/​FrameExtractor/​FrameTypes.swift Adds preview mode and shared mode policies.
Sources/​AetherEngine/​FrameExtractor/​FrameExtractor.swift Exposes preview extraction and integrates caching.
Sources/​AetherEngine/​FrameExtractor/​FrameDecodeContext.swift Adds accurate preview decoding and scaling selection.
Sources/​AetherEngine/​FrameExtractor/​FrameCache.swift Adds a bounded, size-aware preview cache.
Sources/​aetherctl/​main.swift Parses preview flags and rejects conflicting modes.
Sources/​aetherctl/​Extract.swift Routes CLI requests to preview extraction.
docs/​cli.md Documents preview flags and usage.
docs/​architecture.md Includes preview mode in the extraction architecture.
docs/​api.md Documents preview behavior and lifecycle.
CHANGELOG.md Records the new preview API and CLI option.

Review effort: Balanced

// tone-map failed: fall through to sws path (degraded but non-nil)
}
return convertToCGImage(frame: f, targetWidth: width)
return convertToCGImage(frame: f, targetWidth: width, highQuality: mode.isFrameAccurate)
@jonluca
jonluca force-pushed the feat/accurate-frame-previews branch from 2f7a3b0 to 087ee88 Compare October 25, 2026 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants