Skip to content

perf(subtitles): retain bitmap images with deferred PNG backing - #879

Open
qoli wants to merge 1 commit into
superuser404notfound:mainfrom
qoli:fix/subtitle-bitmap-storage
Open

qoli wants to merge 1 commit into
superuser404notfound:mainfrom
qoli:fix/subtitle-bitmap-storage

Conversation

@qoli

@qoli qoli commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

Summary

Retained bitmap subtitle history keeps every cropped cue expanded to RGBA. A previous ten-minute downstream PGS observation retained 2,185 images with approximately 289 MB of expanded image layout, including historical cues needed for subtitle offsets and backward seeks.

Retain each cropped bitmap through a lossless PNG-backed ImageIO provider and expand pixels when drawn. PNG encoding stays on the existing subtitle decode worker. This trades encoding/redraw work for smaller retained image backing without shortening cue history.

What changed

  • Preserve rendered pixels, alpha, crop, placement, canvas, forced flags and interpolation.
  • Add optional SubtitleImage.encodedBackingBytes for known compressed backing; document that it excludes framework overhead and temporary drawing pixels.
  • Document the CGImage drawing contract: callers must not assume a particular raw provider layout.
  • Compare actual rendered premultiplied RGBA in the existing palette regression, and cover all alpha values, provider lifetime, repeated drawing and compressed backing.
  • Update the changelog and public API documentation.

Test plan

Apple Silicon Mac14,14, macOS 27.0.1 (26A434), Xcode 27.1 (27A9275), Swift 6.4. Based on upstream 370b0e4dcc6fb49fca1b6999b075616d3442cb04; only the two subtitle production files, two test files and two documentation files are changed.

  • swift build: passed.
  • Focused subtitle/API regression: 6 XCTest cases and 28 Swift Testing tests passed.
  • LIBDISPATCH_COOPERATIVE_POOL_STRICT=1 focused subtitle regression: 28 tests in 5 suites passed.
  • python3 Scripts/check-doc-links.py and diff whitespace checks: passed.
  • Full default-traits run: 807 XCTest cases completed with zero failures and one skip; 5,011 Swift Testing tests in 695 suites completed with two issues:
    • Issue220SoftwareDecoderDrainTests.decodesFixtureFrames: counter.value >= packets - 16 failed.
    • HLSLocalServerSlowPeerTests.lanPeersLeaveLoopbackSlots: exceeded its existing 120-second time limit.

An isolated rerun of those two suites also reproduced both failures (12 tests in 2 suites). The failing tests and their production implementations are outside this diff. Their cause is not established by these runs, so this PR remains a draft rather than claiming a green full suite.

Tests use generated RGBA/palette images and the existing synthetic PGS fixtures, including independent primary/secondary palettes, clear/reselection and cached seek reconstruction. All four dependency versions and SHAs match upstream's checked-in Package.resolved; local official Git mirrors were used to avoid repeated downloads. Platform/all-traits CI and new physical-device playback or process-footprint measurements were not run locally.

Checklist

  • CHANGELOG.md updated
  • Conventional Commit
  • The change lives in the engine
  • Public API changes are intentional and documented

Keep authored subtitle history lossless while avoiding expanded RGBA storage
for every retained cue. Document the drawing contract and encoded backing
accounting, with pixel, provider lifetime and subtitle regressions.
@qoli
qoli marked this pull request as ready for review October 25, 2026 18:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant