From df8fc1f8d137368ccddf41dc6596f6e3d7243090 Mon Sep 17 00:00:00 2001 From: JonLuca DeCaro Date: Sun, 25 Oct 2026 09:30:22 -0700 Subject: [PATCH] feat(video): decode supported VP9 streams with VideoToolbox --- CHANGELOG.md | 6 + README.md | 2 +- .../Decoder/HardwareVideoDecoder.swift | 37 ++- .../Decoder/VideoDecoderOpener.swift | 39 +++ .../Decoder/VideoDecoderTypes.swift | 10 +- .../Decoder/VideoToolboxVideoFormat.swift | 246 ++++++++++++++++++ .../Native/SoftwarePlaybackHost.swift | 38 ++- .../ActiveVideoDecoderTests.swift | 19 +- .../SoftwareDecoderSelectionTests.swift | 132 +++++++++- docs/architecture.md | 12 +- docs/formats.md | 2 +- 11 files changed, 507 insertions(+), 36 deletions(-) create mode 100644 Sources/AetherEngine/Decoder/VideoDecoderOpener.swift create mode 100644 Sources/AetherEngine/Decoder/VideoToolboxVideoFormat.swift diff --git a/CHANGELOG.md b/CHANGELOG.md index cadfbc99f..d14400df5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,12 @@ the public-API contract. ### Added +- **VP9 can use VideoToolbox hardware decoding on the software playback host.** WebM streams + synthesize the required `vpcC` record from probed parameters. The hardware candidate accepts + 8-bit profile 0 and 10-bit profile 2 with 4:2:0 chroma, requires a hardware session, and falls + back to libavcodec if registration, configuration or session creation fails. Decoder diagnostics + report the implementation that actually opened. + - **Remote MPEG-DASH on-demand manifests play (AE#859, thanks @qoli).** A static, single-Period MPD whose representations are single files addressed by `SegmentBase` failed at open with `Invalid data found when processing input`: the FFmpeg build has no DASH diff --git a/README.md b/README.md index 636bb2969..5fea092d7 100644 --- a/README.md +++ b/README.md @@ -66,7 +66,7 @@ A scannable summary; the depth for each row lives in **[docs/formats.md](docs/fo | MPEG-DASH | Remote on-demand manifests (static, one Period, `SegmentBase`, unencrypted): the video and every audio language are laid out as one fragmented MP4, so seeks, audio switching and both playback paths work unchanged. Live and `SegmentTemplate` manifests are refused with a reason | | Disc | DVD-Video and Blu-ray ISO (decrypted): selectable titles and chapters, demuxed through the normal path | | Video (HW) | H.264, HEVC, HEVC Main10 via VideoToolbox; AV1 Main profile where HW AV1 exists | -| Video (SW) | AV1 (dav1d) without HW, and AV1 High / Professional (4:4:4, 4:2:2, 12-bit) everywhere, VP9 / VP8, MPEG-4 Part 2 / MPEG-2 / VC-1, QuickTime RLE, the Flash tail (Sorenson Spark, On2 VP6), RealVideo 1-4, H.263, Theora and anything else the FFmpeg build carries a decoder for (software is the default route; only HEVC, H.264 and HW-decodable AV1 go native), H.264 High 4:2:2 / 4:4:4 / 10 and HEVC Rext where VideoToolbox has no HW decoder (Intel Macs, older chips), interlaced H.264 (AVPlayer does not deinterlace; on VOD the declared field order is verified against decoded frames, so progressive-in-interlaced-carriage keeps hardware decode); GPU deinterlace (yadif_videotoolbox, Metal, field-rate by default) with a CPU bwdif fallback | +| Video (SW) | AV1 (dav1d) without HW, and AV1 High / Professional (4:4:4, 4:2:2, 12-bit) everywhere, VP9 (VideoToolbox hardware when supported, otherwise libavcodec) / VP8, MPEG-4 Part 2 / MPEG-2 / VC-1, QuickTime RLE, the Flash tail (Sorenson Spark, On2 VP6), RealVideo 1-4, H.263, Theora and anything else the FFmpeg build carries a decoder for (software is the default route; only HEVC, H.264 and HW-decodable AV1 go native), H.264 High 4:2:2 / 4:4:4 / 10 and HEVC Rext where VideoToolbox has no HW decoder (Intel Macs, older chips), interlaced H.264 (AVPlayer does not deinterlace; on VOD the declared field order is verified against decoded frames, so progressive-in-interlaced-carriage keeps hardware decode); GPU deinterlace (yadif_videotoolbox, Metal, field-rate by default) with a CPU bwdif fallback | | HDR | HDR10, HDR10+ (per-frame ST 2094-40), Dolby Vision (P5, P7 on the native route as single-layer 8.1 (HDR10 base layer on a display without Dolby Vision), or composed dual-layer on Metal with per-frame DV metadata for FEL when `preferDualLayerDolbyVision` is set, the `DualLayerDolbyVision` trait is on and VideoToolbox decodes HEVC in hardware, P8.1, P8.4, AV1 P10.x), HLG (declared, or named by an alternative transfer SEI as broadcast HLG is), HDR Vivid (played on its HLG / PQ base, CUVA metadata reported by `probe`) | | Audio | AAC, AC3, EAC3, FLAC, ALAC stream-copy lossless; TrueHD / MLP / DTS / DTS-HD MA / MP3 / MP2 / Opus / Vorbis / LPCM (incl. Blu-ray, G.711) / WMA (Standard, Pro, Lossless, Voice) / Nellymoser / ADPCM-SWF / Speex / RealAudio (Cook, ATRAC3, Sipro, Lossless, 14.4 / 28.8) / AMR-NB / AMR-WB / DVD LPCM bridge to EAC3 5.1 (default) or lossless FLAC | | Dolby Atmos | EAC3+JOC stream-copied on every route (HDMI MAT 2.0, AirPods spatial, BT downmix) except the opt-in Profile 7 FEL dual-layer route (`preferDualLayerDolbyVision`, `DualLayerDolbyVision` trait), which runs on the software host and plays an E-AC-3 JOC track as its 5.1 / 7.1 bed without the objects unless `LoadOptions.objectAudioRendering` is set (then the objects are kept: compressed JOC to the renderer, TrueHD Atmos and DTS:X as APAC, in a build with the object-audio traits). No container reliably declares JOC pre-decode, so an honest `TrackInfo.isAtmos` needs a bounded decode: `AetherEngine.probeDetectingAtmos(url:/source:)` answers for a details screen without starting playback, and `LoadOptions.confirmAtmos` has the running session confirm its own tracks in the background and republish `audioTracks`. Both are opt-in and neither sits on the playback-start path. TrueHD Atmos keeps its heights too, opt-in through `LoadOptions.objectAudioRendering` in a build with the `TrueHDObjectAudio` trait ([Optional components](#optional-components)): the engine decodes the Atmos objects, renders them into a speaker bed and delivers Apple Positional Audio, which tvOS sends to an Atmos receiver as Dolby MAT and iOS plays as Spatial Audio ([details](docs/formats.md#truehd-atmos-with-heights)). DTS:X takes the same route with the `DTSXObjectAudio` trait, its four heights unfolded from the compatible 7.1 into 7.1.4 or 9.1.6 ([details](docs/formats.md#dtsx-with-heights)) | diff --git a/Sources/AetherEngine/Decoder/HardwareVideoDecoder.swift b/Sources/AetherEngine/Decoder/HardwareVideoDecoder.swift index f919bf767..4c793cdf3 100644 --- a/Sources/AetherEngine/Decoder/HardwareVideoDecoder.swift +++ b/Sources/AetherEngine/Decoder/HardwareVideoDecoder.swift @@ -6,7 +6,7 @@ import AetherLibavformat import AetherLibavcodec import AetherLibavutil -/// VTDecompressionSession-backed HEVC decoder for the SoftwarePlaybackHost pipeline. +/// VTDecompressionSession-backed HEVC / VP9 decoder for the SoftwarePlaybackHost pipeline. /// Owns the decoded-frame pool, IOSurface lifetime, and session teardown explicitly /// (AVPlayer's opaque state grows unbounded on long 4K HDR sessions). /// Same surface as SoftwareVideoDecoder so the host can swap without rewiring the demux loop. @@ -79,6 +79,7 @@ final class HardwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { private var timeBase: AVRational = AVRational(num: 1, den: 90000) private var width: Int32 = 0 private var height: Int32 = 0 + private var codecName = "HEVC" /// Color metadata from codecpar, re-applied to every CVPixelBuffer. /// VTDecompressionSession should propagate these from SPS+hvcC but has been observed not to; @@ -273,27 +274,25 @@ final class HardwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { ) } - guard codecpar.pointee.codec_id == AV_CODEC_ID_HEVC else { - throw VideoDecoderError.unsupportedCodec(id: codecpar.pointee.codec_id.rawValue) + let configuration = try VideoToolboxVideoFormat.make(stream: stream) + codecName = configuration.codecName + if codecpar.pointee.codec_id == AV_CODEC_ID_VP9 { + if #available(macOS 11.0, iOS 26.2, tvOS 26.2, visionOS 26.2, *) { + VTRegisterSupplementalVideoDecoderIfAvailable(kCMVideoCodecType_VP9) + } + guard VTIsHardwareDecodeSupported(kCMVideoCodecType_VP9) else { + throw VideoDecoderError.hardwareDecoderUnavailable(codec: "VP9") + } } - // 1. Build CMVideoFormatDescription from the hvcC extradata via - // kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms (same shape AVFoundation uses for .mp4/.mkv). - guard let extradata = codecpar.pointee.extradata, codecpar.pointee.extradata_size > 0 else { - throw VideoDecoderError.noExtradata - } - let hvcCData = Data(bytes: extradata, count: Int(codecpar.pointee.extradata_size)) + // WebM VP9 has no extradata; its vpcC record comes from the probed stream parameters. var fd: CMVideoFormatDescription? - let atomsDict: NSDictionary = ["hvcC": hvcCData] - let extensions: NSDictionary = [ - kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms: atomsDict, - ] let fdStatus = CMVideoFormatDescriptionCreate( allocator: kCFAllocatorDefault, - codecType: kCMVideoCodecType_HEVC, + codecType: configuration.codecType, width: width, height: height, - extensions: extensions, + extensions: configuration.extensions, formatDescriptionOut: &fd ) guard fdStatus == noErr, let formatDesc = fd else { @@ -308,16 +307,14 @@ final class HardwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { ] // 3. Pixel buffer attributes: 10-bit biplanar for HDR, 8-bit for SDR; IOSurface-backed for Metal rendering. - let bitsPerSample = codecpar.pointee.bits_per_raw_sample - let isHDRTransfer = ColorAttachments.isHDRTransfer(codecpar.pointee.color_trc) - let use10Bit = bitsPerSample > 8 || isHDRTransfer + let use10Bit = configuration.bitDepth > 8 streamColor = ColorDescription(codecpar: codecpar) streamCodecID = codecpar.pointee.codec_id streamProfile = codecpar.pointee.profile scansHDR10Plus = codecpar.pointee.color_trc == AVCOL_TRC_SMPTE2084 hdr10PlusFraming = .lengthPrefixed(size: NALUnitChain.lengthPrefixSize( - codecID: streamCodecID, extradata: extradata, + codecID: streamCodecID, extradata: codecpar.pointee.extradata, extradataSize: Int(codecpar.pointee.extradata_size)) ?? 4) self.colorPrimaries = ColorAttachments.primaries(codecpar.pointee.color_primaries) self.colorTransfer = ColorAttachments.transfer(codecpar.pointee.color_trc) @@ -349,7 +346,7 @@ final class HardwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { session = createdSession EngineLog.emit( - "[HardwareVideoDecoder] opened HEVC \(width)x\(height) " + "[HardwareVideoDecoder] opened \(codecName) \(width)x\(height) " + "\(use10Bit ? "10-bit" : "8-bit") " + "transfer=\(codecpar.pointee.color_trc.rawValue)", category: .swPlayback diff --git a/Sources/AetherEngine/Decoder/VideoDecoderOpener.swift b/Sources/AetherEngine/Decoder/VideoDecoderOpener.swift new file mode 100644 index 000000000..37670a661 --- /dev/null +++ b/Sources/AetherEngine/Decoder/VideoDecoderOpener.swift @@ -0,0 +1,39 @@ +import Foundation +import CoreMedia +import AetherLibavformat + +/// Opens a preferred decoder and atomically falls back before the demux loop captures its pipeline. +enum VideoDecoderOpener { + struct Result { + let decoder: any VideoDecodingPipeline + let preferredFailureDescription: String? + + var usedFallback: Bool { preferredFailureDescription != nil } + } + + static func openPreferredOrFallback( + preferred: any VideoDecodingPipeline, + makeFallback: () -> any VideoDecodingPipeline, + stream: UnsafeMutablePointer, + onFrame: @escaping DecodedFrameHandler + ) throws -> Result { + do { + try preferred.open(stream: stream, onFrame: onFrame) + return Result(decoder: preferred, preferredFailureDescription: nil) + } catch { + preferred.close() + let preferredFailureDescription = error.localizedDescription + let fallback = makeFallback() + do { + try fallback.open(stream: stream, onFrame: onFrame) + return Result( + decoder: fallback, + preferredFailureDescription: preferredFailureDescription + ) + } catch let fallbackError { + fallback.close() + throw fallbackError + } + } + } +} diff --git a/Sources/AetherEngine/Decoder/VideoDecoderTypes.swift b/Sources/AetherEngine/Decoder/VideoDecoderTypes.swift index 0f801a3cc..f66ded41d 100644 --- a/Sources/AetherEngine/Decoder/VideoDecoderTypes.swift +++ b/Sources/AetherEngine/Decoder/VideoDecoderTypes.swift @@ -9,8 +9,8 @@ import AetherLibavcodec /// (kCMSampleAttachmentKey_HDR10PlusPerFrameData format); nil for non-HDR10+ streams. typealias DecodedFrameHandler = @Sendable (CVPixelBuffer, CMTime, Data?) -> Void -/// Common video decoder protocol. SoftwareVideoDecoder (libavcodec, AV1/VP9) and -/// HardwareVideoDecoder (VTDecompressionSession, HEVC) both conform; the host swaps per codec without rewiring the demux loop. +/// Common video decoder protocol. SoftwareVideoDecoder (libavcodec) and +/// HardwareVideoDecoder (VTDecompressionSession, HEVC/VP9) both conform; the host swaps per codec without rewiring the demux loop. // Sendable: both conformers (SoftwareVideoDecoder, HardwareVideoDecoder) are @unchecked Sendable // (internally lock-guarded), so `any VideoDecodingPipeline` is safe to capture in @Sendable closures. protocol VideoDecodingPipeline: AnyObject, Sendable { @@ -58,6 +58,8 @@ enum VideoDecoderError: Error, LocalizedError { case noCodecParameters case unsupportedCodec(id: UInt32) case noExtradata + case unsupportedConfiguration(codec: String, detail: String) + case hardwareDecoderUnavailable(codec: String) case formatDescriptionFailed(status: OSStatus) case sessionCreationFailed(status: OSStatus) /// `DualLayerVideoDecoder` is single use (dolby-vision-fel.md §5.8): `open` after `close` refuses @@ -75,6 +77,10 @@ enum VideoDecoderError: Error, LocalizedError { case .noCodecParameters: "No codec parameters" case .unsupportedCodec(let id): "Unsupported video codec (id: \(id))" case .noExtradata: "Missing codec extradata" + case .unsupportedConfiguration(let codec, let detail): + "Unsupported \(codec) configuration: \(detail)" + case .hardwareDecoderUnavailable(let codec): + "Hardware \(codec) decoder unavailable" case .formatDescriptionFailed(let s): "Format description failed (\(s))" case .sessionCreationFailed(let s): "Decoder session failed (\(s))" case .decoderClosed: "Decoder is closed (single use)" diff --git a/Sources/AetherEngine/Decoder/VideoToolboxVideoFormat.swift b/Sources/AetherEngine/Decoder/VideoToolboxVideoFormat.swift new file mode 100644 index 000000000..932711381 --- /dev/null +++ b/Sources/AetherEngine/Decoder/VideoToolboxVideoFormat.swift @@ -0,0 +1,246 @@ +import Foundation +import CoreMedia +import CoreVideo +import AetherLibavcodec +import AetherLibavformat +import AetherLibavutil + +/// Builds the codec-specific Core Media description consumed by `HardwareVideoDecoder`. +/// +/// VP9 in WebM has no codec initialization payload. VideoToolbox nevertheless requires an ISO +/// `vpcC` configuration record, so it is synthesized from FFmpeg's probed stream parameters rather +/// than treating the expected lack of `AVCodecParameters.extradata` as an error. +enum VideoToolboxVideoFormat { + struct Configuration { + let codecName: String + let codecType: CMVideoCodecType + let extensions: NSDictionary + let bitDepth: Int + } + + struct VP9Configuration: Equatable { + let profile: UInt8 + let level: UInt8 + let bitDepth: UInt8 + let chromaSubsampling: UInt8 + let fullRange: Bool + let colorPrimaries: UInt8 + let transferCharacteristics: UInt8 + let matrixCoefficients: UInt8 + + /// VP Codec ISO Media File Format Binding `vpcC` atom payload: FullBox version/flags followed by the + /// eight-byte VPCodecConfigurationRecord. Core Media's sample-description atom dictionary + /// consumes this complete box payload, as emitted by FFmpeg's `ff_isom_write_vpcc`. + var configurationBoxPayload: Data { + Data([ + 1, 0, 0, 0, // FullBox version 1 + flags + profile, + level, + (bitDepth << 4) | (chromaSubsampling << 1) | (fullRange ? 1 : 0), + colorPrimaries, + transferCharacteristics, + matrixCoefficients, + 0, 0 // codecInitializationDataSize; VP9 has no initialization data + ]) + } + } + + static func make(stream: UnsafeMutablePointer) throws -> Configuration { + guard let codecpar = stream.pointee.codecpar else { + throw VideoDecoderError.noCodecParameters + } + let pixelAspectRatio = pixelAspectRatio(codecpar: codecpar, stream: stream) + + switch codecpar.pointee.codec_id { + case AV_CODEC_ID_HEVC: + guard let extradata = codecpar.pointee.extradata, + codecpar.pointee.extradata_size > 0 else { + throw VideoDecoderError.noExtradata + } + let data = Data(bytes: extradata, count: Int(codecpar.pointee.extradata_size)) + let atoms: NSDictionary = ["hvcC": data] + let bitDepth = codecpar.pointee.bits_per_raw_sample > 8 + || ColorAttachments.isHDRTransfer(codecpar.pointee.color_trc) ? 10 : 8 + return Configuration( + codecName: "HEVC", + codecType: kCMVideoCodecType_HEVC, + extensions: [kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms: atoms], + bitDepth: bitDepth + ) + + case AV_CODEC_ID_VP9: + let vp9 = try vp9Configuration( + codecpar: UnsafePointer(codecpar), + frameRate: stream.pointee.avg_frame_rate + ) + let atoms: NSDictionary = ["vpcC": vp9.configurationBoxPayload] + return Configuration( + codecName: "VP9", + codecType: kCMVideoCodecType_VP9, + extensions: extensions(atoms: atoms, pixelAspectRatio: pixelAspectRatio), + bitDepth: Int(vp9.bitDepth) + ) + + default: + throw VideoDecoderError.unsupportedCodec(id: codecpar.pointee.codec_id.rawValue) + } + } + + private static func pixelAspectRatio( + codecpar: UnsafeMutablePointer, + stream: UnsafeMutablePointer + ) -> AVRational? { + PixelAspectPolicy.declaredPixelAspect( + bitstream: codecpar.pointee.sample_aspect_ratio, + container: stream.pointee.sample_aspect_ratio, + width: codecpar.pointee.width, + height: codecpar.pointee.height + ) + } + + private static func extensions( + atoms: NSDictionary, + pixelAspectRatio: AVRational? + ) -> NSDictionary { + let extensions = NSMutableDictionary(dictionary: [ + kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms: atoms + ]) + if let pixelAspectRatio { + extensions[kCMFormatDescriptionExtension_PixelAspectRatio] = [ + kCMFormatDescriptionKey_PixelAspectRatioHorizontalSpacing: + Int(pixelAspectRatio.num), + kCMFormatDescriptionKey_PixelAspectRatioVerticalSpacing: + Int(pixelAspectRatio.den) + ] as NSDictionary + } + return extensions + } + + static func vp9Configuration( + codecpar: UnsafePointer, + frameRate: AVRational + ) throws -> VP9Configuration { + let parameters = codecpar.pointee + let pixelDescription = av_pix_fmt_desc_get(AVPixelFormat(rawValue: parameters.format)) + let describedBitDepth = pixelDescription.map { Int($0.pointee.comp.0.depth) } + let bitDepth = describedBitDepth ?? { + let raw = Int(parameters.bits_per_raw_sample) + if raw > 0 { return raw } + switch parameters.profile { + case 0, 1: return 8 + // Profiles 2/3 permit both 10 and 12 bits; do not guess without a probed depth. + case 2, 3: return 0 + default: return 0 + } + }() + + let profile: Int + if parameters.profile >= 0 { + profile = Int(parameters.profile) + } else if bitDepth == 8 { + profile = 0 + } else if bitDepth == 10 { + profile = 2 + } else { + throw VideoDecoderError.unsupportedConfiguration( + codec: "VP9", detail: "profile and bit depth are unknown" + ) + } + + guard profile == 0 || profile == 2 else { + throw VideoDecoderError.unsupportedConfiguration( + codec: "VP9", detail: "profile \(profile) is not a hardware 4:2:0 profile" + ) + } + guard (profile == 0 && bitDepth == 8) || (profile == 2 && bitDepth == 10) else { + throw VideoDecoderError.unsupportedConfiguration( + codec: "VP9", detail: "profile \(profile) with \(bitDepth)-bit samples" + ) + } + if let pixelDescription { + guard pixelDescription.pointee.log2_chroma_w == 1, + pixelDescription.pointee.log2_chroma_h == 1 else { + throw VideoDecoderError.unsupportedConfiguration( + codec: "VP9", detail: "only 4:2:0 chroma is hardware-decodable" + ) + } + } + + let level = parameters.level >= 0 + ? UInt8(clamping: parameters.level) + : estimatedVP9Level( + width: parameters.width, + height: parameters.height, + frameRate: frameRate + ) + // vpcC represents left (vertical) and top-left (colocated) 4:2:0 only. Do not + // relabel an explicitly centered source; let libavcodec handle that declaration. + let chromaSubsampling: UInt8 + switch parameters.chroma_location { + case AVCHROMA_LOC_LEFT: chromaSubsampling = 0 + case AVCHROMA_LOC_UNSPECIFIED, AVCHROMA_LOC_TOPLEFT: chromaSubsampling = 1 + default: + throw VideoDecoderError.unsupportedConfiguration( + codec: "VP9", detail: "chroma location is not representable in vpcC") + } + guard parameters.color_space != AVCOL_SPC_RGB else { + throw VideoDecoderError.unsupportedConfiguration( + codec: "VP9", detail: "RGB signaling requires a 4:4:4 profile") + } + + return VP9Configuration( + profile: UInt8(profile), + level: level, + bitDepth: UInt8(bitDepth), + chromaSubsampling: chromaSubsampling, + fullRange: parameters.color_range == AVCOL_RANGE_JPEG, + colorPrimaries: UInt8(clamping: parameters.color_primaries.rawValue), + transferCharacteristics: UInt8(clamping: parameters.color_trc.rawValue), + matrixCoefficients: UInt8(clamping: parameters.color_space.rawValue) + ) + } + + /// Mirrors FFmpeg's `get_vp9_level`: first matching luma sample-rate and picture-size ceiling. + private static func estimatedVP9Level( + width: Int32, + height: Int32, + frameRate: AVRational + ) -> UInt8 { + let pictureSize = Int64(max(width, 0)) * Int64(max(height, 0)) + guard pictureSize > 0 else { return 0 } + let sampleRate: Int64 + if frameRate.num > 0, frameRate.den > 0 { + let (product, overflow) = pictureSize.multipliedReportingOverflow( + by: Int64(frameRate.num) + ) + sampleRate = overflow ? Int64.max : product / Int64(frameRate.den) + } else { + sampleRate = 0 + } + + struct LevelLimit { + let level: UInt8 + let sampleRate: Int64 + let pictureSize: Int64 + } + let limits = [ + LevelLimit(level: 10, sampleRate: 829_440, pictureSize: 36_864), + LevelLimit(level: 11, sampleRate: 2_764_800, pictureSize: 73_728), + LevelLimit(level: 20, sampleRate: 4_608_000, pictureSize: 122_880), + LevelLimit(level: 21, sampleRate: 9_216_000, pictureSize: 245_760), + LevelLimit(level: 30, sampleRate: 20_736_000, pictureSize: 552_960), + LevelLimit(level: 31, sampleRate: 36_864_000, pictureSize: 983_040), + LevelLimit(level: 40, sampleRate: 83_558_400, pictureSize: 2_228_224), + LevelLimit(level: 41, sampleRate: 160_432_128, pictureSize: 2_228_224), + LevelLimit(level: 50, sampleRate: 311_951_360, pictureSize: 8_912_896), + LevelLimit(level: 51, sampleRate: 588_251_136, pictureSize: 8_912_896), + LevelLimit(level: 52, sampleRate: 1_176_502_272, pictureSize: 8_912_896), + LevelLimit(level: 60, sampleRate: 1_176_502_272, pictureSize: 35_651_584), + LevelLimit(level: 61, sampleRate: 2_353_004_544, pictureSize: 35_651_584), + LevelLimit(level: 62, sampleRate: 4_706_009_088, pictureSize: 35_651_584) + ] + return limits.first { + sampleRate <= $0.sampleRate && pictureSize <= $0.pictureSize + }?.level ?? 0 + } +} diff --git a/Sources/AetherEngine/Native/SoftwarePlaybackHost.swift b/Sources/AetherEngine/Native/SoftwarePlaybackHost.swift index 54fe47192..b5789e384 100644 --- a/Sources/AetherEngine/Native/SoftwarePlaybackHost.swift +++ b/Sources/AetherEngine/Native/SoftwarePlaybackHost.swift @@ -270,7 +270,7 @@ final class SoftwarePlaybackHost { // MARK: - Internals private let renderer: SampleBufferRenderer - /// Swapped per codec at load(): SoftwareVideoDecoder for AV1/VP9, HardwareVideoDecoder for HEVC. Protocol keeps the demux loop codec-agnostic. + /// Swapped per codec at load(): SoftwareVideoDecoder for AV1 and unsupported hardware formats, HardwareVideoDecoder for HEVC/VP9. Protocol keeps the demux loop codec-agnostic. private var videoDecoder: any VideoDecodingPipeline private var audioDecoder: (any SoftwareAudioProducing)? /// How this session's audio reaches the renderer (`SoftwareAtmosRoute`), for the logs and the tests. @@ -1104,6 +1104,9 @@ final class SoftwarePlaybackHost { canHardwareDecode: Bool, dvProfile: Int? ) -> any VideoDecodingPipeline { + if codecID == AV_CODEC_ID_VP9 && canHardwareDecode { + return HardwareVideoDecoder() + } if codecID == AV_CODEC_ID_HEVC && canHardwareDecode { if Self.shouldUseDualLayerDecoder( dvProfile: dvProfile, @@ -1219,15 +1222,17 @@ final class SoftwarePlaybackHost { guard !stopRequested else { return } } - // HEVC -> VTDecompressionSession (HW) when VideoToolbox PROVABLY HW-decodes this exact format; + // VP9 tries a hardware session with a software fallback at open. HEVC uses VideoToolbox + // only when the probe proves that this exact format opens a hardware session; // everything else -> libavcodec. Replace wholesale to prevent state bleed. #2: HardwareVideoDecoder // requires a HW decoder and has no software fallback, so an HEVC Rext stream on hardware without a // HW decoder must go to libavcodec. AE#461: so must every format the probe cannot classify (in-band // parameter sets, Annex-B extradata), which the routing gate reads as "keep native" but which // HardwareVideoDecoder, building from the hvcC alone, can never open. if let codecpar = vStream.pointee.codecpar, - codecpar.pointee.codec_id == AV_CODEC_ID_HEVC, - VTCapabilityProbe.hardwareDecodeVerdict(codecpar: codecpar).opensHardwareDecoder { + codecpar.pointee.codec_id == AV_CODEC_ID_VP9 + || (codecpar.pointee.codec_id == AV_CODEC_ID_HEVC + && VTCapabilityProbe.hardwareDecodeVerdict(codecpar: codecpar).opensHardwareDecoder) { videoDecoder.close() let dvProfile = configuredDVProfile ?? AetherEngine.dvProfile(stream: vStream) let chosen = selectVideoDecoder( @@ -1261,7 +1266,7 @@ final class SoftwarePlaybackHost { ) } else { EngineLog.emit( - "[SWHost] selected HardwareVideoDecoder (VT HEVC) for codec_id=\(codecpar.pointee.codec_id.rawValue)", + "[SWHost] selected HardwareVideoDecoder (VT) for codec_id=\(codecpar.pointee.codec_id.rawValue)", category: .swPlayback ) } @@ -1306,7 +1311,7 @@ final class SoftwarePlaybackHost { // Applied here (not init) so it also covers a decoder replaced above. (videoDecoder as? SoftwareVideoDecoder)?.deinterlaceConfig = deinterlaceConfig - try videoDecoder.open(stream: vStream) { [weak self] pixelBuffer, pts, hdr10PlusData in + let onFrame: DecodedFrameHandler = { [weak self] pixelBuffer, pts, hdr10PlusData in guard let self else { return } // AE#491: a frame decoded from a pre-seek packet is not late, it is from somewhere else. guard self.decodeGeneration == self.seekGeneration else { return } @@ -1332,6 +1337,27 @@ final class SoftwarePlaybackHost { } } + if vStream.pointee.codecpar?.pointee.codec_id == AV_CODEC_ID_VP9 { + let opened = try VideoDecoderOpener.openPreferredOrFallback( + preferred: videoDecoder, + makeFallback: { + let decoder = SoftwareVideoDecoder() + decoder.deinterlaceConfig = self.deinterlaceConfig + return decoder + }, + stream: vStream, + onFrame: onFrame) + videoDecoder = opened.decoder + renderer.setReorderDepth(SampleBufferRenderer.reorderDepth( + forHardwareDecoder: videoDecoder is HardwareVideoDecoder)) + if let reason = opened.preferredFailureDescription { + EngineLog.emit("[SWHost] VP9 hardware open failed; using libavcodec: \(reason)", + category: .swPlayback) + } + } else { + try videoDecoder.open(stream: vStream, onFrame: onFrame) + } + videoDecoder.onFirstHDR10PlusDetected = { [weak self] in self?.onFirstHDR10PlusDetected?() } diff --git a/Tests/AetherEngineTests/ActiveVideoDecoderTests.swift b/Tests/AetherEngineTests/ActiveVideoDecoderTests.swift index 8c17146a4..e8f16d0cf 100644 --- a/Tests/AetherEngineTests/ActiveVideoDecoderTests.swift +++ b/Tests/AetherEngineTests/ActiveVideoDecoderTests.swift @@ -19,7 +19,7 @@ struct ActiveVideoDecoderTests { FileManager.default.fileExists(atPath: fixture(name).path) } - private func checkLoadAndReload(_ name: String, label: String) async throws { + private func checkLoadAndReload(_ name: String, label: String, requiresFrames: Bool = false) async throws { let engine = try AetherEngine() defer { engine.stop() } var options = LoadOptions() @@ -29,6 +29,7 @@ struct ActiveVideoDecoderTests { // reloadWithAudioOverride path, rather than calling the initial URL load again. let reader = DataIOReader(data: try Data(contentsOf: Self.fixture(name))) _ = try await engine.load(source: .custom(reader), options: options) + if requiresFrames { try await waitFor { engine.softwareHostFramesEnqueued > 0 } } engine.pause() #expect(engine.playbackBackend == .software) #expect(engine.activeVideoDecoder == label, "initial load must name the actual decoder") @@ -36,6 +37,11 @@ struct ActiveVideoDecoderTests { _ = try await engine.reloadAtCurrentPosition() #expect(engine.playbackBackend == .software) #expect(engine.activeVideoDecoder == label, "reload must name the newly opened decoder") + if requiresFrames { + // Reload preserves the pause above; a fresh host starts reading only when played. + engine.play() + try await waitFor { engine.softwareHostFramesEnqueued > 0 } + } } @Test("HEVC decoded by VideoToolbox on the software path (#753)", @@ -51,10 +57,15 @@ struct ActiveVideoDecoderTests { try await checkLoadAndReload("hev1-inband-xps.mp4", label: "libavcodec HEVC (SW)") } - @Test("VP9 still reports libavcodec", + @Test("VP9 reports the hardware decoder after load and reload", .enabled(if: hasFixture("vp9.webm")), .timeLimit(.minutes(1))) - func softwareVP9() async throws { - try await checkLoadAndReload("vp9.webm", label: "libavcodec VP9 (SW)") + func hardwareOrFallbackVP9() async throws { + if #available(macOS 11.0, iOS 26.2, tvOS 26.2, visionOS 26.2, *) { + VTRegisterSupplementalVideoDecoderIfAvailable(kCMVideoCodecType_VP9) + } + let label = VTIsHardwareDecodeSupported(kCMVideoCodecType_VP9) + ? "VideoToolbox VP9 (HW)" : "libavcodec VP9 (SW)" + try await checkLoadAndReload("vp9.webm", label: label, requiresFrames: true) } @Test("AV1 still reports dav1d", diff --git a/Tests/AetherEngineTests/SoftwareDecoderSelectionTests.swift b/Tests/AetherEngineTests/SoftwareDecoderSelectionTests.swift index 07de672c1..32d24d14c 100644 --- a/Tests/AetherEngineTests/SoftwareDecoderSelectionTests.swift +++ b/Tests/AetherEngineTests/SoftwareDecoderSelectionTests.swift @@ -1,4 +1,8 @@ import Testing +import Foundation +import CoreMedia +import AetherLibavformat +import AetherLibavutil import AetherLibavcodec @testable import AetherEngine @@ -144,7 +148,7 @@ struct SoftwareDecoderSelectionTests { } @MainActor - @Test("Non-HEVC or unsupported hardware decode falls back to SoftwareVideoDecoder") + @Test("AV1 and unsupported hardware decode use SoftwareVideoDecoder") func nonHardwareFallback() { let host = SoftwarePlaybackHost() host.elType = "FEL" @@ -189,4 +193,130 @@ struct SoftwareDecoderSelectionTests { dvProfile: nil, elType: "FEL", preferDualLayerDolbyVision: true, dolbyVisionHandling: .automatic )) } + @MainActor + @Test("VP9 tries hardware without changing the software presentation route") + func vp9SelectsHardwareCandidate() { + let host = SoftwarePlaybackHost() + #expect(host.selectVideoDecoder(codecID: AV_CODEC_ID_VP9, + canHardwareDecode: true, dvProfile: nil) is HardwareVideoDecoder) + #expect(host.selectVideoDecoder(codecID: AV_CODEC_ID_VP9, + canHardwareDecode: false, dvProfile: nil) is SoftwareVideoDecoder) + } + + private func withVP9Stream(_ body: (UnsafeMutablePointer) throws -> Void) throws { + let context = try #require(avformat_alloc_context()) + defer { avformat_free_context(context) } + let stream = try #require(avformat_new_stream(context, nil)) + let parameters = try #require(stream.pointee.codecpar) + parameters.pointee.codec_id = AV_CODEC_ID_VP9 + parameters.pointee.codec_type = AVMEDIA_TYPE_VIDEO + parameters.pointee.width = 1920 + parameters.pointee.height = 1080 + parameters.pointee.profile = 0 + parameters.pointee.format = AV_PIX_FMT_YUV420P.rawValue + parameters.pointee.color_primaries = AVCOL_PRI_BT709 + parameters.pointee.color_trc = AVCOL_TRC_BT709 + parameters.pointee.color_space = AVCOL_SPC_BT709 + parameters.pointee.chroma_location = AVCHROMA_LOC_LEFT + stream.pointee.avg_frame_rate = AVRational(num: 24, den: 1) + stream.pointee.time_base = AVRational(num: 1, den: 1000) + try body(stream) + } + + @Test("WebM VP9 produces vpcC without codec extradata") + func webMVP9Configuration() throws { + try withVP9Stream { stream in + #expect(stream.pointee.codecpar.pointee.extradata == nil) + let configuration = try VideoToolboxVideoFormat.make(stream: stream) + #expect(configuration.codecName == "VP9") + #expect(configuration.bitDepth == 8) + let atoms = try #require(configuration.extensions[ + kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms] as? NSDictionary) + #expect(atoms["vpcC"] as? Data == Data([1, 0, 0, 0, 0, 40, 0x80, 1, 1, 1, 0, 0])) + } + } + + @Test("VP9 profile 2 preserves ten-bit HDR and full-range signaling") + func vp9TenBitConfiguration() throws { + try withVP9Stream { stream in + let parameters = stream.pointee.codecpar! + parameters.pointee.profile = 2 + parameters.pointee.format = AV_PIX_FMT_YUV420P10LE.rawValue + parameters.pointee.color_range = AVCOL_RANGE_JPEG + parameters.pointee.color_primaries = AVCOL_PRI_BT2020 + parameters.pointee.color_trc = AVCOL_TRC_SMPTE2084 + parameters.pointee.color_space = AVCOL_SPC_BT2020_NCL + parameters.pointee.chroma_location = AVCHROMA_LOC_TOPLEFT + let configuration = try VideoToolboxVideoFormat.vp9Configuration( + codecpar: parameters, frameRate: stream.pointee.avg_frame_rate) + #expect(configuration.configurationBoxPayload == Data([1, 0, 0, 0, 2, 40, 0xa3, 9, 16, 9, 0, 0])) + } + } + + @Test("unsupported VP9 chroma and bit depth refuse the hardware candidate") + func vp9UnsupportedConfiguration() throws { + try withVP9Stream { stream in + let parameters = stream.pointee.codecpar! + for (profile, format) in [(Int32(1), AV_PIX_FMT_YUV444P), + (Int32(2), AV_PIX_FMT_YUV420P12LE), + (Int32(0), AV_PIX_FMT_YUV422P), + (Int32(2), AV_PIX_FMT_NONE)] { + parameters.pointee.profile = profile + parameters.pointee.format = format.rawValue + #expect(throws: VideoDecoderError.self) { + try VideoToolboxVideoFormat.make(stream: stream) + } + } + } + } + + @Test("VP9 does not relabel chroma siting or RGB as hardware 4:2:0") + func vp9UnrepresentableMetadata() throws { + try withVP9Stream { stream in + let parameters = stream.pointee.codecpar! + parameters.pointee.chroma_location = AVCHROMA_LOC_CENTER + #expect(throws: VideoDecoderError.self) { + try VideoToolboxVideoFormat.make(stream: stream) + } + parameters.pointee.chroma_location = AVCHROMA_LOC_LEFT + parameters.pointee.color_space = AVCOL_SPC_RGB + #expect(throws: VideoDecoderError.self) { + try VideoToolboxVideoFormat.make(stream: stream) + } + } + } + + @Test("a rejected hardware open closes its state and opens libavcodec before returning") + func failedHardwareOpenFallsBack() throws { + try withVP9Stream { stream in + // Profile 1 is valid VP9, but intentionally outside this hardware route. + stream.pointee.codecpar.pointee.profile = 1 + stream.pointee.codecpar.pointee.format = AV_PIX_FMT_YUV444P.rawValue + let preferred = HardwareVideoDecoder() + let opened = try VideoDecoderOpener.openPreferredOrFallback( + preferred: preferred, makeFallback: { SoftwareVideoDecoder() }, + stream: stream, onFrame: { _, _, _ in }) + defer { opened.decoder.close() } + #expect(opened.usedFallback) + #expect(opened.decoder is SoftwareVideoDecoder) + #expect(preferred.onFrame == nil) + } + } + + @Test("a failed software fallback is closed and its error reaches the caller") + func failedFallbackIsClosed() throws { + try withVP9Stream { stream in + stream.pointee.codecpar.pointee.codec_id = AV_CODEC_ID_NONE + let preferred = HardwareVideoDecoder() + let fallback = SoftwareVideoDecoder() + #expect(throws: VideoDecoderError.self) { + try VideoDecoderOpener.openPreferredOrFallback( + preferred: preferred, makeFallback: { fallback }, + stream: stream, onFrame: { _, _, _ in }) + } + #expect(preferred.onFrame == nil) + #expect(fallback.onFrame == nil) + } + } + } diff --git a/docs/architecture.md b/docs/architecture.md index fab48e1bc..3e8187d9e 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -81,7 +81,17 @@ When a restart does run, it must reproduce segments on the SAME media timeline t ### Software decoder pipeline (AV1 + VP9 + VP8 + legacy fallback) -Demux the source, run video packets through libavcodec (dav1d for AV1, FFmpeg's native decoder for VP9 / VP8 / MPEG-4 Part 2 / MPEG-2 / VC-1) into `CVPixelBuffer`s, run audio through libavcodec into `CMSampleBuffer`s, render via `AVSampleBufferDisplayLayer` + `AVSampleBufferAudioRenderer` with `AVSampleBufferRenderSynchronizer` as the master clock. Used for codecs AVPlayer's HLS-fMP4 pipeline doesn't accept: AV1 (no Apple TV currently ships an AV1 hardware decoder, and Apple bundles dav1d only on iOS / macOS, so AV1 always routes here today; the engine still registers the supplemental VideoToolbox AV1 decoder and gates on `VTIsHardwareDecodeSupported` (`VTCapabilityProbe`), so a future Apple TV chip with HW AV1 is picked up automatically), VP9 / VP8 (AVPlayer parses the HLS manifest, sees `vp09` / `vp08` in the CODECS attribute, then silently stops fetching. `item.status` never leaves `.unknown`. VideoToolbox HW-decodes VP9 fine, but only outside the HLS pipeline), and legacy MPEG-4 Part 2 (XVID / DIVX / SP / ASP), MPEG-2 video, and VC-1 (none of `mp4v.20.X` / `mp2v` / `vc-1` are in Apple's HLS Authoring Spec CODECS list). Interlaced H.264 also routes here (`VideoRoutingPolicy`, keyed on the declared field order), because AVPlayer does not deinterlace and 1080i / 576i broadcast would comb; the SW path runs it through `DeinterlaceFilter` (#107). On seekable VOD that declaration is verified against decoded frames first (`InterlaceProbe`, #232): the filter engages on `AV_FRAME_FLAG_INTERLACED` alone, so a sample in which that flag never appears proves the detour would be a no-op and the stream goes back to the native path. +Demux the source, run video packets through libavcodec (dav1d for AV1, VideoToolbox with a libavcodec fallback for VP9, FFmpeg's native decoder for VP8 / MPEG-4 Part 2 / MPEG-2 / VC-1) into `CVPixelBuffer`s, run audio through libavcodec into `CMSampleBuffer`s, render via `AVSampleBufferDisplayLayer` + `AVSampleBufferAudioRenderer` with `AVSampleBufferRenderSynchronizer` as the master clock. Used for codecs AVPlayer's HLS-fMP4 pipeline doesn't accept: AV1 (no Apple TV currently ships an AV1 hardware decoder, and Apple bundles dav1d only on iOS / macOS, so AV1 always routes here today; the engine still registers the supplemental VideoToolbox AV1 decoder and gates on `VTIsHardwareDecodeSupported` (`VTCapabilityProbe`), so a future Apple TV chip with HW AV1 is picked up automatically), VP9 / VP8 (AVPlayer parses the HLS manifest, sees `vp09` / `vp08` in the CODECS attribute, then silently stops fetching. `item.status` never leaves `.unknown`. VideoToolbox HW-decodes VP9 fine, but only outside the HLS pipeline), and legacy MPEG-4 Part 2 (XVID / DIVX / SP / ASP), MPEG-2 video, and VC-1 (none of `mp4v.20.X` / `mp2v` / `vc-1` are in Apple's HLS Authoring Spec CODECS list). Interlaced H.264 also routes here (`VideoRoutingPolicy`, keyed on the declared field order), because AVPlayer does not deinterlace and 1080i / 576i broadcast would comb; the SW path runs it through `DeinterlaceFilter` (#107). On seekable VOD that declaration is verified against decoded frames first (`InterlaceProbe`, #232): the filter engages on `AV_FRAME_FLAG_INTERLACED` alone, so a sample in which that flag never appears proves the detour would be a no-op and the stream goes back to the native path. + +The software host attempts a hardware-only VideoToolbox session for VP9 profile 0 (8-bit 4:2:0) +and profile 2 (10-bit 4:2:0). `VideoToolboxVideoFormat` synthesizes the `vpcC` record WebM omits +using the [VP codec configuration box](https://www.webmproject.org/vp9/mp4/#vp-codec-configuration-box) +layout. Supplemental decoder registration is gated by OS availability. `VideoDecoderOpener` closes +a failed candidate and opens libavcodec before the host publishes its decoder or starts reading +packets; a failed fallback is also closed and its error is propagated. Playback remains on the +software presentation route because AVPlayer's HLS path does not accept VP9. Existing HEVC +hardware probing, Dolby Vision selection and HDR10+ delivery are unchanged. + ``` Source URL ──► Demuxer ──┬─► SoftwareVideoDecoder (dav1d) ──► SampleBufferRenderer diff --git a/docs/formats.md b/docs/formats.md index 8162524cb..f29c0cd19 100644 --- a/docs/formats.md +++ b/docs/formats.md @@ -25,7 +25,7 @@ The engine does not demux DASH. `DASHSourceOpener` reads the manifest with Found **Software decode** (`SoftwareVideoDecoder` + `AVSampleBufferDisplayLayer`): - AV1 (libavcodec / dav1d) on devices without HW AV1 (currently all Apple TVs, M1 / M2 Macs, pre-A17-Pro iPhones). -- VP9 and VP8 (libavcodec native) unconditionally, since AVPlayer's HLS pipeline rejects the `vp09` / `vp08` CODECS attributes even where VideoToolbox can HW-decode them. +- VP9 and VP8 always use the software presentation route, since AVPlayer's HLS pipeline rejects the `vp09` / `vp08` CODECS attributes. VP9 tries a VideoToolbox hardware session for 8-bit profile 0 or 10-bit profile 2 with 4:2:0 chroma, synthesizing WebM's missing `vpcC` record from probed parameters. Other profiles or failed hardware opens fall back to libavcodec; VP8 uses libavcodec. `activeVideoDecoder` names the decoder that actually opened. - MPEG-4 Part 2 (XVID / DIVX / SP / ASP), MPEG-2 video, and VC-1, none of which AVPlayer's HLS-fMP4 pipeline accepts; libavcodec ships native decoders for all three. - The legacy Microsoft tail (FFmpegBuild 2.4.3): MS-MPEG4 v1 / v2 / v3, the DivX 3.x that pre-2005 AVI rips carry, and WMV1 / WMV2 / WMV3 (WMV9). The routing was already correct before that release, so these reached `SoftwareVideoDecoder` and failed the load with `unsupportedCodec` for want of a compiled-in decoder. A native `.wmv` / `.asf` file plays whole since FFmpegBuild 3.1.0, which added the `asf` demuxer and the entire WMA decoder family (Standard, Pro, Lossless, Voice). That chain is all-or-nothing on purpose: the demuxer without a matching WMA decoder opens the file and then has nothing to decode its audio with, `AudioBridge` init fails, the cascade ends in `droppedNoPipeline`, and the session serves video-only and plays the file silently, which is a worse failure than an honest one. WMA is not fMP4-legal in any flavour, so it always takes the AudioBridge, like MP2 and Blu-ray LPCM. The video of such a file is usually WMV3 or VC-1, both of which were already here for their Matroska and MPEG-TS shapes. None of the six supports frame threading, so they decode single-threaded; the content that carries them is SD. Reported by cmcpherson274 (FFmpegBuild#3), whose library carries WMV only inside Matroska and MPEG-TS, which is why the native form waited for a second field report. - The Flash tail (FFmpegBuild 3.2.0): FLV1 (Sorenson Spark, the H.263 variant every pre-2008 `.flv` carries) and the On2 VP6 family (`vp6`, `vp6f`, `vp6a`) that Flash 8 brought. The `flv` demuxer has shipped from the start, so a modern Flash file (H.264 + AAC) always direct-played and only the legacy decoders were missing; that release also adds the era's audio, Nellymoser Asao, ADPCM-SWF, Speex and FLV's G.711 / big-endian PCM shapes, for the same all-or-nothing reason the WMA family shipped whole. Note where that reason lives: in the FFmpeg build, not in the routing table. `AudioCodecCompat` decides whether a codec is offered a stream-copy; the bridge cascade below it asks libavcodec for a decoder by id and never reads the table, so a codec the table does not name still plays when the build carries its decoder (measured on Nellymoser-in-FLV before its entry existed), and a codec the build lacks is silent whatever the table says. Flash Screen Video (`flashsv` / `flashsv2`) is deliberately absent: it needs zlib, which the build does not link, and screen recordings are not what a media library holds. Requested in the Sodalite Discord.