Repository navigation
Conversation
The HUD maps the pointer into image space with round(), and the inside-image test it guards accepts the full closed range [-1, 1]. On the last column or row that rounds to width or height - one past the last pixel - so every reader's pixel picker rejects it as out of bounds and returns nothing. The result is that the probe reads N/A along the right and top edges of every image, which looks like a reader problem rather than a rounding one. Clamp the coordinate to the last pixel. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Sam.Richards@taurich.org <Sam.Richards@taurich.org>
The OIIO reader never overrode pixel_picker_func(), so it inherited
MediaReader::default_pixel_picker, which returns an empty PixelInfo. The
colour pipeline needs at least three raw channels to derive the linear and
display rows, so for anything OIIO reads every row of the pixel probe showed
N/A - not just the code values.
That covers a lot of media, because the OIIO preferences mark DPX, PNG, TIFF
and JPEG as MRC_FORCE, which takes them away from the ffmpeg reader even
though ffmpeg also claims them.
Add oiio_buffer_pixel_picker, a C++ mirror of fetch_rgba_pixel in the OIIO
shader, reporting raw values for every format and code values for the
integer ones.
OIIO stores integer data that is not a whole number of bytes at the next byte
boundary, and expands it by replicating the high bits into the low bits
(BaseTypeConvertU10ToU16 in OIIO's libdpx: (v << 6) | (v >> 4)). Reporting
the stored value would therefore show 26214 for a 40% grey in a 10 bit DPX
rather than 409. The reader now carries oiio:BitsPerSample through the
shader params as native_bits_per_sample, and the picker shifts the replicated
bits back off. That inverts the expansion exactly for every 10 and 12 bit
value. PNG does not set oiio:BitsPerSample, so it falls back to the stored
depth, which is correct since PNG data is a whole number of bytes.
Measured on SMPTE HD bars at a 40% grey pixel:
10 bit DPX code R=G=B=409 raw 0.3998
16 bit DPX code R=G=B=26214 raw 0.4
16 bit PNG code R=G=B=26214 raw 0.4
Also turn the placeholder test into real coverage. The tests synthesize
images with known code values rather than depending on media outside the
repo, and cover the 10 bit round trip, 16 bit DPX and PNG, 8 bit PNG, the
three raw channels the colour pipeline needs, and out of bounds probes.
On macOS these tests need the companion change "cmake: give test targets the
same macOS defines as the libraries". Test targets currently miss the
__apple__ define the libraries get, so the '#undef nil' guards in atoms.hpp
and friends are skipped and caf::uuid::nil() fails to compile in any test
that includes media.hpp. That is pre-existing and breaks the stock
ffmpeg_test too. With the companion change these tests build and pass on
macOS; it is not part of this change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Sam.Richards@taurich.org <Sam.Richards@taurich.org>
…formats
Pixel formats the GPU shader cannot unpack were converted by sws to 8 bit
RGBA on the CPU. That had three effects:
- The YUV data was gone before the pixel picker ever saw it, and the RGBA
path recorded no code values, so the probe read N/A for common
uncompressed formats - uyvy422, v30xle (v410), uyva (v408) and rgb48be.
- Display quality dropped to 8 bits, so a 10 bit v410 movie or a 16 bit
PNG was being shown at 8 bits.
- sws_getCachedContext never reads the frame's colourspace. Unless told
otherwise with sws_setColorspaceDetails it uses SWS_CS_DEFAULT, which is
SWS_CS_ITU601. So a stream correctly tagged BT.709 was converted as
BT.601 anyway, giving visibly wrong colour.
Instead, repack each unsupported format to the closest one the shader already
handles, via shader_friendly_pix_format: packed YUV to planar at the same bit
depth and subsampling, and big endian RGB to little endian. These are lossless
repacks, so the original code values survive for the probe and the frame goes
down the planar path, where our own shader applies the matrix built from the
frame's colourspace in set_shader_pix_format_info. 8 bit input is never
promoted to a deeper format, since that would rescale its code values.
Only formats with no sensible lossless target still fall back to RGBA, and
that path now sets the stream's real coefficients and range on the sws
context, so it is colour correct too.
8 bit YUV with alpha (uyva) had no planar target at all, so add YUVA420P,
YUVA422P and YUVA444P to the supported formats, and read the alpha plane in
the 8 bit YUV branch of both the shader and its C++ mirror - previously only
the 10 and 12 bit branches did.
Fixes in the pixel picker along the way:
- Report the YUV chroma channels as Cb and Cr rather than U and V.
- Record code values for 8 bit packed RGB and RGBA, which recorded none.
- The RGBA case returned all zeros and BGRA had its channels in the wrong
order. Both now match the shader.
- The alpha code value was being given the normalised 0-1 float, so it
reported 0 or 1. It now reports the stored value.
Measured through the reader on a v408 (uyva, BT.709) SMPTE bars movie, at a
yellow bar:
code Y=168 Cb=44 Cr=136 A=255
raw R=0.7503 G=0.7476 B=-0.0018
The code values match ffmpeg's own format=yuva444p extraction exactly, and
the raw values match BT.709 to four places. BT.601 would give 0.745, 0.798,
0.030.
shader_friendly_pix_format and sws_colourspace_from_av move out of the
anonymous namespace so they can be tested, and ffmpeg_test now links ffmpeg
directly, since the plugin links it privately.
As with the OIIO tests, on macOS these need the companion change "cmake: give
test targets the same macOS defines as the libraries" to build. With it they
build and pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Sam.Richards@taurich.org <Sam.Richards@taurich.org>
Signed-off-by: tedwaine <59868698+tedwaine@users.noreply.github.com>
tedwaine
merged commit Oct 22, 2026
d997675
into
AcademySoftwareFoundation:develop
3 of 4 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Pixel Probe Extension
This is an extension of the fabulous pixel probe. I am currently needing to view the underlying data in the quicktime file, and while for some cases it correctly returns the YCrCb for some formats this doesnt work. This PR fixes some of those cases, and also uses OIIO to get values for DPX files (and other formats).
Summarize your change.
For the OIIO reader, it never overrode pixel_picker_func, so it would return N/A by default. This PR addresses that, and also handles the 10-bit case where within OIIO the data is 16-bit, but we are only looking at the top 10-bits. This should affect DPX, PNG, TIFF and JPEG.
For the ffmpeg YUV reader, while there was some support for YCrCb, there were a few formats that were not supported (and admittedly are a bit more obscure, such as uyvy422, v30xle (v410), uyva (v408) and rgb48be. So for those cases we repack the data to a format that the shader supports.
Also change was a change to the label changing it from YUV to YCrCb
Describe the reason for the change.
There were quite a few image types that gave N/A for the Source Data, this addresses many of those cases.
Describe what you have tested and on which operating system.
This has only been tested on MacOS 26.6
I can supply a set of additional test quicktimes if thats helpful.
Add a list of changes, and note any that might need special attention during the review.
Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com