Repository navigation
Add Filtergram.ratio_noise() to measure the noise of XRT images from consecutive frames - #6
Merged
Merged
Conversation
…e of images from the difference between each image and the next. The measurement of the DEM tutorial, as a method, so that the DEM of every frame of a movie can scale its uncertainty the same way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011VGPHLGJi8acbuLYSfBHTk
Documentation build overview
4 files changed± genindex.html± _autosummary/hinode.xrt.Filtergram.html± tutorials/dem.html± _modules/hinode/xrt/_filtergrams.html |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #6 +/- ##
==========================================
+ Coverage 99.84% 99.86% +0.01%
==========================================
Files 18 18
Lines 1968 2160 +192
==========================================
+ Hits 1965 2157 +192
Misses 3 3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
- Compare an image with the next only if the next begins after it, so images out of time order or loaded twice are not compared, and only if both have the same plate scale and point within a pixel of each other. - Leave out pixels whose `z` is not finite, saturated pixels, and raise a `ValueError` when there are fewer than two images or no pixel to measure the noise in, instead of returning NaN. - Select pixels by the mean of the two images instead of requiring both below `threshold`, which left out the largest differences: for noise of 0.2 at the threshold that gave 0.08, and now gives 0.21. - Start the clipping from the median of z^2, so outliers in many pixels cannot raise the first estimate enough to survive it. - Reduce over the time and detector axes only, giving a ratio for each entry of the other axes, and accept a `where` per image and a scalar `timedelta`. - Take the intervals by named indexing, find the variance once with a shared `_variance()`, and note in the docstring that the errors of the dark and vignetting models cancel in the difference. - The tutorial calls the method, and its example leaves out the jet. On event E the ratio is 0.1527 (0.1525 before). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011VGPHLGJi8acbuLYSfBHTk
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
hinode.xrt.Filtergram.ratio_noise(noise_photon, where=True, threshold=10 DN/s, gap=25 s, clip=5, num_iteration=3). It returns the ratio of the variance of the noise measured in a sequence of images to the variance ofFiltergram.uncertainty().This is the measurement from the "Uncertainty of XRT" section of the DEM tutorial, moved into the library. The tutorial now calls it, and the DEM movie of event E for the ESIS jet paper uses it.
What it does
gap, with the same exposure (to 0.01 s), and with the same plate scale and pointing, so that their first pixels land within a pixel of each other on the Sun. Images out of time order, a file loaded twice, or interleaved programs with another field or binning are not compared.wherein both images (it may differ from image to image), not saturated in either, with a finitez, and whose mean over the two images is fainter thanthreshold. Selecting on the mean rather than on each image keeps the largest differences nearthreshold.clip²·ratio left out, repeatednum_iterationtimes.noise_photon, give one ratio each.ValueErrorif there are fewer than two images, or if no pair has a pixel to measure the noise in, instead of returning NaN.The docstring explains why the ratio is below one for XRT: charge diffusion and the JPEG compression. It also notes that the ratio is measured in faint pixels, and that the errors of the dark and vignetting models are the same in both images and cancel in the difference, so the ratio describes only the noise that changes from image to image.
uncertainty()now takes the square root of a private_variance(), whichratio_noise()uses directly.Checked against the tutorial
On the tutorial's frames from 18:06:11 to 18:11:01, with the 128-pixel box excluded, it gives 0.1527. The tutorial's old inline code gave 0.1525. The correlations of neighboring pixels in the next cell are unchanged (0.425 inside a block, 0.282 across an edge). All 21 cells of the tutorial run.
Tests
On synthetic flat sequences with noise of a known ratio of 0.2:
test_ratio_noise: the ratio comes back to 2%.test_ratio_noise_pairs: a pair with a different exposure and one with a long gap are skipped; a longergaplets the second in.test_ratio_noise_order: an image that began before the previous one, and a duplicate, are skipped.test_ratio_noise_aligned: an image two pixels off and a binned one are skipped; one half a pixel off is compared.test_ratio_noise_excluded: cosmic rays are clipped, and the median start already ignores them; brighter pixels and a changing region outsidewhereare left out.test_ratio_noise_outliers: a 9σ change in 5% of the pixels of one image is clipped (the old mean start gave 0.30).test_ratio_noise_threshold: pixels at the threshold give 0.207 (selecting on each image gave 0.078).test_ratio_noise_where_time,_saturated,_axes,_timedelta,_nan,_none: a per-imagewhere, saturated pixels, an extra axis, a scalar exposure, a NaN innoise_photon, and the errors.The whole suite passes (255 passed, 1 skipped).
black --checkandruff checkpass, and pyright 1.1.414 reports the 4 errorsmainalready has.🤖 Generated with Claude Code
https://claude.ai/code/session_011VGPHLGJi8acbuLYSfBHTk