Correct the pointing of XRT images with the co-alignment databases of SolarSoft, by default from the cross-correlation with AIA.

[roytsmart]

Corrects the pointing of every XRT image as SolarSoft's xrt_read_coaldb.pro does. By default it uses the co-alignment database from the cross-correlation of XRT with AIA 335 Å (Yoshimura & McKenzie 2015), so the images line up with AIA without the manual step in the DEM tutorial.

API

• coalign on hinode.xrt.open, Filtergram.from_time_range and Filtergram.from_fits:
  • "aia" (the default): coaldb_c, falling back to coaldb_u for images it has no entry for, as xrt_read_coaldb, /aia_cc does.
  • "ufss": coaldb_u alone, SolarSoft's default.
  • None: the pointing of the headers, the behavior before this PR.
• Filtergram.coalignment: the CALIBRATION_TYPE of each image, numbered as in xrt_read_coaldb.pro (1 AIA cross-correlation, 2 G-band limb, 3 UFSS, 4 X-ray limb, 6 other, −1 no entry). It is None when coalign=None. It is indexed with the images.
• from_fits also takes num_retry now, for the database downloads.
• Checked first: a wrong coalign raises ValueError before anything is downloaded or read.

What the correction does

This is a port of xrt_read_coaldb.pro (v2016.07.06) and xrt_rollangle.pro (v2022.09.06):

• Matching: an image gets the database entry whose DATE_OBS is within 0.1 s of its own.
• Header values: a matched image gets CRVAL = (XCEN, YCEN) from the database, CRPIX = NAXIS // 2 + 0.5 (IDL integer division) and CDELT = 1.0286″ (X-ray) or 1.0302″ (G-band) × CHIP_SUM.
• Roll: every image, matched or not, gets the roll of xrt_rollangle.pro. That is the piecewise fits before the attitude anomaly of 2021-12-27, the model during it, and after it a linear interpolation in xrt_rollangle_db.geny.
• Extrapolated roll: past the last measurement the roll is extrapolated, as IDL's interpol does, but with a warning. Measurements reach the server days after the images (the copy downloaded on 2026-10-10 ends on 2026-10-04). The roll can change by degrees in days: it went from 2.38° on 2024-08-06 to 0.31° on 2024-08-12, so extrapolating that slope for ten days would be off by about 3.5°.

What the Level 1 files already have

Every file I checked (2007, 2012, 2015, 2019, 2022, 2025, 2026) says (XRT_READ_COALDB) Coaligned using UFSS data. Calib. type: 3. So xrt_prep already applied the UFSS database, without /aia_cc.

• "ufss" reproduces the centers in the headers.
• "aia" moves the 2019-09-30 18:08:00 Al_poly image by −0.34″, +0.68″, the same shift the tutorial applies by hand. It moves a 2022-06-01 Al_poly image by +4.4″, +2.5″.

The roll in some headers is stale

When the image was taken	Header vs xrt_rollangle.pro now
Before 2022	within 5×10⁻⁵° (they agree)
2025, 2026	exact
Prepped Jan–Oct 2022 (seen for 2021-12-27, 2022-06-01, 2022-10-01)	still the pre-anomaly roll (about −0.3°), off by 1.1–1.4° after the anomaly and by up to 23° during it

So with either database those images rotate. I haven't checked AIA independently; this reproduces what current SolarSoft gives.

One more to check against AIA some time: the roll database starts on 2022-03-03 10:02 with 28.86°, then 2.78° a day later. So the first images after the anomaly (2022-03-03 11:30) get a roll of about −27°, against −0.25° in their headers. XRT has no images between 2021-12-27 15:02 and then, so the backward extrapolation before the first measurement never applies.

Downloads and caching

The databases come from https://sohoftp.nascom.nasa.gov/sdb/hinode/xrt/xrt_msu_coalign/ and are stored under directory (by default hinode.directory_default) at the same paths.

• Listings: the coaldb_c listing is 4.7 MB (about 3 s). So listing.txt keeps each file's name, the time the listing says it last changed, and when we first saw that version.
• Relisting: a directory is listed again after a day for an image after the start of the last file, and after a month for any image, since the server has rewritten more than a thousand files of each database on one day (2015-11-06 and 2020-06-19 for coaldb_u).
• Database files: a file is downloaded again only if the copy is older than the version the listing shows. So the last file is downloaded again once the database grows, and a rewritten file once a relisting shows the change. A complete file before a gap between files is not downloaded again; it used to be, every day.
• Roll database: downloaded again after a day for an image after its last measurement, and after a month for any image.
• Offline: if a listing or file can't be downloaded again, the copy from before is used with a warning. If it was never downloaded, ConnectionError says to pass coalign=None. Old data that was loaded once loads offline.
• Load times: the first "aia" load of the 2019 images took 17 s, mostly the listings. After that it took about 2 s.

Behavior change

Because it's on by default, coordinates change for existing code: by about an arcsecond in 2019, several arcseconds and degrees of roll for 2022, and 23° during the anomaly. A load needs sohoftp the first time it touches a database file, and recent images check again after a day. coalign=None restores the old behavior.

Other changes

• Download refactor: download is split into _path_local, _write and _download_file, so the databases (IDL save files, signature SR\x00\x04) use the same atomic write and retries as the FITS files.
• Dependency: scipy, imported lazily for scipy.io.readsav. It was already a dependency of xrtpy.
• Bibliography: Yoshimura & McKenzie (2015), the same entry PR Add a tutorial inverting AIA and XRT images for the DEM of a jet #4 adds, so refs.bib will need a trivial merge with whichever lands second.
• Docs index: mentions the correction.

Follow-up

Once this merges, the DEM tutorial in PR #4 can drop its manual co-alignment cell. Its shift would then come out as zero.

Test plan

• pytest hinode passes locally. New tests in _coalign_test.py:
  • Roll: the roll against seven Level 1 headers, one in each roll model and two from the roll database, plus the anomaly model.
  • Lookup: both databases, including the 0.1 s tolerance and an image before the first file.
  • Refresh rules, with mocks: the relisting after a day and after a month, the times kept for changed and unchanged files, the re-download of a copy older than the listed version, and the roll database's refresh.
  • Offline: the fallback to the old listing and old files with a warning, and the ConnectionError when nothing was downloaded.
  • Roll warning: the extrapolation past the last measurement warns, and the interpolation doesn't.
  • Header values: the G-band plate scale, and an image with no entry keeping its header but getting the roll.
  • Lookup: only the files an unmatched image needs are read, and the nearest entry is found in unsorted entries.
  • Loaders: Filtergram and open with each coalign (exact calibration types), coalignment after indexing, the databases saved in directory, a wrong coalign caught before downloads, and the astropy WCS comparison with and without the correction.
  • Coverage: _coalign.py is 100% covered locally.
• ruff check . and black --check . pass.
• Local pyright 1.1.414 gives the same 4 errors as main (astropy Time overloads); CI pins 1.1.411.
• sphinx-build -b html docs succeeds with every example executing.

🤖 Generated with Claude Code

https://claude.ai/code/session_011VGPHLGJi8acbuLYSfBHTk

[read-the-docs-community]

Documentation build overview

📚 hinode | 🛠️ Build #35065507 | 📁 Comparing dfb4f18 against latest (13f1915)

  🔍 Preview build  

7 files changed · ± 7 modified

± Modified

• genindex.html
• index.html
• _autosummary/hinode.xrt.Filtergram.html
• _autosummary/hinode.xrt.open.html
• _modules/hinode/xrt/_data.html
• _modules/hinode/xrt/_filtergrams.html
• _modules/hinode/xrt/_xrt.html

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.84%. Comparing base (13f1915) to head (dfb4f18).

Additional details and impacted files

@@            Coverage Diff             @@
##             main       #5      +/-   ##
==========================================
+ Coverage   99.79%   99.84%   +0.05%     
==========================================
  Files          16       18       +2     
  Lines        1432     1968     +536     
==========================================
+ Hits         1429     1965     +536     
  Misses          3        3              

Flag	Coverage Δ
unittests	99.84% <100.00%> (+0.05%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.