[jkrick]

Closes #392 (IRSA-8059). related to #374 ( I think this PR closes that issue, but waiting for Andreas to confirm). Draft until SPHEREx DR1 is released (2026-11-12) -- do not merge before then.

Summary

All five SPHEREx tutorials were reviewed and tested for DR1:

• spherex_intro.md: query the DR1 SIA collections (spherex_dr1, spherex_dr1_deep); describe the EPSF extension instead of the old PSF cube; read MSKN* FLAGS keywords (22 flags) and cite the Explanatory Supplement's bad-pixel list; explain the combined-L2 N_* pixel-count keywords; Explanatory Supplement link renamed.
• spherex_cutouts.md: filter the TAP query to DR1 so QR2/QR3 rows aren't mixed in; Explanatory Supplement link renamed.
• spherex_psf.md: filter the TAP query to DR1; guard the sec. 5.2 QR ePSF fallback so it no longer overwrites a DR1 image's own ePSF; fix the extension-count text; Explanatory Supplement link renamed.
• spherex_source_discovery_tool_demo.md (Source Discovery Tool): DR1 SIA collection and DR1 solid-angle pixel maps (cal-sapm-v3-2026-191); read QR3/DR1 FLAGS header keywords in aperture_photometry.py; build the D4 mask from D4 flags; bad_flags = the supplement's CONTAINS_BAD_PIXEL list; select the D4 image by detector name (groups[3] picked D5 when a detector has no images at the search position); DR1-filtered TAP query in sec. 12; Explanatory Supplement link renamed.
• spherex_mosaic.md: Explanatory Supplement link renamed only. The M101 demo cube is a static file and is intentionally not regenerated with DR1.

Also: the spherex.md landing page now mentions periodic Data Releases alongside the weekly Quick Releases.

Before merging (release day, 2026-11-12 or later)

This PR was written and tested before DR1 was public, so three notebooks temporarily point at IRSA's internal test server. Do these steps in order once DR1 is public, and push the changes to this PR's branch.

• 1. Confirm DR1 is public on IRSA. All of these should work from outside the IPAC network:

  • In Python, from astroquery.ipac.irsa import Irsa; Irsa.list_collections(servicetype='SIA', filter='spherex') lists spherex_dr1 and spherex_dr1_deep. If the names differ, update them in spherex_intro.md (sec. 5, the collection list and the query_sia call) and in the Source Discovery Tool (sec. 5, data_release / sia_collection).
  • https://irsa.ipac.caltech.edu/ibe/data/spherex/dr1/ opens in a browser.
  • https://irsa.ipac.caltech.edu/data/SPHEREx/docs/SPHEREx_Expsupp.pdf opens (the notebooks link to it). If the supplement ends up at a different URL, update the link in all five notebooks: grep -rn "SPHEREx_Expsupp.pdf" tutorials/ finds all of them.
  • aws s3 ls --no-sign-request s3://nasa-irsa-spherex/dr1/ lists folders (the Source Discovery Tool reads images from there).

• 2. Point the notebooks back at the public IRSA server. In spherex_cutouts.md (sec. 5), spherex_psf.md (sec. 4), and the Source Discovery Tool (sec. 12.1) there is a code cell starting with # TESTING ONLY. In each one:

  • delete the # TESTING ONLY ... comment lines and the IRSA_HOST = "https://irsatest.ipac.caltech.edu" line;
  • in cutouts and psf, also delete Irsa.tap_url = f"{IRSA_HOST}/TAP" (the public server is the default);
  • in the Source Discovery Tool, change pyvo.dal.TAPService(f"{IRSA_HOST}/TAP") to pyvo.dal.TAPService("https://irsa.ipac.caltech.edu/TAP");
  • in all three notebooks, the TAP query string a few lines below (the SELECT ... AS uri line) contains '{IRSA_HOST}/' || a.uri; change it to 'https://irsa.ipac.caltech.edu/' || a.uri.

  Then search the repo: grep -rn "irsatest\|irsadev\|IRSA_HOST" tutorials/ should find nothing.

• 3. Update the data citation. In the Source Discovery Tool's Acknowledgements, replace the QR2 DOI sentence ("The DOI for SPHEREx QR2 data is 10.26131/IRSA629 ...") with the DR1 DOI, which IRSA will publish at release.

• 4. Update the dates. Set **Updated:** (in "About this Notebook" at the bottom) to the release date in all five SPHEREx notebooks.

• 5. Run all five notebooks against the public server and check they finish without errors and the plots look reasonable: spherex_intro.md, spherex_cutouts.md, spherex_psf.md, spherex_mosaic.md, and the Source Discovery Tool. For example, jupytext --to notebook --execute spherex_intro.md.

  • The Source Discovery Tool needs SExtractor; its sec. 3.1 explains the conda environment (conda-spherex_sdt.yml).
  • The Firefly cells open a viewer tab; check the images show up there.
  • In the Source Discovery Tool, sec. 12.1 measures a hard-coded source, selected_ids = [65]. Its number comes from SExtractor and can change if the public search returns different images. Check that the spectrum plot at the end looks like a real source (before release it had ~276 points across 0.75-5 um). If not, pick another source number from the sec. 11 plots.

• 6. Update the SPHEREx data-release skill PR (#SKILL_PR). That PR adds
  .claude/skills/spherex-data-release/ on the branch spherex-release-skill, which documents how
  this release was done. Once steps 1-5 are finished, commit to that branch (not this one):

  • in references/touchpoints.md, add the DR1 DOI, and correct the SIA collection names or the
    Explanatory Supplement URL if either changed at release;
  • add anything that went differently or took extra work on release day to
    references/decisions.md;
  • mark that PR ready for review after this one merges.

Testing so far

• spherex_cutouts.md, spherex_psf.md: executed against irsatest with real DR1 data.
• spherex_intro.md: executed against the pre-release DR1 SIA (179 results, 7 HDUs, EPSF 441 rows). Firefly displays the DR1 image (checked with a pre-release viewer); the production viewer gets a final check on release day.
• Source Discovery Tool: executed end to end (secs. 4-12, including SExtractor, bokeh, and Firefly) against pre-release DR1 SIA + IBE, with SExtractor 2.29.0. Source 65 spectrum: 276 points, 0.75-5.0 um, median S/N ~11.
• spherex_mosaic.md: link-only change; executed end to end (2026-10-29), no errors.

CI is expected to fail until release day: the notebooks point at irsatest, which isn't reachable from GitHub runners.
🤖 Generated with Claude Code

[jkrick]

Narrative in all SPHEREx notebooks and the landing page now states that QR3 and DR1 supersede QR2 and that SPHEREx recommends them; QR2 material (e.g. psf sec. 5.2) is kept for users still analyzing QR2.

[jkrick]

Updated spherex_psf.md for the recent clarification that IRSA searches won't return QR2 after the Nov 12 DR1 release:

• Top note and Section 5.2 note now say IRSA searches no longer return QR2 images, and address users who already have QR2 images
  and prefer not to switch to DR1 (without explaining how to find QR2 data).
• Section 5.2 note tells those users to set spectral_image_url in Section 4.1 to their QR2 image and rerun from there.
• Section 5 intro now says "Older images" include the old PSF, instead of "Some of the images".

No code changes needed: the Section 4.1 query returns only DR1 images, and Section 5.2 gets the ePSF from the QR3 calibration
files on S3 using fixed URLs. I ran Sections 4.2–6.2 on a QR2 cutout opened directly by URL and Section 5.2 worked.

Also checked the other SPHEREx notebooks: none of them search for QR2 (intro and SDT use spherex_dr1, cutouts and SDT TAP
queries filter to DR1), and the existing QR2 wording is still accurate.