Consistent use of the external healsparse masks

[cailmdaley]

Closes #846. Closes #837. Part of the #845 epic; builds on #852 and merges after it.

Until now, ShapePipe made its own masks: the mask module rasterized GSC star positions, Messier/NGC regions, and the instrument flags into a per-image flag file, and the star selection cut on that image. The mask force's external masks never entered the pipeline.

This PR removes all mask generation from ShapePipe. Two kinds of mask input remain, ingested differently.

1. The instrument flags delivered with each exposure (bad columns, saturation) — the only masks that touch pixels. They flow as before: SExtractor reads them per CCD and records them per object as IMAFLAGS_ISO; ngmix sees them in its stamp weights.
2. The UNIONS healsparse masks, fixed on the sky — never rasterized. Each map is queried once per object at its (RA, Dec) and the answer becomes a catalogue column: make_cat writes every bit, per band, into MASK_<band>, unfiltered. All selection on these columns happens downstream.

PSF star selection — the one decision that cannot be redone later — cuts only on the instrument flags: they mark corrupted measurements, stars whose own photometry and shape are unreliable. For everything the sky masks describe, the pipeline deliberately starts by relying on PSFEx's outlier rejection alone, and the first runs measure how well that holds. If it proves insufficient, there is machinery to pass any of the external masks — for example the star mask — to the PSF star selection, switched on in config.

Shape measurement uses no external mask: ngmix zero-weights instrument-flagged pixels and drops heavily flagged epochs, exactly as before. Whether an object in a halo, a trail, or a masked star enters the science sample is decided downstream from the MASK_<band> columns.

The survey footprint is a separate, positive coverage map — the exposure map of CCDs with a PSF model (#797), or the pointing-coverage map — and the mask bits subtract from it to give the final footprint.

The mask products

Sky-fixed boolean healsparse maps, one per bit, nside 131072 ≈ 1.6″; current products mask_ugriz_nside131072_n<bit>.hsp (2026-08-05, post-GSC2-fix; locations on the wiki):

bit	value	meaning
0, 1	1, 2	faint / bright star halo
2	4	star mask (the star's core and spikes)
3	8	manual mask (large galaxies)
4–8	16–256	no u g r i z data
9	512	outside the tile's unique region
10	1024	MaxiMask (trails, dead columns; tile-level)
11	2048	no Pan-STARRS z2 data

Paths and bit choices live only in config, so regenerated products cost no code.

What changed in the code

The mask module is deleted, together with everything that existed only to feed it: the GSC star-catalogue machinery, the WeightWatcher configs, and the tile-level masking variants. random_cat is deleted too — randoms and unmasked area are map algebra on the coverage map downstream, which is what closes #837. Exposure SExtractor now reads the instrument flag file straight from split_exp, and tiles are detected without any flag image. In the Snakemake workflow the exposure chain shrinks to get-images → split → psf: the star-catalogue staging, its config keys, and the mask rules are removed. The old CANFAR bash drivers still name the deleted configs; they are superseded by the workflow and left for a separate cleanup.

The first run of this branch refreshes the #850 old-vs-new comparison against the 2026 products.

— Claude on behalf of Cail

[cailmdaley]

Policy update from the 2026-07-21 mask-force telecon (details on #845): the rasterizer design here is unchanged, but the BIT_FLAG_MAP policy is now consumer-aware. Star-halo bits (1, 2) must not enter the flags that setools' IMAFLAGS_ISO==0 cut sees for PSF star selection — halo masks flag, they don't reject. Defect bits (e.g. MaxiMask/streaks) may still cut. The issue body's PSF-star-selection paragraph has been updated accordingly; provisional on the surviving-star overlay check (#850).

— Claude (fable) on behalf of Cail

[martinkilbinger]

Looks all pretty good.

One question: Why is only the footprint mask ("BIT_FLAG_MAP = 64:1") in the config files? Should these not also include the star, maximask, and manual masks?

[cailmdaley]

Looks all pretty good.

One question: Why is only the footprint mask ("BIT_FLAG_MAP = 64:1") in the config files? Should these not also include the star, maximask, and manual masks?

Question moot now, BIT_FLAG_MAP was a holdover from when the masks were getting rasterized onto the image grids. All bit masks are now consistently written to the catalog.