foundation: sources split into inherited child tables by UT date and SCA, loaded by COPY (sources-children)

system/decisions.md

@@ -1071,3 +1071,6 @@ The SFFT rows are runs 145 and 146. Run 146's false positives, 102.7, are one pe
 
 (decision-production-retained-releases)=
 **Production runs on retained releases only; the release is the commit**: `rapid-rebuild-production`, the one job definition whose role writes products, runs only a rapid commit the retained-releases marker lists. Its identity is `rebuild-<commit>`; its image is one build of that commit, tagged `rebuild-<commit>-<base8>` after the first eight hex of the base image's digest, so the same commit can be built again on each patched base under ECR's immutable tags. Four pieces carry it, all in rapid_systems. `verify-release-retention.sh --retain <release>` keeps a release: each image goes to the backups bucket's no-expiry `archive/` prefix as an OCI archive with its digest preserved, never to a registry, and the marker records each object and digest. The Wednesday `production-rebuild` operation builds production's commit on the newest base and pushes it, registering and repinning nothing. `release-hooks/promote <commit>` alone repins production: it refuses unless the marker lists the commit, promotes the newest build of it, and executes a change set of production's row only; `commit` and `deploy` repin scratch's `rapid-rebuild` and never production. The Friday retention deletes every unpinned build image older than seven days, so a build `promote` supersedes is gone on the first Friday after. Why: Inspector scans every ECR repository in the account and the Science Cloud weekly vulnerability notice counts every digest there, so a release kept in a registry carries its findings onto the notice for as long as it is kept, while a production run must stay on one code release for months inside the patch windows. A promote is due when the operations ledger shows a fixable finding on the production digest against its SLA; a finding in the application's dependencies is fixed by a lock-bump commit on `rapid`, a new release through retain and promote, because the lock files are part of the commit, and the rebuild patches only the OS layer. A promote takes effect at the next processing date. (Ben, 2026-11-04 00:32 UTC; release-retention-cycle tasks 1 to 5, rapid_systems PRs 470 to 474)
+
+(decision-sources-children)=
+**`sources` is split into child tables by UT observation date and SCA, inheriting the parent and loaded by `COPY`, as in `dev`, logged**: one child per UT day of `mjdobs` and SCA, `sources_<yyyymmdd>_<sca>`, made by the publisher when a source set first needs it, through `create_catalog_child`, which runs as the schema's owner: `LIKE` the parent, then `INHERIT`, as `dev`'s loaders make theirs. The publisher loads each child by bulk `COPY`, as `dev`'s loaders do, for fast loading. The parent holds no rows: a `CHECK (false)` its children do not inherit refuses one. Each child carries a `CHECK` on its date range and SCA, which `dev`'s lack, so a row cannot land in the wrong child and a query on a date or SCA skips the others. The children are logged, where `dev`'s are unlogged, because PostgreSQL empties an unlogged table after a crash and a published row is never deleted. SMDC's existing rows move into their children by `database/2026-11-04-sources-children.sql`. `xsources`, `dev`'s table of the SFFT difference image's SExtractor catalogs, comes to the rebuild with the same children and the same `COPY` in its own change. (Russ Laher, 2026-11-04)

system/foundation.md

@@ -120,6 +120,34 @@ memberships is. A set is
 identified by the job that wrote it, so a job that writes a set binds
 exactly one key of that set's product.
 
+**Source rows are stored by observation date and SCA**
+({ref}`ruling <decision-sources-children>`). As in `dev`, each row of
+`sources` lives in a child table that inherits it, the one for the UT day
+of the row's `mjdobs` and its SCA, named `sources_<yyyymmdd>_<sca>`, and
+the parent holds none. `sources_20261104_8` holds SCA 8's sources from
+exposures that began on 2026-11-04 UT (an MJD day begins at 0h UT). A
+source set is one exposure on one SCA, so a whole set lands in one child.
+A `CHECK (false)` that the children do not inherit refuses any row in the
+parent, and each child's own `CHECK` holds it to its date and SCA, so a
+row cannot land in the wrong child and a query that filters on `mjdobs`
+or `sca` skips the children it excludes. A query on `sources` or
+`sources_official` reads every child.
+
+The publisher makes the children a set needs before the transaction that
+loads it, in a short transaction of its own, by calling
+`create_catalog_child(parent, mjdobs, sca)`. The schema's owner runs that
+function, because the pipeline's role holds table grants only. A child is
+made as `dev`'s loaders make one, `LIKE` the parent with its defaults and
+its `(job, sid)` unique index, then `INHERIT`, which locks the parent
+`SHARE UPDATE EXCLUSIVE`, a lock that loads and reads do not wait for.
+The publisher then loads each child by bulk `COPY`, as `dev`'s loaders do;
+every other table takes `INSERT`. The children are logged. `dev`'s are
+unlogged, and PostgreSQL empties an unlogged table after a crash, while a
+published row here is never deleted. A child carries no foreign key to
+`jobs`, as `dev`'s carry none, since adding one would lock `jobs` against
+every publish in flight; the publisher sets `job` from the job row it
+holds locked.
+
 The pipeline id that `dev` carries in nearly every image query is not
 in any key: a run has one differencer, and a second differencer is
 another run. A PSF's kind is in its key, `science` or `reference`