P1: science corrections in results and methods (UQ-05, DET-01, SCI-05, S-RE.2-4, SCI-06/INV-01, UQ-03/04) by mdenolle · Pull Request #52 · Denolle-Lab/codameter

mdenolle · 2026-09-13T19:05:20Z

Closes #39. Package P1 of the iteration-3 plan: the corrections that change what the paper says about its results. Evidence for every finding: reviews/iter2/block_S-*.md.

Ledger entries addressed

Entry	What changed	Where
UQ-05 remnant, S-RE.2	The multiverse stacks on the daily grid and decimates only the output. 12 of 108 pipelines return no epoch (MWCS with the 4-14 s window holds two 6-s sub-windows; the delay fit needs three); the text, the panel annotation and the Sobol basis count 96. New numbers in Section 4, Fig 13, Discussion, Conclusions	`deviations.multiverse`, qmd
S-RE.3	Step amplitude is post-event median minus pre-event median, on the truth and the recovered series alike (`synthetic_demo.step_amplitude`); the minimum-based metric and its extreme-value bias are named and withdrawn. Signed errors are in the Fig 7 sidecar; the step is under-estimated at every stack length. Default is option 1 of #50	`synthetic_demo`, `deviations._drop_amplitude`, Section 3.6, Fig 7 caption
S-RE.4, SCI-09	Branch rule rewritten as a recommendation; the pipeline fits one stretch over both branches and C_d has no branch term	Section 3.7
S-FD.3, S-RE.1, S-ME.N2	Table 2 datum row split by experiment; estimator-defaults row added	Table 2
DET-01	Inversion reference rejects non-stretching estimators and uses the stack length as its block length; the coherence gate applies through a stretching-coherence probe to every estimator on fixed and moving references (the inversion has no per-epoch coherence, stated); OAT sweep on three paired seeds with common-support RMS and availability; Fig 12 with whiskers	`deviations.run_pipeline`, `oat_effects`, Fig 12, Section 4
SCI-05 residual, S-ME.N4	Hyper-priors, chain settings, seeds and the 0.5 coherence rule stated. `codameter.calibration` adds a held-out half-member coverage (C_d fitted on six members, scored on the other six) and a split R-hat over two chains; clean scenario rerun: held-out 95% coverage 0.937 +- 0.004 (within the margin), R-hat 1.006 / 1.002 / 1.022, lambda above 1.05 in 22 of 200 realisations with no coverage moved by more than 0.002	`calibration.py`, Section 5, Table 5
UQ-03 residual	Duplicating the ensemble shrinks the credible half-width by 0.83 (x2) and 0.64 (x4) against 0.71 and 0.50 for independent data: the smoothness prior supplies 80% of the precision of mu. Stated in Section 5; pinned by a test	`scripts/check_ensemble_duplication.py`, `tests/test_uq_bayes.py`
UQ-04 residual	Whitening the residuals with C_d: the old correlation-length fit (intercept, 40 lags) returned 39 d and left a whitened variance of 3.05 with negative autocorrelation at two and three epochs. The fit now runs through the origin on the leading lags (L = 5.7 d on the Fig 14 realisation, 6.1 d over 200); whitened variance 0.62, lag-one 0.26. Stated in Section 5 with the withdrawal	`uq_bayes._estimate_corr_length`, `scripts/check_cd_whitening.py`
SCI-06, INV-01	Depth stage "implemented, not evaluated"; C_m inherits whatever correlation the supplied covariance carries; density field is a stated, unimplemented extension; docstrings of `uq_measurement` and `uq_depth` no longer promise a closed loop	Section 6, docstrings
SCI-04, S-CD.2	Table 4 estimator cell agrees with Results; Yuan regime waits on #47	Table 4
SCI-03	Table 3 row and the scale-of-effect sentence name the ablation	Section 3.5
SCI-02	Envelope is the all-orders 2-D radiative transfer solution; the synthetic is random phases under it; departures are processing or additive noise	Section 2
S-DI.R2 to R6	Realisations redraw only the noise; both bands see one imposed dv/v; comparator named; "committed as produced"	Discussion, table caption

Two findings beyond the plan

Correlation length. The earlier fit of L was wrong, not merely unvalidated; the whitening check the plan asked for exposed it. With the corrected fit n_eff rises from 23 to 29 on the shown realisation (45 on average over 200); no coverage statistic changes, since C_d's diagonal is untouched.
Prior-dominated credible band. The duplication test shows the band on mu is set mostly by the smoothness prior, which is one reason its 0.59 coverage has to be tested rather than assumed. Stated in Section 5.

Calibration reruns (complete)

All three locked scenarios were rerun with the new diagnostics and the corrected correlation-length fit (clean in 7b67e53, shared-source and clock-drift in 86dd78c after a machine restart lost the first attempt). Every in-sample statistic is bit-identical to the archived 2026-09-10 runs; Table 5 gains the held-out half coverage (0.937 / 0.930 / 0.936), L (6.1 / 6.2 / 6.2 d) and the split R-hat rows. paper/data/calibration/README.md records the provenance caveat on git_commit.

Verification

Clean calibration rerun with the same seeds: every in-sample statistic bit-identical to the archived 2026-09-10 run (member 95% 0.9561, posterior 0.590, s 11.67); only L and n_eff changed.
Copilot review threads (zero-variance rows in the residual autocorrelation; the run_pipeline gating docstring) are fixed and answered.
pytest on the touched modules: test_deviations (17), test_calibration, test_uq_bayes (22), test_figures, test_synthetic_demo, test_audit_small_fixes, test_uq_depth, test_uq_measurement, test_sign_convention: all pass.
Pre-commit hooks pass. PDF rebuilt: 87 pages; ?? 0, ,% 0, << 0; [Funding 1 (P6).
Runtime note: the daily-grid multiverse takes about 15 minutes; the reduced-factorial test keeps the suite fast.

🤖 Generated with Claude Code

…erage and R-hat, corrected correlation length (#39) deviations.py - multiverse() runs every pipeline on the daily CCF grid and decimates only the output (UQ-05 hand-off); returns n_valid, n_empty and the list of empty pipelines with the reason (MWCS on the 4-14 s window holds two 6-s sub-windows, fewer than the three its delay fit needs: 12 of 108). - run_pipeline(): reference="inversion" rejects estimators other than stretching and uses the stack length as its block length (DET-01); the coherence gate applies on fixed and moving references to every estimator through the stretching-coherence probe of the same band, window, stack and reference (GATE_CC = 0.6); the inversion has no per-epoch coherence. - oat_effects(): three paired seeds (OAT_SEEDS), common-support RMS and availability per row, min/max over seeds; fig_deviation_ranking plots the common-support RMS with whiskers and the availability in the label. - _drop_amplitude delegates to synthetic_demo.step_amplitude. synthetic_demo.py - step_amplitude(): post-event median minus pre-event median over 120-day windows, used for the truth and the recovered series alike (S-RE.3; the minimum over the post-event window had an extreme-value bias); the stacking figure records the signed step errors in its sidecar through fig.codameter_arrays, which figures.py now collects from any builder. uq_bayes.py - _estimate_corr_length(): fit through the origin on the leading lags above CORR_FIT_FLOOR = 0.1; the previous fit (intercept, 40 lags) let the noise floor of the long-lag autocorrelation flatten the slope and returned 39 d on the Fig 14 realisation where the short-lag decay gives 5.7 d; whitening the residuals with the old C_d left a variance of 3.05 and autocorrelation -0.22/-0.27 at two and three epochs, with the new one 0.62 and 0.26 at one epoch (scripts/check_cd_whitening.py). - residual_autocorrelation() and split_rhat() exported; gibbs_dvv keeps the tau2, s2 and lambda chains in BayesResult.samples_hyper. calibration.py - run_realization(): held-out half-member coverage (C_d fitted on a random six members, scored on the other six) and split R-hat over a second chain from another seed; summarize() adds the margin check on the held-out coverage and rhat_max; CLI prints them. scripts/check_ensemble_duplication.py: duplicating the ensemble shrinks the credible half-width by 0.83 (x2) and 0.64 (x4) against 0.71 and 0.50 for independent data; the smoothness prior supplies 80% of the precision of mu at the posterior mean (UQ-03). uq_measurement.py, uq_depth.py: docstrings no longer promise a closed loop or a cross-band covariance built from the temporal C_d (INV-01). Tests: reduced-factorial multiverse test with the empty-pipeline count, gate and inversion rules, OAT common support, split R-hat, kept chains, duplication behaviour, correlation-length recovery on AR(1) residuals. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…s checks (#39) - demo_18 (Fig 7): step errors with the median definition, signed values in the sidecar (data/snr*/signed_step_error_pct); RMS and coherence arrays unchanged. - demo_10 (Fig 12): three paired seeds, common-support RMS with whiskers, availability in the label, inversion with 10-day blocks. - demo_11 (Fig 13): daily-grid stacking; 96 of 108 pipelines return a series; RMS 0.020-2.84%, per-epoch SD 0.39-1.72%; first-order indices window 0.27, estimator 0.19, stack 0.17 for the RMS (sum 0.65) and reference 0.08, estimator 0.05, window 0.03 for the step (sum 0.17). - demo_12 (Fig 14): corrected correlation length 5.7 d (was 39 d), n_eff 29 (was 23); members, mu, coverage bands unchanged. - locked_clean_n200.json rerun with the same seeds: every in-sample statistic bit-identical to the 2026-09-10 run (member 95% 0.9561, posterior 0.590, s 11.67); new: held-out half-member 95% coverage 0.937 +- 0.004 (within the 0.03 margin), split R-hat 1.006 / 1.002 / 1.022 for tau2 / s2 / lambda with lambda above 1.05 in 22 of 200 realisations (largest 1.19; excluding them moves no coverage by more than 0.002), L 6.1 d, n_eff 45. The shared_source and clock_drift reruns are in progress and follow in the next commit; until then their columns show the old L and no held-out or R-hat rows (see paper/data/calibration/README.md). - calibration table: held-out row, L row, split R-hat row, caption states that a realisation redraws only the additive noise on one fixed coda. - paper/data/bayes: cd_whitening.json and duplication_check.json with their README. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

- S-RE.2 / UQ-05: Section 4 states that pipelines stack on the daily grid with every third output day retained, that 12 of 108 pipelines return no epoch (MWCS with the 4-14 s window, two sub-windows) and that the statistics are over the 96 that do; numbers from the regenerated sidecar (SD 0.39-1.72%, RMS 0.020-2.84%, indices 0.27/0.19/0.17 and 0.08/0.05/0.03, sums 0.65 and 0.17, the stack length with no first-order effect on the median-based step). Fig 13 caption updated; Discussion and Conclusions count 96 of 108. - S-RE.3: Section 3.6 defines the step as post- minus pre-event medians, names the withdrawn minimum-based definition and its bias, and reports the new signed errors (shallower step at every stack length, growing with the stack: 0.002 to 0.014% workable, 0.005 to 0.066% poor, against a 0.26% step); Fig 7 caption defines the metric. - S-RE.4 / SCI-09: the branch rule is a recommendation; the pipeline fits one stretch jointly over both branches (branch="both") and C_d carries no branch term; per-branch measurement deferred. - S-FD.3 / S-RE.1 / S-ME.N2: Table 2 datum row split by experiment (noise-free generating reference for the per-choice figures, the five schemes of Section 3.5, first 60% for sweeps and ensemble with common support and three seeds for the OAT) and an estimator-defaults row (6 s / 3 s sub-windows, gamma 0.3 and 0.8 s, stretching grids and ranges, gate). - DET-01: OAT paragraph reports common-support RMS over three realisations (baseline 0.033%, 0.031-0.035), the ranking (46x, 12x, 6x, 5x, 0.7-2.3), the uncumulated trailing reference's 96% availability, the gate's null effect and the inversion with 10-day blocks at 1.7x; Fig 12 caption. - SCI-05 / S-ME.N4: hyper-priors (InvGamma(2, 1e-8) for tau2 and s2, Gamma(2, 1e-10) for lambda), one chain of 1200 sweeps with 400 burn-in and thinning 2, the sampler seed, the 0.5 coherence rule; the in-sample caveat with the held-out half coverage 0.937 +- 0.004 and split R-hat 1.006 / 1.002 / 1.022 (lambda above 1.05 in 22 of 200, no coverage moved by more than 0.002). - UQ-03: the duplication result (half-width factors 0.83 and 0.64 against 0.71 and 0.50; 80% of the precision of mu from the smoothness prior; sigma_Cd within 4%). - UQ-04: the whitening check (raw 0.57 / 0.21 at one and two epochs, the stacking signature; whitened 0.26 at one epoch, variance 0.62 with L = 5.7 d) and the withdrawn 39-day fit; L 6.1 d and n_eff 29 / 45 in the fit paragraph; the correlation-length fit described as through the origin on the leading lags. - S-DI.R2: repeated realisations redraw only the additive noise on one fixed coda (Discussion and table caption). - SCI-06 / INV-01: depth stage "implemented, not evaluated"; C_m inherits whatever correlation the supplied per-band covariance carries; the density field is a stated extension that codameter does not implement. - SCI-04: Table 4 estimator cell agrees with the Results wording; the Yuan et al. (2021) regime waits on #47. - SCI-03: Table 3 reference row and the scale-of-effect sentence name the uncumulated trailing reference as an ablation; cumulated workflows are not priced. - SCI-02: the envelope is the closed-form all-orders solution of the 2-D radiative transfer equation, the synthetic is random phases under it, and departures from the truth are due to the processing or the additive noise. - S-DI.R3, R5, R6: both bands see one imposed dv/v on the synthetic; the comparator for the processing spread is the best-practice RMS; the real-data figures are committed as produced. PDF rebuilt: 87 pages, no ??, no ,%, no <<; the Acknowledgements placeholder remains for P6. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot

🟡 Changes recommended

residual_autocorrelation can yield incorrect lag-0 behavior and biased mean autocorrelation when residual rows have zero variance; this should be fixed before merging.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

This PR implements the “P1 science corrections” package for iteration 3 by updating both the analysis code (Bayesian UQ, deviations/multiverse, calibration diagnostics) and the manuscript/figure sidecars so the paper’s reported results, methods, and numbers match regenerated evidence.

Changes:

Reworked daily-grid multiverse/OAT logic (common-support metrics, paired seeds, empty-pipeline accounting, post-stack decimation) and updated figure annotations/sidecars accordingly.
Added Bayesian calibration diagnostics (held-out-member coverage and split \hat R from two chains) plus scripts and tests to pin ensemble duplication behavior and C_d whitening checks.
Updated manuscript text/tables/captions and regenerated sidecars/tables to reflect corrected definitions and numbers (step amplitude metric, correlation-length fit, pipeline validity counts, etc.).

File summaries

File	Description
tests/test_uq_bayes.py	Adds tests for split R-hat, hyperparameter chain retention, ensemble-duplication invariance, and corrected correlation-length behavior.
tests/test_deviations.py	Updates OAT/multiverse tests for paired seeds, common-support RMS/availability, inversion-reference constraints, and empty-pipeline accounting.
tests/test_calibration.py	Adapts calibration tests to new run_realization wiring and adds assertions for held-out coverage and R-hat fields.
src/codameter/uq_measurement.py	Docstring corrections to remove “closed loop” claim and clarify current covariance propagation boundaries.
src/codameter/uq_depth.py	Docstring corrections to clarify covariance responsibility and scope (caller-supplied band covariance; no auto construction).
src/codameter/uq_bayes.py	Exposes split_rhat + residual_autocorrelation, revises correlation-length fit, and retains hyperparameter draws for diagnostics.
src/codameter/synthetic_demo.py	Introduces robust step_amplitude and updates stack-coherence figure to use it and emit signed-step sidecar arrays/meta.
src/codameter/figures.py	Allows figure builders to attach codameter_arrays/meta for richer sidecar outputs.
src/codameter/deviations.py	Revises gating behavior, OAT paired-seed aggregation, multiverse stacking/decimation semantics, empty-pipeline reporting, and deviation figures.
src/codameter/calibration.py	Adds held-out-half coverage and two-chain split R-hat diagnostics; documents realization semantics.
scripts/check_ensemble_duplication.py	New audit script quantifying effects of duplicating/quadruping ensemble members on mu credible band and C_d.
scripts/check_cd_whitening.py	New audit script whitening member residuals with C_d and reporting remaining autocorrelation/variance.
review/REVISION_PLAN_ITER3.md	Appends P1 execution log line for iteration-3 plan tracking.
paper/manuscript_marine.tex	Updates Methods/Results/Discussion/Conclusions text and table/caption wording to match corrected metrics and regenerated outputs.
paper/manuscript_marine.qmd	Same manuscript updates as .tex source (Quarto).
paper/data/calibration/README.md	Documents calibration run provenance, locking rules, and pending reruns for shared_source/clock_drift.
paper/data/bayes/README.md	Documents UQ-03/UQ-04 audit JSON outputs and how to regenerate them.
paper/data/bayes/duplication_check.json	Adds archived output from the ensemble-duplication audit script.
paper/data/bayes/cd_whitening.json	Adds archived output from the C_d whitening audit script.
paper/calibration_table.tex	Updates calibration table caption and rows to include held-out coverage, L, and split \hat R.
paper/build_calibration_table.py	Extends table builder to emit held-out coverage, correlation length, and max split \hat R summary row.
literature/figs/demo_18_stack_coherence.json	Updates figure sidecar to include new signed-step arrays and metadata.
literature/figs/demo_12_bayes.json	Updates sidecar values/titles for corrected correlation length and N_eff.
literature/figs/demo_11_multiverse.json	Updates multiverse sidecar to include n_valid/n_empty/empty list and revised Sobol indices.
literature/figs/demo_10_deviations.json	Updates deviations sidecar to include common-support RMS, whiskers across seeds, availability, and updated labeling.

Review details

Files reviewed: 25/34 changed files
Comments generated: 1
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

…pilot review) A constant or all-NaN residual row has no autocorrelation; it is now excluded from the mean over rows (with a rounding-safe criterion relative to the row's mean square) instead of contributing zeros, and entry 0 is set to 1 whenever the estimate is defined. The paper's realisation has no such row; the whitening numbers are unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Copilot

🔵 Needs a closer look

It changes core analysis logic and regenerates many manuscript-facing results across multiple modules and artifacts, warranting careful final human review despite only a minor code-level issue found.

Review details

Files reviewed: 25/34 changed files
Comments generated: 1
Review effort level: Lite

… estimator (#39, Copilot review) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Copilot

🔵 Needs a closer look

There are a couple of correctness/documentation issues in the updated analysis code paths (notably multiverse valid/empty accounting and a docstring mismatch) that should be addressed before approval.

Review details

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

src/codameter/deviations.py:520

In multiverse(), n_valid is currently computed as np.isfinite(rms_arr).sum(), which implicitly ties the “valid pipeline” count to whether RMS is defined (and to the v.sum() > 10 threshold). That can undercount pipelines that do return some epochs (or that are only sparse after decimation), and it can make n_valid + n_empty != n_pipelines, which then makes the Fig. 11 panel annotation potentially incorrect.

Consider defining n_valid purely as “pipelines that return at least one epoch” (i.e., n_pipelines - n_empty), and keep RMS-defined counts separate if needed.

src/codameter/uq_bayes.py:304

The residual_autocorrelation docstring says “entry 0 is 1”, but the implementation intentionally returns all-NaN when every row has zero/undefined variance (see tests expecting all-NaN for np.ones((2, 50))). This makes the docstring inaccurate for that edge case.

Please qualify the statement (e.g., “entry 0 is 1 when the estimate is defined”).

def residual_autocorrelation(residuals: np.ndarray, maxlag: int = 40) -> np.ndarray:
    """Mean lag-autocorrelation of the rows of ``residuals`` (NaN-aware).

    Each row is demeaned and normalised by its own variance; the lag products
    are averaged over the observed pairs of each row, then over rows. Entry
    ``k`` is the autocorrelation at ``k`` epochs; entry 0 is 1.
    """

Files reviewed: 25/34 changed files
Comments generated: 0 new
Review effort level: Lite

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…new diagnostics (#39) Same seeds as the archived 2026-09-10 runs; every in-sample statistic is bit-identical (member 95% 0.9496 / 0.9543, posterior 0.340 / 0.604, s 12.70 / 10.88); the corrected correlation-length fit gives L 6.2 d in both (was 30 d) and n_eff 33 / 43 (was 26 / 32). New rows of Table 5: held-out half-member 95% coverage 0.930 / 0.936 (both within the 0.03 margin) and split R-hat 1.036 / 1.030; lambda exceeds 1.05 in 40 and 27 of 200 realisations (largest 1.90 and 1.81) and excluding those changes no coverage by more than 0.004. Text and README updated; the three columns now come from the same code. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

mdenolle · 2026-09-14T00:24:22Z

Complete: the shared-source and clock-drift reruns are in (86dd78c); Table 5 now has all three columns from the same code. The PR is ready for review and merge; merge order note: P1, P3 (#54) and P4 (#53) all touch the manuscript, and P1 and P3 both edit the 'make it executable' paragraph and Data availability, so the second of those to merge will need a small conflict resolution.

Copilot

🟡 Changes recommended

Two confirmed correctness issues remain (multiverse event-day selection can misclassify pre-event epochs at decimated cadence; split-R-hat table aggregation can drop valid realisations due to NaN handling).

Get a fresh assessment by requesting another Copilot review.

Review details

Files reviewed: 26/37 changed files
Comments generated: 2
Review effort level: Lite

The paper is about the method, not the software: the Bayes section, the field comparison, the Fig 15 and Table 6 captions, the calibration table caption and Data availability now describe what was computed and where the record lives without naming scripts, modules, command lines or file paths. The package name and the repository URL stay in Data availability. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

mdenolle · 2026-09-14T18:34:37Z

Manuscript text in this PR no longer names scripts, modules, command lines or file paths (the paper is about the method, not the software); the package name and the repository URL remain in Data availability. The pre-existing mentions in the base text are removed in #52, so #52 should merge first; #53 and #54 strip only what they had added.

Copilot

🔵 Needs a closer look

The calibration-table builder can incorrectly drop realisations when any single hyper-parameter R-hat is NaN, potentially misreporting the split R-hat summary.

Review details

Suppressed comments (1)

paper/build_calibration_table.py:64

Per-realisation max split R-hat can become NaN if any of the three hyper-parameter R-hats is NaN, because max(...) will propagate a leading NaN; the subsequent v == v filter then drops the whole realisation even if other hyper-parameters had finite R-hat values. Compute the max over finite R-hats per realisation instead, so partially-defined diagnostics still contribute correctly.

    for r in runs:  # per-realisation max of the three split R-hats, then mean, se
        vals = [
            max(x.get(k, float("nan")) for k in ("rhat_tau2", "rhat_s2", "rhat_lambda"))
            for x in r["results"]
            if x.get("ok")

Files reviewed: 26/37 changed files
Comments generated: 0 new
Review effort level: Lite

…um over finite values (#39, Copilot review) - multiverse(): the step amplitude is measured about ERUPT_DAY itself, not the nearest decimated epoch, which at cadence 3 is day 729 and put one pre-event epoch on the post-event side. Regression test with a spy. - build_calibration_table.py: the per-realisation maximum of the split R-hats ignores NaN and missing entries instead of propagating them. Table unchanged (no NaN in the archived runs). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

… numbers from the event-day fix (#39) With the step measured about the event day the first-order indices for the step move to reference 0.13, estimator 0.06, window 0.06, band 0.03, stack 0.00 (sum 0.28, was 0.17); the RMS indices and ranges are unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

mdenolle · 2026-09-14T20:24:16Z

Both threads of the 86dd78c review are fixed (d1cefa7): the multiverse step is measured about the event day and the R-hat maximum ignores NaN. Fig 13 was regenerated and its step attribution numbers updated in the text (6bbe687). Ready for another look.

Copilot

🔵 Needs a closer look

It changes core analysis code and manuscript claims/figures, so the scientific outputs and regenerated sidecars need final human validation despite tests passing.

Review details

Files reviewed: 26/37 changed files
Comments generated: 1
Review effort level: Lite

…rse (#39, Copilot review) n_valid (finite RMS, the attribution basis), n_sparse (one to ten epochs) and n_empty now add up to n_pipelines; the archived run has n_sparse 0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Copilot

🔵 Needs a closer look

A few small but concrete provenance/annotation issues remain (figure annotation denominator consistency; scripts’ git commit capture not robust to cwd) that can misreport archived evidence.

Review details

Suppressed comments (3)

Previously missed (2) — in code that hasn't changed since the last review.

scripts/check_cd_whitening.py:83

The recorded git_commit depends on the current working directory. If this script is executed from outside the repository root, git rev-parse HEAD can fail or return an unrelated commit, undermining provenance in the generated JSON. Set cwd explicitly to the repo root (relative to this script) when calling git.
scripts/check_ensemble_duplication.py:78
The recorded git_commit depends on the current working directory. If this script is executed from outside the repository root, git rev-parse HEAD can fail or return an unrelated commit, undermining provenance in the generated JSON. Set cwd explicitly to the repo root (relative to this script) when calling git.

src/codameter/deviations.py:706

In fig_multiverse_full, n_off is computed over pipelines with any output (has_output), but the annotation uses n_valid (pipelines with finite RMS). If n_sparse > 0, the text can undercount the denominator and misreport how many pipelines run off-axis. Use a count consistent with has_output (or include n_sparse) for the annotation denominator.

        f"{n_empty} of {mv['n_pipelines']} pipelines return no epoch\n"
        f"{n_off} of the {n_valid} others leave the axis range\n"

Files reviewed: 26/37 changed files
Comments generated: 0 new
Review effort level: Lite

…annotation denominator is the pipelines with output (#39, Copilot review) The annotation text of the archived Fig 13 is unchanged (no sparse pipeline: 96 with output, 12 empty). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

mdenolle · 2026-09-14T23:08:44Z

The three suppressed comments of the 697d253 review are addressed in f38f098: both check scripts run git from the repository root (cwd pinned relative to the script), and the Fig 13 annotation's denominator is the number of pipelines with any output (finite RMS or too few epochs for one), which equals the archived 96 since no pipeline is sparse; the figure text is unchanged.

Copilot

🟢 Approval recommended

Only a minor docstring contract mismatch was found in the updated code; the functional and test changes appear consistent with the stated corrections.

Review details

Suppressed comments (1)

src/codameter/uq_bayes.py:304

The residual_autocorrelation docstring says “entry 0 is 1”, but when all rows are constant/invalid the function returns an all-NaN array (by design, per the new tests) and rho[0] remains NaN. Please qualify the docstring (e.g., “entry 0 is 1 when the estimate is defined”) to match the actual contract.

Files reviewed: 26/37 changed files
Comments generated: 0 new
Review effort level: Lite

mdenolle · 2026-09-14T23:18:49Z

Docstring qualified in f38f098: entry 0 is 1 when the estimate is defined, all entries NaN when no row has a finite non-zero variance.

…opilot review) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

mdenolle · 2026-09-14T23:19:22Z

Docstring qualified in 688cbcb: entry 0 is 1 when the estimate is defined, all entries NaN when no row has a finite non-zero variance.

Copilot

🔵 Needs a closer look

src/codameter/calibration.py documents run_realization as “never raises” but still performs an import before the try/except, which can violate the stated contract.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

src/codameter/calibration.py:152

The docstring says run_realization “never raises”, but the from .uq_bayes import ... is executed before the try: block, so an import-time failure would still raise and bypass the failure-recording path. Move the import inside the try to make the contract accurate.

Files reviewed: 26/37 changed files
Comments generated: 0 new
Review effort level: Lite

…olds (#39, Copilot review) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

mdenolle · 2026-09-14T23:25:34Z

Addressed in 9088088: the import moved inside the try block, so an import failure is recorded as a failed realisation like any other error.

Copilot

🔵 Needs a closer look

It changes core analysis semantics and published scientific numbers across code, tests, and manuscript artifacts, so it warrants careful human verification of the regenerated evidence chain.

Review details

Files reviewed: 26/37 changed files
Comments generated: 0 new
Review effort level: Lite

mdenolle and others added 4 commits September 13, 2026 12:03

Log the P1 execution in the iteration-3 plan record (#39)

813c1b2

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot AI lite review requested due to automatic review settings September 13, 2026 19:05

mdenolle added this to the Iteration 3 revision milestone Sep 13, 2026

mdenolle added P1 science corrections review-iter2 from the iteration-2 pre-submission review labels Sep 13, 2026

Copilot started reviewing on behalf of mdenolle September 13, 2026 19:05 View session

Copilot AI reviewed Sep 13, 2026

View reviewed changes

Comment thread src/codameter/uq_bayes.py Outdated

Copilot AI review requested due to automatic review settings September 13, 2026 21:42

Copilot started reviewing on behalf of mdenolle September 13, 2026 21:43 View session

Copilot AI reviewed Sep 13, 2026

View reviewed changes

Comment thread src/codameter/deviations.py

This was referenced Sep 13, 2026

P4: survey hygiene (S-CD.1-3, SCI-08): DOI-aware keys, denominators, coded reporting rates #53

Open

P3 (part 1): sidecar provenance, --check mode, in-repo Gate 1 figure, paper workflow (S-RP.1/3/4, S-RE.5, S-FD.4/5) #54

Open

run_pipeline docstring: the gate applies to both references and every…

1345a47

… estimator (#39, Copilot review) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Copilot AI review requested due to automatic review settings September 13, 2026 23:26

Copilot started reviewing on behalf of mdenolle September 13, 2026 23:27 View session

Copilot AI reviewed Sep 13, 2026

View reviewed changes

mdenolle and others added 2 commits September 13, 2026 17:16

Record the 2026-09-13 prompts in the collaboration log

64e439c

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Copilot AI review requested due to automatic review settings September 14, 2026 00:24

Copilot started reviewing on behalf of mdenolle September 14, 2026 00:24 View session