diff --git a/docs/design/2026-11-14-merge-honesty.adoc b/docs/design/2026-11-14-merge-honesty.adoc index f0c1ff99..6a2489c3 100644 --- a/docs/design/2026-11-14-merge-honesty.adoc +++ b/docs/design/2026-11-14-merge-honesty.adoc @@ -6,7 +6,7 @@ [cols="1,3"] |=== -|Status |PROPOSED. Not ruled. No code changes with this note. +|Status |RULED by the owner on 2026-11-14. See <>. No code changes with this note. |Date |2026-11-14 |Asks |5 decisions from the owner, in <>. |Authority |The owner asked for this note on 2026-11-14, before any work order changes. @@ -72,8 +72,9 @@ category. qualified failure. |`lib/neural/radial_neural_network.ex`, `detect_novelty/2` (line 145) -|A network that is not trained returns `:untrained`. The gate reads that as -"not novel". +|A network that is not trained returns `:untrained`. The coordinator then +sets `is_novel` to `false` (`lib/neural/coordinator.ex`, line 765), and the +dispatch guard reads that field. |The one gate for unknown findings is open until training. |`lib/neural/variational_autoencoder.ex` (lines 80 to 84) @@ -92,8 +93,9 @@ things become 1, and no record says so. The dispatch tier of a recipe follows from its confidence and its annealing stage. With a merged subject, a recipe can reach `auto_execute` (0.95 or more) on the successes of findings that only look like the finding in front of it. -This is the reverse of acceptance A3 of issue 984, which asks that each -learning update rest on evidence for that update. +Acceptance A3 of issue 984 does not forbid this: it asks that no learning +rests on unverified evidence, and a verified outcome under a wide subject +meets it. The risk here is a separate one, about the subject of the evidence. echo-types gives 2 proved results that bound what a later repair can do: @@ -113,9 +115,13 @@ Each outcome row, and each receipt of `E0-08`, records the pattern id and the kind of match (`exact`, `category`, `fuzzy`). The confidence stays one value for each recipe. -* Result: each merge becomes `Fused(τ)`. An audit can split the outcomes of a - recipe by pattern. Nothing in the dispatch changes. -* Cost: small. 2 fields, in the one place that writes the row. +* Result: a merge by the recipe matcher becomes `Fused(τ)`. An audit can + split the outcomes of a recipe by pattern. Nothing in the dispatch changes. +* Limit: 2 findings that share 1 pattern id stay merged. The label cannot + tell them apart until the pattern identity changes (decision 3). +* Cost: medium. The matcher returns the recipe only, and the router carries + no kind of match. So the 2 fields must go from the matcher through the + router to the place that writes the row. * Limit: the risk stays. A recipe can still reach `auto_execute` on merged evidence. The record only makes it visible. @@ -138,8 +144,11 @@ Do option A in the E0 work. Then measure, from the labelled rows, how often a recipe gets outcomes from more than 1 pattern and how far their success rates differ. Rule on option B with those numbers. -* Result: no evidence is lost from now on, and the larger change rests on a - measurement, not on a guess. +* Result: the pattern and the kind of match are kept from now on, and the + larger change rests on a measurement, not on a guess. +* Needs: the fix check by finding (the second plain defect) before the + measurement. With the check of today, a rate for a pattern also measures + how often a category occurs twice in a repository. * Cost: the risk of option A stays until the ruling on B. === Option D: no change @@ -168,6 +177,25 @@ the direct choice is on purpose, for that reason. the ruling? . The novelty gate. Does `:untrained` mean "novel" (fail closed) from now on? +[[rulings]] +== Rulings + +The owner ruled on 2026-11-14, through the selector. + +[cols="1,4,2",options="header"] +|=== +|Decision |Ruling |Work order + +|1 |Option C: label the merge now, and rule on the split after a measurement. |`E0-18` +|2 |A `category` match or a `fuzzy` match reaches `review` at most. |`E0-19` +|3 |A new key that does not depend on wording, as its own design work order. |`E0-20` +|4 |3 small work orders now for the 3 plain defects. |`E0-21`, `E0-22`, `E0-23` +|5 |An untrained novelty network means novel (fail closed). |`E0-24` +|=== + +The measurement of option C has no work order yet. It needs `E0-18` and +`E0-22` first. + == What this note does not propose * No new way to decide whether 2 findings are alike. The trope repositories @@ -191,6 +219,6 @@ the direct choice is on purpose, for that reason. == When to update this memo -When the owner rules on <>, set the status and record each ruling -here. When a work order shows that an "inferred" result is false, correct the -table. +When the measurement of option C exists, record it here and ask the owner +for the ruling on option B. When a work order shows that an "inferred" +result is false, correct the table. diff --git a/docs/operations/e0-closeout/README.adoc b/docs/operations/e0-closeout/README.adoc index 50835473..7b7c69e5 100644 --- a/docs/operations/e0-closeout/README.adoc +++ b/docs/operations/e0-closeout/README.adoc @@ -113,9 +113,17 @@ function name is the stable locator. Measure again before you edit. |E0-15 |Truth update of the documents |S |sonnet |$3 |none |E0-16 |E0 acceptance run and receipt |M |opus |$10 |all |E0-17 |The test suite does not write into the fleet checkout |S |sonnet |$3 |none +|E0-18 |Each outcome names its pattern and its kind of match |M |sonnet |$10 |none +|E0-19 |A merged match reaches review at most |S |sonnet |$5 |E0-18 +|E0-20 |The pattern identity: design and ruling packet |Design |opus |$10 |none +|E0-21 |A fuzzy match ranks by its hits |S |sonnet |$4 |none +|E0-22 |A fix is checked against its own finding |M |sonnet |$8 |none +|E0-23 |A placeholder network adds no confidence |S |sonnet |$4 |none +|E0-24 |An untrained novelty network means novel |S |sonnet |$5 |none |=== -The caps add up to $151. The estimate for the whole close-out, with rework, is +The caps of `E0-01` to `E0-17` add up to $151. The estimate for that +close-out, with rework, is $115 to $315. That figure is an estimate. The first 4 runs (`E0-17`, `E0-06`, `E0-01`, `E0-13`) are the test run that replaces it with measured numbers. @@ -124,6 +132,11 @@ and the headless profile of this kit are in hypatia only, because the owner chose hypatia first on 2026-11-14. Put them in `gitbot-fleet` before those 2 runs, or run those 2 work orders in a session with a person. +`E0-18` to `E0-24` come from the owner rulings of 2026-11-14 on +`docs/design/2026-11-14-merge-honesty.adoc`. They are not part of the 7 +acceptance statements of issue 984. Their caps add up to $46, so all 24 +caps add up to $197. + === Order . First of all: `E0-17`. Until it merges, each `mix test` adds test rows to @@ -133,6 +146,9 @@ runs, or run those 2 work orders in a session with a person. . Second lane, each starts with a design note for the owner: `E0-04`, `E0-07`, `E0-08`. . After the rulings: `E0-05` (after `E0-04`), `E0-02`, `E0-09`, `E0-10`, `E0-11`, `E0-12`. . Last: `E0-16`. +. Beside the lanes, in any order: `E0-21`, `E0-22`, `E0-23`, `E0-24`, + `E0-18`, then `E0-19`. `E0-20` gives the owner a design note. No one + measures a success rate for each pattern before `E0-18` and `E0-22` merge. == Rules for every run diff --git a/docs/operations/e0-closeout/work-orders/E0-07-withdrawn-evidence.adoc b/docs/operations/e0-closeout/work-orders/E0-07-withdrawn-evidence.adoc index 7a96a8f2..cc6396a9 100644 --- a/docs/operations/e0-closeout/work-orders/E0-07-withdrawn-evidence.adoc +++ b/docs/operations/e0-closeout/work-orders/E0-07-withdrawn-evidence.adoc @@ -51,6 +51,26 @@ Commit `4d7f059`, read on 2026-11-14. * A fleet row has the identity `fleet_import_id` (`lib/outcome_tracker.ex`, line 55). A local row has only a provisional identity (file and line). +== Theory inputs + +Read on 2026-11-14 at echo-types `dedda75` and residual-evidence-types +`6f94640`. Each file has `--safe --without-K --cubical-compatible`. No prover +was run for this work order. Use these results as design rules. A use in +Hypatia is a transcription, and its match to the Agda is assumed, not proved. + +* Apply a withdrawal to the log, then compute again. Do not correct the + stored confidence. An aggregation that merges 2 inputs has no canonical + inverse (echo-types `EchoAggregation.agda`, line 210), and a filter + commutes with a summary only when the summary keeps what the filter reads + (`EchoSelectiveProjection.agda`, line 106). The count does not keep the row. +* Check each row against the log that remains, in 1 place. Survival is proved + against the retracted context, never against the full one + (residual-evidence-types `Revision.agda`, lines 180 to 191). +* A stable id for a local row is a need, not a choice: exact recovery forces + distinct residues for distinct inputs (`EchoFiniteResidue.agda`, line 36). +* Not proved anywhere: that retraction is idempotent or free of order, and + any statement on confidence. A withdrawal of a failure row raises confidence. + == Steps . Measure the facts again. Stop if one no longer holds. diff --git a/docs/operations/e0-closeout/work-orders/E0-08-receipt-design.adoc b/docs/operations/e0-closeout/work-orders/E0-08-receipt-design.adoc index 65bb2bcf..28b08741 100644 --- a/docs/operations/e0-closeout/work-orders/E0-08-receipt-design.adoc +++ b/docs/operations/e0-closeout/work-orders/E0-08-receipt-design.adoc @@ -43,6 +43,26 @@ Commit `4d7f059`, read on 2026-11-14. * Estate rules: JSON is I-JSON in JCS form, and a new identifier follows ADR-008 (see `.claude/CLAUDE.md` of `metadatastician/standards`). +== Theory inputs + +Read on 2026-11-14 at echo-types `dedda75` and residual-evidence-types +`6f94640`. Each file has `--safe --without-K --cubical-compatible`. No prover +was run for this work order. Use these results as design rules. A use in +Hypatia is a transcription, and its match to the Agda is assumed, not proved. + +* The receipt names each row of evidence by a stable id. It does not name a + count: a count cannot give back its evidence (echo-types + `EchoFiniteResidue.agda`, lines 36 and 47). +* The receipt is an index into the log that is kept. It is not a record that + can undo an update by itself. +* `missing_evidence` is a set of gaps. A later revision fills a gap and never + overwrites a field that is present (`EchoPartialSection.agda`, line 162, + `fill-guarded`). A withdrawal removes information, so it is not a fill: it + needs its own record. +* By the owner ruling of 2026-11-14 on + `docs/design/2026-11-14-merge-honesty.adoc`, the receipt also names the + pattern and the kind of match (`exact`, `category`, `fuzzy`). See `E0-18`. + == Steps . Measure the facts again. Stop if one no longer holds. diff --git a/docs/operations/e0-closeout/work-orders/E0-10-receipt-idris2-model.adoc b/docs/operations/e0-closeout/work-orders/E0-10-receipt-idris2-model.adoc index 69b4c271..fc6c90a7 100644 --- a/docs/operations/e0-closeout/work-orders/E0-10-receipt-idris2-model.adoc +++ b/docs/operations/e0-closeout/work-orders/E0-10-receipt-idris2-model.adoc @@ -38,6 +38,25 @@ Commit `4d7f059`. The run facts are from 2026-11-14 with Idris 2 version 0.7.0. * `verification/proof-ledger.jsonl` has one row for each proved obligation. `verification/PROOF-STATUS.adoc` is rendered from it. +== Theory inputs + +Read on 2026-11-14 at echo-types `dedda75` and residual-evidence-types +`6f94640`. Each file has `--safe --without-K --cubical-compatible`. No prover +was run for this work order. Use these results as design rules. A use in +Hypatia is a transcription, and its match to the Agda is assumed, not proved. + +Candidate properties for the model, beside those of the design: + +* The log splits exactly into the withdrawn row and the rest, and joins + again (residual-evidence-types `Revision.agda`, lines 101 to 141). Hypatia's + `Retraction.idr` does not have this yet. +* A receipt survives a withdrawal only when each row that it names is in the + log that remains (same file, lines 180 to 191). +* A revision of a receipt only fills gaps (echo-types + `EchoPartialSection.agda`, line 162). +* A known gap, not to hide: `Retraction.idr` retracts by position, and the + run time retracts by name. State which of the 2 the model proves. + == Steps . Confirm that `E0-09` is merged. Measure the facts again. diff --git a/docs/operations/e0-closeout/work-orders/E0-18-merge-label.adoc b/docs/operations/e0-closeout/work-orders/E0-18-merge-label.adoc new file mode 100644 index 00000000..156d4567 --- /dev/null +++ b/docs/operations/e0-closeout/work-orders/E0-18-merge-label.adoc @@ -0,0 +1,64 @@ +// SPDX-License-Identifier: CC-BY-SA-4.0 +// Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) += E0-18: Each outcome names its pattern and its kind of match +:wo-id: E0-18 +:wo-repo: hyperpolymath/hypatia +:wo-size: medium +:wo-agent: e0-executor +:wo-model: sonnet +:wo-effort: medium +:wo-cap-usd: 10 +:wo-max-turns: 200 +:wo-branch: feat/e0-18-merge-label +:wo-depends: none + +== Goal + +Each outcome row records the pattern that it is about and the kind of match +that chose its recipe: `exact`, `category` or `fuzzy`. A merge of 2 +findings under 1 recipe then has a label. + +== Authority + +Owner ruling of 2026-11-14 on `docs/design/2026-11-14-merge-honesty.adoc`, decision 1 (option C: label now, split after a measurement). + +== Facts measured + +Commit `b660d08`, read on 2026-11-14. The cited files have no change up to `3b8551c`. + +* `lib/recipe_matcher.ex`, `best_recipe_for_pattern/2` (line 123) tries an + exact match, then `category_match_recipe/2` (line 142), then + `fuzzy_match_recipe/2` (line 156). It returns the recipe only. The caller + cannot see which of the 3 matched. +* `lib/triangle_router.ex`, `route/3` (line 35) carries no kind of match. +* `lib/outcome_tracker.ex`, `persist_outcome/6` (line 116) writes + `"pattern_id" => nil` (line 122). +* `test/recipe_matcher_test.exs` has the tag `:verisim_data`, so CI does not run it. + +== Steps + +. Measure the facts again. Stop if one no longer holds. +. Write the tests first, in a new test file that has no `:verisim_data` tag + and that uses recipes of its own in a temporary folder. +. Add a function beside `best_recipe_for_pattern/2` that returns the recipe + and the kind of match. Keep the old function and its result, so that no + caller breaks. +. Carry the pattern id and the kind of match from the router to the outcome + row through the `metadata` map of `record_outcome/5`. Write them as + `"pattern_id"` and `"match_kind"`. +. Do not change confidence, the annealing stage or a tier in this work order. + +== Acceptance + +* New tests show the kind `exact`, `category` and `fuzzy`, 1 test for each. +* A test shows a row with the pattern id and the kind of match of its dispatch. +* A row from a caller that gives no kind has `"match_kind" => "unknown"`. It + is not written as `exact`. +* `mix test` has 0 failures. No file that was formatted before the change is unformatted after it. + +== Stop and ask + +* A path writes an outcome and has no pattern at that point. List each such + path in the pull request body. Do not invent a pattern id. +* The change needs a new field in the fleet outcome file of `gitbot-fleet`. +* The cost reaches the cap before the acceptance checks pass. diff --git a/docs/operations/e0-closeout/work-orders/E0-19-merged-match-tier.adoc b/docs/operations/e0-closeout/work-orders/E0-19-merged-match-tier.adoc new file mode 100644 index 00000000..c93e40a2 --- /dev/null +++ b/docs/operations/e0-closeout/work-orders/E0-19-merged-match-tier.adoc @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: CC-BY-SA-4.0 +// Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) += E0-19: A merged match reaches review at most +:wo-id: E0-19 +:wo-repo: hyperpolymath/hypatia +:wo-size: small +:wo-agent: e0-executor +:wo-model: sonnet +:wo-effort: medium +:wo-cap-usd: 5 +:wo-max-turns: 120 +:wo-branch: fix/e0-19-merged-match-tier +:wo-depends: E0-18 + +== Goal + +A recipe that a `category` match or a `fuzzy` match chose never runs with +no person. Its highest tier is `review`. Only an `exact` match can reach +`auto_execute`. + +== Authority + +Owner ruling of 2026-11-14 on `docs/design/2026-11-14-merge-honesty.adoc`, decision 2. + +== Facts measured + +Commit `b660d08`, read on 2026-11-14. + +* `lib/neural/dispatch_guard.ex`, `enforce/2` (line 69) already lowers an + action for a novel finding. +* The tiers are: `auto_execute` from confidence 0.95, `review` from 0.85, + `report_only` below (`.claude/CLAUDE.md`, design constants). +* `E0-18` gives each dispatch its kind of match. + +== Steps + +. Confirm that `E0-18` is merged. Stop if it is not. +. Measure the facts again. Find the 1 place where the tier of a dispatch is + decided, and name it in the pull request body. +. Write the tests first. +. Apply the cap there. An unknown kind of match gets the cap too. + +== Acceptance + +* Tests show: confidence 0.99 with `exact` gives `auto_execute`; confidence + 0.99 with `category`, with `fuzzy` and with `unknown` gives `review`. +* A test shows that the cap never raises a tier. +* `mix test` has 0 failures. No file that was formatted before the change is unformatted after it. + +== Stop and ask + +* The tier is decided in more than 1 place and the places disagree. +* The cost reaches the cap before the acceptance checks pass. diff --git a/docs/operations/e0-closeout/work-orders/E0-20-pattern-identity.adoc b/docs/operations/e0-closeout/work-orders/E0-20-pattern-identity.adoc new file mode 100644 index 00000000..2d7cc14d --- /dev/null +++ b/docs/operations/e0-closeout/work-orders/E0-20-pattern-identity.adoc @@ -0,0 +1,63 @@ +// SPDX-License-Identifier: CC-BY-SA-4.0 +// Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) += E0-20: The pattern identity: design and ruling packet +:wo-id: E0-20 +:wo-repo: hyperpolymath/hypatia +:wo-size: design +:wo-agent: e0-supervisor +:wo-model: opus +:wo-effort: high +:wo-cap-usd: 10 +:wo-max-turns: 160 +:wo-branch: docs/e0-20-pattern-identity +:wo-depends: none + +== Goal + +A design note proposes an identity for a pattern that does not depend on the +wording of a description, with a way to move the old patterns to it. The +owner rules on it. No code changes in this work order. + +== Authority + +Owner ruling of 2026-11-14 on `docs/design/2026-11-14-merge-honesty.adoc`, decision 3. + +== Facts measured + +Commit `b660d08`, read on 2026-11-14. + +* `lib/pattern_registry.ex`, `fingerprint/1` (line 380): the identity is the + rule and the first 30 characters of the description, in lower case, with + each run of other characters as 1 hyphen. +* Same file, line 203: a category with no entry in `@pa_rule_map` (line 51) + gets the rule `PA000`. +* Inferred, no test: 2 findings with the same rule and the same first 30 + characters are 1 pattern, so a label on the outcome row cannot tell them + apart. A reworded description gives a new pattern, and the old one loses + its recipe link. +* `test/pattern_registry_test.exs` has the tag `:verisim_data`. + +== Steps + +. Measure the facts again. Show the 2 inferred results with a test or a + short script, and put the output in the note. +. Count, from `patterns/registry.json` of the test fixtures or of a store + that the owner names, how many patterns have the rule `PA000`. +. Write `docs/design/pattern-identity.adoc`, status PROPOSED. It gives: +.. 2 or 3 candidate keys, each with what it merges and what it splits; +.. what happens to the history and the recipe link of each old pattern; +.. who else reads the pattern id (name each reader with file and line); +.. the option that is most correct for the long term, with its label. +. Write a ruling packet. Stop after it. + +== Acceptance + +* The note has the measured output for both inferred results. +* Each candidate key has 1 example of 2 findings that it keeps apart and 1 + example of 2 findings that it joins. +* `asciidoctor --failure-level WARN` builds the note. The docs index has its line. + +== Stop and ask + +* A candidate key needs data that the scanner does not give today. +* The cost reaches the cap before the note is complete. diff --git a/docs/operations/e0-closeout/work-orders/E0-21-fuzzy-rank.adoc b/docs/operations/e0-closeout/work-orders/E0-21-fuzzy-rank.adoc new file mode 100644 index 00000000..2130e5c9 --- /dev/null +++ b/docs/operations/e0-closeout/work-orders/E0-21-fuzzy-rank.adoc @@ -0,0 +1,57 @@ +// SPDX-License-Identifier: CC-BY-SA-4.0 +// Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) += E0-21: A fuzzy match ranks by its hits +:wo-id: E0-21 +:wo-repo: hyperpolymath/hypatia +:wo-size: small +:wo-agent: e0-executor +:wo-model: sonnet +:wo-effort: medium +:wo-cap-usd: 4 +:wo-max-turns: 100 +:wo-branch: fix/e0-21-fuzzy-rank +:wo-depends: none + +== Goal + +Among the recipes that match a description by keyword, the recipe with the +most keyword hits wins. Confidence decides only between equal hit counts. A +keyword matches a whole word, not a part of one. + +== Authority + +Owner ruling of 2026-11-14 on `docs/design/2026-11-14-merge-honesty.adoc`, decision 4. + +== Facts measured + +Commit `b660d08`, read on 2026-11-14. + +* `lib/recipe_matcher.ex`, `fuzzy_match_recipe/2` (line 156): a recipe + matches when `keyword_hit_count > 0` (line 191). The list is sorted by + confidence only (line 195). +* A hit is `String.contains?(description, kw)` (line 186), so a keyword + matches inside a longer word. +* Inferred, no test: a recipe with 1 hit wins over a recipe with 5 hits and + a lower confidence. The keyword `var` matches `invariant`. + +== Steps + +. Measure the facts again. Stop if one no longer holds. +. Write the tests first, with no `:verisim_data` tag and with recipes of + their own. Each test must fail today. +. Rank by hit count, then by confidence, then by recipe id, so that the + result does not depend on the order of the files. +. Match a keyword as a whole word. + +== Acceptance + +* A test shows that 5 hits win over 1 hit with a higher confidence. +* A test shows that `var` does not match `invariant`. +* A test shows the same result for 2 orders of the recipe list. +* `mix test` has 0 failures. No file that was formatted before the change is unformatted after it. + +== Stop and ask + +* A test of step 2 passes today. Then the inferred result is false: report + it, and correct the table in the design note. +* The cost reaches the cap before the acceptance checks pass. diff --git a/docs/operations/e0-closeout/work-orders/E0-22-fix-check-by-finding.adoc b/docs/operations/e0-closeout/work-orders/E0-22-fix-check-by-finding.adoc new file mode 100644 index 00000000..9542b01d --- /dev/null +++ b/docs/operations/e0-closeout/work-orders/E0-22-fix-check-by-finding.adoc @@ -0,0 +1,61 @@ +// SPDX-License-Identifier: CC-BY-SA-4.0 +// Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) += E0-22: A fix is checked against its own finding +:wo-id: E0-22 +:wo-repo: hyperpolymath/hypatia +:wo-size: medium +:wo-agent: e0-executor +:wo-model: sonnet +:wo-effort: medium +:wo-cap-usd: 8 +:wo-max-turns: 160 +:wo-branch: fix/e0-22-fix-check-by-finding +:wo-depends: none + +== Goal + +A fix counts as failed only when its own finding is still present. Another +finding of the same category in the repository does not make it fail. + +== Authority + +Owner ruling of 2026-11-14 on `docs/design/2026-11-14-merge-honesty.adoc`, decision 4. This work order comes before any measurement of success +rates for each pattern: a rate that rests on the check of today also +measures how often a category occurs twice. + +== Facts measured + +Commit `b660d08`, read on 2026-11-14. + +* `lib/outcome_tracker.ex`, `do_verify_fix/3` (line 202) runs the scanner on + the repository and sets `still_found` when any weak point has the same + category (lines 211 to 214). The argument `pattern_id` is used in the log + line only. +* `record_and_verify/5` (line 320) records the result as a qualified outcome. +* Inferred, no test: a good fix in a repository with 2 findings of 1 + category is recorded as a failure. +* No test covers `do_verify_fix/3`. + +== Steps + +. Measure the facts again. Stop if one no longer holds. +. Read what a weak point of the scanner carries (file, line, rule, + description). Write that list in the pull request body. +. Write the tests first, with a stub scanner on `PATH` that prints a fixed + result. The test with 2 findings of 1 category must fail today. +. Compare the finding by the most exact fields that both sides have. Name + the fields in the pull request body. + +== Acceptance + +* A test shows: the fixed finding is gone and another finding of the same + category stays, and the result is `verified`. +* A test shows: the fixed finding is still there, and the result is a failure. +* A test shows: the scanner fails, and no outcome is recorded as verified. +* `mix test` has 0 failures. No file that was formatted before the change is unformatted after it. + +== Stop and ask + +* The weak point has no field that tells 2 findings of 1 category apart. + Then the fix depends on `E0-20`: say so and stop. +* The cost reaches the cap before the acceptance checks pass. diff --git a/docs/operations/e0-closeout/work-orders/E0-23-placeholder-cluster.adoc b/docs/operations/e0-closeout/work-orders/E0-23-placeholder-cluster.adoc new file mode 100644 index 00000000..2b3652bc --- /dev/null +++ b/docs/operations/e0-closeout/work-orders/E0-23-placeholder-cluster.adoc @@ -0,0 +1,55 @@ +// SPDX-License-Identifier: CC-BY-SA-4.0 +// Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) += E0-23: A placeholder network adds no confidence +:wo-id: E0-23 +:wo-repo: hyperpolymath/hypatia +:wo-size: small +:wo-agent: e0-executor +:wo-model: sonnet +:wo-effort: medium +:wo-cap-usd: 4 +:wo-max-turns: 100 +:wo-branch: fix/e0-23-placeholder-cluster +:wo-depends: none + +== Goal + +A network that returns a fixed placeholder adds nothing to the derived +confidence, and its result says that it is a placeholder. + +== Authority + +Owner ruling of 2026-11-14 on `docs/design/2026-11-14-merge-honesty.adoc`, decision 4. + +== Facts measured + +Commit `b660d08`, read on 2026-11-14. + +* `lib/neural/variational_autoencoder.ex` (lines 80 to 84): the latent + vector is all zeros, and `cluster_id` is always 0. +* `lib/neural/coordinator.ex`, `vae_confidence/1` (line 974) gives 0.6 when + `cluster_id` is not nil. Line 920 gives that value the weight 0.05. +* So each finding gets the same 0.6 at weight 0.05, from no evidence. + +== Steps + +. Measure the facts again. Stop if one no longer holds. +. Write the tests first. +. Mark the result of the placeholder (for example `placeholder: true`). +. Give a placeholder result the weight 0.0 in the derived confidence. +. Do not write a real autoencoder in this work order. + +== Acceptance + +* A test shows that the derived confidence of 1 finding is the same with and + without the placeholder network. +* A test shows the placeholder mark in the result. +* `.claude/CLAUDE.md` says, in the row of the autoencoder, that it is a + placeholder. +* `mix test` has 0 failures. No file that was formatted before the change is unformatted after it. + +== Stop and ask + +* Another network of the 8 also returns a fixed value. List it. Do not + change it here. +* The cost reaches the cap before the acceptance checks pass. diff --git a/docs/operations/e0-closeout/work-orders/E0-24-novelty-fail-closed.adoc b/docs/operations/e0-closeout/work-orders/E0-24-novelty-fail-closed.adoc new file mode 100644 index 00000000..ade9808b --- /dev/null +++ b/docs/operations/e0-closeout/work-orders/E0-24-novelty-fail-closed.adoc @@ -0,0 +1,57 @@ +// SPDX-License-Identifier: CC-BY-SA-4.0 +// Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) += E0-24: An untrained novelty network means novel +:wo-id: E0-24 +:wo-repo: hyperpolymath/hypatia +:wo-size: small +:wo-agent: e0-executor +:wo-model: sonnet +:wo-effort: medium +:wo-cap-usd: 5 +:wo-max-turns: 120 +:wo-branch: fix/e0-24-novelty-fail-closed +:wo-depends: none + +== Goal + +Until the novelty network is trained, each finding counts as novel and goes +to `report_only`. + +== Authority + +Owner ruling of 2026-11-14 on `docs/design/2026-11-14-merge-honesty.adoc`, decision 5 (fail closed). + +== Facts measured + +Commit `b660d08`, read on 2026-11-14. + +* `lib/neural/radial_neural_network.ex`, `detect_novelty/2` (line 145) + returns `:untrained` for a network that is not trained. +* `lib/neural/coordinator.ex` (line 765) sets + `is_novel: match?({:novel, _}, novelty)`, so `:untrained` gives `false`. + `is_novel?/1` (line 999) reads that field. +* `lib/neural/dispatch_guard.ex`, `enforce/2` (line 69) acts on `is_novel: true`. +* `test/neural_dispatch_guard_test.exs` tests the guard with a given + `is_novel`. No test covers the untrained case from end to end. + +== Steps + +. Measure the facts again. Stop if one no longer holds. +. Write the tests first. +. Make `:untrained` give `is_novel: true`. Keep the board status `:unknown`, + so that a reader can tell "untrained" from "novel by distance". +. Find each other caller that asks for novelty, and name each in the pull + request body with its result for an untrained network. + +== Acceptance + +* A test shows: untrained network, confidence 0.99, and the action is + lowered to `report_only`. +* A test shows: trained network and a known finding, and the action is not lowered. +* `mix test` has 0 failures. No file that was formatted before the change is unformatted after it. + +== Stop and ask + +* The change stops every automatic fix in the test suite or in a fixture, + and a test of another module depends on that. Report the count. +* The cost reaches the cap before the acceptance checks pass.