Repository navigation
Replace the prober with ValueRanges' oracle kit: claims, surface coverage, oracles and per-oracle seeds #97
Description
The prober costs more than it pays here. prober seeds builds once per seed: 170 seeds at about four minutes each, so a release's seeds --all takes roughly eleven hours, and a change to one fault-response file stales 23 seeds (about 90 minutes) of five probes. The replay, the footprints, the ledger, the covering arrays and the release gate carry the same weight. CodoMetis.ValueRanges kept the parts of this approach that pay for themselves (its docs/oracles.md): no covering arrays, no ledger, no replay footprints, no agent.
Proposal
Replace the prober with ValueRanges' oracle kit, ported to xunit v3:
test/oracles/claims.json: one entry per consumer-facing promise, with itssourceheading, a verbatimquotethat a conventions test finds in the document, itssurfaces, andheldBy, the test classes that hold it. An emptyheldByis the worklist.- API-surface coverage: every public member of the shipping assemblies is named by a claim's
surfaces, or waived with a reason insurface-waivers.json. oracles.json: the registry of test classes that are oracles (*OracleTests,*ProbeTests), ordinary tests in the projects that have their harness.seeds/*.patch: one or more per oracle, each naming the oracles it fails. CI checks without building that every oracle has a seed and every seed still applies.dotnet run tools/oracles.cs seedsproves locally that each is caught by building one project and running one class, before a release.candidates.json: known disagreements tied to an issue, so a sweep stays green without hiding them.axes.md: hand-written, the settings a consumer varies and the pairs no test varies yet.
Retired:
tools/Proberand its tests, the corpus undertools/probe/, and the 170 seeds;- the
probesandprobe-statusjobs ofdotnet.yml, and the replay and gate steps ofrelease.yml; - the probe conventions tests,
.claude/probe-axes.md, theprobe-triageskill and the post-commit replay hook.
One decision record supersedes decisions 96, 100, 101, 103, 104, 107 and 108, and the gate part of 97. DocsQuery keeps its facts. The probes that found a bug are ported as oracle classes; the rest go.
Done when
- No file in the repository runs, builds or documents the prober, and CI and the release run without it.
- The oracle kit's conventions tests pass in CI, and every registered oracle has a seed that the local tool shows caught.
- AGENTS.md, CONTRIBUTING.md and
design/plan.mddescribe the new approach.
Activity
CaffeinatedCoder commentedon Oct 28, 2026
Done in PR #108, merged into release/2.3.0.