[Epic][LLM Wiki v2] Evaluate reference engines, harden contracts, and govern migration #871

Purpose

Successor to completed #863 (do not reopen or rewrite historical receipts). Evaluate adoption/migration of the PStack project-local .llm-wiki/ against:

• atomicstrata/llm-wiki-compiler @ 0f8ff86ef0a28f4c15627a2ea5e94a96cadd834e (v1.5.0, candidate compiler and review pipeline)
• geronimo-iia/llm-wiki @ 9d974c9e4d9d6c21638a5f520e6d89f79647724e (Rust headless Git/Markdown search, typed graph and MCP/ACP)
• PStack @ e21a196797969691106e6bba095c4438b9ffe643
• QStack governance and historical consolidation #2624, @ 149b6082fe7bcd00e86a13ed48fbf8ee51268882.

Decision policy

Compare keep minimal, optional external engines, full migration in an evidence-based Arena. Default is keep the Python/Markdown baseline until parity, safety, provenance and cost thresholds justify change. No migration by analogy. Avoid requiring Node 24, a daemon, provider credentials, or Rust for the six-host offline portability baseline.

PStack: project-local docs and verification; raw bytes immutable once ingested, no unsafe path traversal, derived index and no silent claims of schema compliance.
QStack: its existing knowledge/ canonical store, knowledge/operations.jsonl authoritative log, .wiki/ quarantine, and governed publisher MUST remain sole mutation authority. Engine output is untrusted candidate/optional projection, not direct publisher. Keep nine stable semantic responsibilities and explicit source/claim provenance.

Audit findings to validate in first child

• scripts/llm_wiki.py::ingest_file writes dest.write_bytes(content) even when raw/sources/<name> already exists, contradicting raw immutability.
• check_wiki checks file presence and Markdown links but does not actually validate typed page schemas, source hashes, or index equivalence. Path confinement and symlink handling need negative controls before trusting an untrusted source.
• index --write has no atomic replace; indexing scans category root *.md only; whether that is deliberate must be specified.
  These are source-inspection findings, not executed exploit or runtime proof.

DAG (logical dependencies, not GitHub-native subissues)

S0 exact-head inventory/security+requirements -> A0 scored Arena benchmark -> F1 verification contracts, F2 optional candidate adapter proof -> F3 docs/wiki and semantic primitive alignment -> V0 independent Thermos, Swarm/Arena/Interrogate, exact-head suite and release decision -> root.
F1 and F2 may run in parallel after A0. Migration and canonical publication are separate decisions. Each child must link code/PR, exact revision, reproducible commands, failure-first results (FAIL > BLOCKED > UNTESTED > PASS) and negative controls.

Exit criteria

• Source and schema crosswalk with data/code flow and rollback.
• Real user-path verifier generated using create-verification-skill and formal model/check generated using create-formal-verification-skill, with executable negative controls; absent checker means BLOCKED, not PASS.
• Rights-safe source-capture, symlink/path escape, concurrent writes, crash recovery, stale CAS, review approval and ledger identity tests.
• Side-by-side corpus compile / retrieval / failure / footprint benchmark, sources and model versions pinned; independently reviewed.
• No merge on incomplete CI or invented multi-model evidence. A draft planning PR is not a migration implementation.

Materialized graph (2026-10-31)

Draft research PR: #878 on research/llm-wiki-reference-eval-871 (planning-only, not a verified migration).
Design/spec source: docs/research/llm-wiki-reference-evaluation-2026-10-31.md.
Compiled wiki analysis: .llm-wiki/wiki/analyses/llm-wiki-reference-evaluation.md.

flowchart TD
  S0["#872 S0 inventory/security"] --> A0["#873 A0 Arena"]
  A0 --> F1["#874 F1 verification + formal"]
  A0 --> F2["#875 F2 candidate adapter pilot"]
  F1 --> F3["#876 F3 specs + wiki"]
  F2 --> F3
  F3 --> V0["#877 V0 Thermos + Interrogate"]
  V0 --> Root["#871 acceptance"]

Initial frontier: #872. S0/A0/F1/F2/F3/V0 were created as open work, not verified completion. Logical links exist in bodies; no GitHub-native parent linkage claimed. Do not close this epic when merely creating the plan. If the #875 pilot is explicitly deferred by #873, mark it not_planned with rationale rather than fabricating its results.

Implementation follow-on, verified offline slice (2026-10-31)

• Source-pinned research and compiled wiki: draft #878, CI 41213291228 SUCCESS.
• Write-once CLI safety, negative tests, bounded real-code verification skill/formal-analysis skill and wiki schema alignment: stacked draft #879 at 0029b35fad97aa19fd67c42f35bd0ff3ec6f15ce, CI 41215096803 SUCCESS. Inspected earlier identical implementation tests at run 41214985678: existing wiki 9 passed, 11 safety, 3 model, 39 bounded real-code traces + counterexample to an overwrite mutant.
• Provisional Arena options documented in [LLM-Wiki-v2][A0] Arena decision: keep vs optional adapters vs migration with pinned corpus #873; comparison is source-reviewed only, not an independent multi-model outcome. Neither upstream engine was installed/executed; no live model/provider, TLC, external benchmark, or six-host observed_live receipt. No PR merged or issue closed on documentation alone.
• Status policy: [LLM-Wiki-v2][S0] Exact-head inventory, security audit, and schema authority crosswalk #872 S0 / [LLM-Wiki-v2][A0] Arena decision: keep vs optional adapters vs migration with pinned corpus #873 A0 still OPEN; [LLM-Wiki-v2][F1] Build real and formal wiki verification skills; close safety and schema gaps #874 F1 has a tested partial implementation but remains OPEN for remaining formal/security and independent review; [LLM-Wiki-v2][F2] Optional upstream engine adapters: candidate-only parity pilot and fallback #875 optional SDK/engine evaluation OPEN; [LLM-Wiki-v2][F3] Recompose formal specs, semantic blocks and project-local wiki navigation #876 spec/wiki acceptance and [LLM-Wiki-v2][V0] Exact-head Thermos, adversarial review and migration acceptance #877 final closeout OPEN. A source-reviewed CI PASS does not satisfy cross-repo write authority or formal/host-runtime proof.

Stacked mainline restoration and formal specs (2026-10-31)

• GitHub base-ref deletion closed former fix(wiki): write-once capture and real CLI/formal-bounded verification (#874) #879 without merge. New draft PR #881 on current main restores 16 tested safety/verification files; its exact-head CI #41216358045 SUCCESS. Do not count old fix(wiki): write-once capture and real CLI/formal-bounded verification (#874) #879 or new fix(wiki): mainline write-once capture and verification skills (#874) #881 as merged while draft.
• New stacked draft PR #884 on branch spec/llm-wiki-reference-contracts-871 adds source-pinned OpenSpec requirements, candidate data models, 19 CI-verified structural fixture tests and indexed wiki synthesis, with corrected docs(arena): record Arena decision on LLM wiki engine options (#871, #873) #880 unsupported performance/host/security PASS claims. Exact-head CI #41216989067 SUCCESS.
• Existing [LLM-Wiki-v2][S0] Exact-head inventory, security audit, and schema authority crosswalk #872/[LLM-Wiki-v2][A0] Arena decision: keep vs optional adapters vs migration with pinned corpus #873/[LLM-Wiki-v2][F1] Build real and formal wiki verification skills; close safety and schema gaps #874/[LLM-Wiki-v2][F2] Optional upstream engine adapters: candidate-only parity pilot and fallback #875 are recorded as CLOSED in GitHub, but issue body text still contains prior OPEN TODOs and [LLM-Wiki-v2][F1] Build real and formal wiki verification skills; close safety and schema gaps #874's source-hardening PR remained unmerged when it was closed. Do not silently equate closed issue with shipped code; [LLM-Wiki-v2][F2] Optional upstream engine adapters: candidate-only parity pilot and fallback #875 deferral (no optional adapter) remains the binding option-A choice. Keep [LLM-Wiki-v2][F3] Recompose formal specs, semantic blocks and project-local wiki navigation #876/[LLM-Wiki-v2][V0] Exact-head Thermos, adversarial review and migration acceptance #877 and root open until actual merge/review proof.
• Unsupported: external engine execution, cross-platform observed-live six-host runs, complete adversarial symlink race safety, source+log crash atomicity, TLC, full independent Swarm/Arena/Interrogate, cited claim entailment, external benchmark. Offline test success not formal proof.