docs: sync documentation with recent changes

[nadeem4]

Review window

Prior "docs: sync documentation with recent changes" PR: #135, created 2026-09-22T12:17:37Z. This reviews everything merged into main since then: #133, #137–#165 (28 PRs merged after that timestamp; excludes #135 itself and #153, which was closed without merging).

Four parallel audits covered this window, each cross-checked against the current code on main, not just the PR diffs: (1) API/SDK — auth, registry, facade, dead-code removal; (2) pipeline/architecture — generator, validator, decomposer, aggregator fixes; (3) playground/demo — page nav, sample databases, hosted mode; (4) eval/benchmark — tier 2 scoring, regression gate.

Most of the window already keeps docs in sync as part of each PR's own commits — in particular #154, #156, #157, #160, #163, #164, #165 all touched README.md/docs/ themselves. This PR covers what survived that self-documentation.

Corrections

• README.md — tier 2 benchmark CLI table and gate description: the nl2sql benchmark --tier 2 row's flag list and the "--baseline fails on a regression" sentence still described the old flat accuracy-drop gate. feat(eval): alternative gold answers, confidence intervals and a paired regression gate #156 added --max-regressions (default 2: how many questions may flip pass→fail before the run fails) as the primary gate, made --max-accuracy-drop optional and deprecated, and switched the comparison to a paired McNemar significance test on flipped questions (packages/nl2sql/src/nl2sql/cli/main.py:361-375). Added --max-regressions, --max-accuracy-drop (deprecated) and the pre-existing but likewise-missing --max-cost-increase to the table, and rewrote the gate description.

  • Necessitated by: feat(eval): alternative gold answers, confidence intervals and a paired regression gate #156

• docs/adapters/architecture.md, docs/extensions/add-adapter.md, docs/adapters/sdk.md: all three still told adapter authors to declare supports_schema_introspection, supports_dry_run or supports_cost_estimate capability flags. chore: remove dead code and fix the timeout message and CANCELLED code #163 pruned DatasourceCapability down to only SUPPORTS_SQL and SUPPORTS_REST (packages/adapter-sdk/src/nl2sql_adapter_sdk/capabilities.py) and fixed most references itself, but missed these three spots — following add-adapter.md's old instructions today would raise AttributeError on the deleted enum members. Also fixed sdk.md's claim that fetch_schema_snapshot() is "required only if supports_schema_introspection is advertised" — it's a required protocol method now (packages/adapter-sdk/src/nl2sql_adapter_sdk/protocols.py:26), not gated by any capability flag.

  • Necessitated by: chore: remove dead code and fix the timeout message and CANCELLED code #163

• packages/nl2sql/README.md — three gaps, all in a file docs(playground): describe the pages and several databases in the README and rail #160 and feat(demo): hosted mode so a public demo can take each visitor's own key #165 edited elsewhere but not here:

  • Still said nl2sql demo writes a project "with the Chinook sample database" (singular). feat(demo): add support and web-analytics sample databases linked to Chinook customers #157 added two more sample databases; the demo now copies in three (packages/nl2sql/src/nl2sql/cli/demo/manager.py:67-68: chinook, support, webanalytics), as root README.md:100-101 already correctly says.
  • Still said "a key pasted into the playground's Settings panel". feat(playground): give Settings and Retrieval their own pages with a header nav #154 gave Settings its own routed page (#/settings); feat(playground): give Settings and Retrieval their own pages with a header nav #154's own diff fixed this phrase everywhere else (docs/configuration/llm.md, docs/configuration/secrets.md, docs/getting_started/demo.md) but missed this file.
  • Never mentioned nl2sql demo --hosted at all. feat(demo): hosted mode so a public demo can take each visitor's own key #165 added a public-facing hosted mode (no server-side key; each visitor supplies their own, packages/nl2sql/src/nl2sql/cli/demo/playground/hosted.py) and documented it in root README.md:149-155, web/playground/README.md and docs/getting_started/demo.md, but this package-level README — which is the one PyPI actually shows — had no mention of it.
  • Necessitated by: feat(demo): add support and web-analytics sample databases linked to Chinook customers #157, feat(playground): give Settings and Retrieval their own pages with a header nav #154, feat(demo): hosted mode so a public demo can take each visitor's own key #165

Verification

• Every claim above was checked against the current code on main, not just the PR diffs.
• pip install -r requirements-docs.txt && python -m mkdocs build --strict: fails with exactly one warning — mkdocs-mermaid2-plugin cannot reach unpkg.com to verify the mermaid.js CDN URL, rejected by this sandbox's outbound network policy (connect_rejected, gateway 403 per $HTTPS_PROXY/__agentproxy/status). This is the identical sandbox-only failure documented by all four prior sync PRs (docs: sync documentation with recent changes #60, docs: sync documentation with recent changes #77, docs: sync documentation with recent changes #103, docs: sync documentation with recent changes #135). With no other warnings, every content and nav validation passes.

Not touched

• CHANGELOG.md and version numbers (release-please owns those).
• Everything else in fix(generator): quote select aliases and column names that are not bare identifiers #133, fix(validator): only allow known function names in plans #137–feat(eval): report a lenient accuracy that allows extra and reordered columns #152, feat(playground): give Settings and Retrieval their own pages with a header nav #154, feat(eval): alternative gold answers, confidence intervals and a paired regression gate #156–feat(demo): hosted mode so a public demo can take each visitor's own key #165 already kept in sync via each PR's own commits (verified, not assumed, by the four audits above).

---

🤖 Generated with Claude Code

https://claude.ai/code/session_01Bft4Niqsd2GWnPopt1VAhd

---

Generated by Claude Code