diff --git a/CHANGELOG.md b/CHANGELOG.md index 30824c6..31efd52 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,11 @@ three commits past it), and a bug report can name a release instead of a sha nob Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- fix(project-bin/context-pack.sh): **the pack finds the brief in both layouts.** It looked only for `architecture/modules/-brief.md`; the toolkit's own convention (`iterative-build-loop.md`) and two real specs use `architecture/modules//module-brief.md`, so every dispatch there exited 2 ("no brief") and the helper fell back to a reading list. The folder form is tried first, then the flat form; `--brief` still overrides. Probed on a real app's scratch copy in both layouts: identical packs, exit 0; an unknown module still exits 2 and names both paths. — MendixMau +- fix(project-bin/context-pack.sh): **the pack names the rule files a step must read, and an entity step no longer stops to ask for a folder.** A new "Rules" section lists the files by step kind: the preflight STOP table always, microflow patterns for logic, page pre-flight plus the widget syntax skill for pages, and the domain-model skill for entities. Both mxcli skill layouts resolve (`.md` and `/SKILL.md`). Why: in a five-step module bench on a real app's scratch copy (pack vs no pack, two lanes each), helpers given a pack skipped skill reads the pack did not mention, and one reported the preflight skill "not available" although it was installed. An entity step whose name is not in the folder plan was told "escalate", and the helper stopped; the cell now says entities, associations and enumerations live in the domain model. — MendixMau +- new(project-bin/context-pack.sh): **a helper gets one file for its build step instead of a reading list.** The module brief gains a `### Build steps` table (one row per dispatch: what the step builds, which existing elements it reads, the app's own example to copy, and the `mxcli brain` slice). `bin/context-pack.sh ` turns that row into one pack: the folder from the brief's folder plan, the *why* from `mxcli brain brief`, the brief's arch constraints, and a live `DESCRIBE` of every element it reads, of the example, and of anything the step changes rather than creates. Nothing is copied into the brief; the pack is regenerated per dispatch and never committed. A name the model does not have exits 1 and is listed, so a typo in the brief fails at dispatch, not inside the helper's script. `iterative-build-loop.md` now turns the project brain on at build start (`brain init`, capture with forward anchors, `brain plan` as the progress measure) and has each dispatch carry a pack; `mdl-agent` reads the pack first and captures a *why* it learned with `mxcli brain capture`. Field run: a real app's scratch copy, one bulk-accept microflow, 3 helper runs with a pack vs 3 without, same brief and brain: ~9% fewer tokens (130k vs 144k), ~40% fewer tool calls (13 vs 21) and ~40% less time (60 s vs 101 s); every run passed `mxcli check`, exec'd with 0 new mx errors and used the planned folder. Probed on single-tree and two-tree (`app/`) layouts; not yet run on Windows. — MendixMau +- fix(project-bin/context-pack.sh): **the pack now names the traps a step walks into.** A page step lists the attributes of every entity it reads that a `TEXTBOX` cannot bind (DateTime, Boolean, enumeration) — `mxcli check` passes them, mx check fails with CE2421. A step that changes an existing element gets its `mxcli impact` list, so the helper keeps its callers' contract without looking them up. The brain brief's headings now nest under the pack's own. Field run on a real app's scratch copy, 10 helper runs across five step kinds (new page, change an existing microflow, a misspelt name in the brief row, a 5k-token pack, no brain yet): the two page runs with the earlier pack each hit CE2421 (1 and 3 errors) although the types were in the pack; the two re-runs with the watch-out had 0, as did the two runs with no pack. Page runs with a pack used ~25% fewer tool calls and ~30% less time than without (n=2 each; token counts too noisy at n=2 to quote). The change run moved the microflow into its planned folder and kept both roles' execute rights; the misspelt-name run reported the typo and used the real association; microflow packs are byte-identical to before. — MendixMau +- learn(process/context-pack-bench-2026-10-01.md): **the context-pack A/B bench is written up, with where the pack should live.** 16 scored helper runs on a scratch copy of a real app: all pass check and exec; pages lost CE2421 errors (4 to 0) once the watch-out landed; tool calls about 25-38% fewer and time about 30-40% less; tokens -9% on microflows but not proven on pages (noise 69k to 173k). Proposes the split: mxcli owns turning names into one context document (the read-side twin of folding `mx check` into `mxcli check`), the toolkit keeps the brief row, folder plan and dispatch. — MendixMau - fix(bin/context-audit.sh): **a Read `offset` or `limit` stored as a string no longer crashes the audit, and the script now exits 0 as its header promises.** Older transcripts store these as strings, sometimes as junk like `'30, 90'`, and `offset - 1` raised a TypeError in the embedded reader, which stopped the whole run. They are now read as numbers when they parse and fall back to the Read tool's defaults (offset 1, limit 2000) when they do not; if the reader ever dies on an unseen transcript shape, the script says the numbers are partial and still exits 0. Field run: 1,300 sessions and 724 subagent runs on a Mac (2026-08-27 to 2026-09-30), which crashed on the old version. — MendixMau - fix(routing): **`learned-mcp-patterns.md` is no longer always-on in the build stage; it loads before the first MCP write in a session.** It sat in the Stage 5 baseline pack and in `mdl-agent`'s always-read rows, so every build session and every MDL helper agent carried ~4,900 tokens of MCP save/handoff rules and JSON payloads, including sessions that never open Studio Pro and cloud containers where MCP does not exist. Choosing the write mode is already Step 0 of `learned-mdl-preflight.md`, which stays always-on, so nothing is lost at the moment of choice; the MCP skill's trigger now names the moment it is needed (`mxcli --mcp` exec or a `pg_*`/`ped_*` call). Stage 5 pack: 74,055 → 71,443 words, 23 → 22 files; baseline 79,752 → 77,140 words. Found by the context report (`bin/context-audit.sh`, `bin/render-routing.sh --check`). — MendixMau - new(bin/context-audit.sh): **what fills the context window, per file, from the real Claude Code transcripts on this machine.** `token-burn.sh` says how many tokens a project burned; this says which files burned them, so decisions about splitting, trimming or un-routing a skill rest on measured runs instead of `wc` on the skill files. It reports: the context size before any work (first call, input + cache, for sessions and for subagents separately); the instruction files loaded every run (CLAUDE.md, CLAUDE.local.md) and their size; every file read (Read tool and simple shell reads like `cat`, `sed -n`, `git show REV:path`, following `cd` and `VAR=`) with reads, sessions, total and per-read size, and re-reads within a session (paging through a file is not a re-read; asking for the same part again is); other tool output by tool; and each compaction with the files read before it. Project names are masked by default (`project-1/architecture/modules/*.md`), so the output is safe to paste; `--names` shows them locally. First field numbers, from a captured pipeline-start subagent: it starts at 52,503 tokens before reading anything, then reads the runbook in 6 pages with 2 repeats. Fixture: `tests/wave2/test-context-audit.sh` over a scrubbed real capture. — MendixMau diff --git a/README.md b/README.md index 3694cca..b039ff4 100644 --- a/README.md +++ b/README.md @@ -596,6 +596,7 @@ Every mxcli project has a `.ai-context/skills/` directory (bundled by `mxcli ini | Task | Skill to load | |---|---| +| Dispatching a build step to a helper — generate its one-file pack (the brief's Build steps row, mxcli brain brief, live DESCRIBEs, example, folder) and hand over the path instead of a reading list. Measured: ~9% fewer tokens, ~40% fewer tool calls, same quality | `project-bin/context-pack.sh` | | Building a module with mxcli — verified, iterative, coverage-checklist gated | `skills/iterative-build-loop.md` | | After marking a module done, or any time "how much is built vs proven" is asked — renders build-plan.html from done- prefixes and verify-module.sh/improvement-register.md, kept as two honestly separate views; --json writes architecture/build-plan.json parsed from build-plan.md's Phase headings (a plan with no Phase headings gets no file) | `project-bin/build-plan-status.sh` | | Turning a client-derived Mendix app into a clean, shareable demo with zero client fingerprint — branding, data, custom widgets | `skills/anonymize-client-app-for-demo.md` | diff --git a/ROUTING.md b/ROUTING.md index c9e7af5..9965123 100644 --- a/ROUTING.md +++ b/ROUTING.md @@ -134,6 +134,7 @@ picks the row up. That is the whole procedure — there is no second list to rem |---|---|---|---|---| | Reviewing any module before calling it done — the ONE pass: build, gate, prove, LOOK (is it logical, does it look right, does it match our design, over every page not just the tested ones), confirm with the denominator stated | `skills/module-review.md` | mdl,review,test | 5,6 | baseline | | Before any mxcli exec / exec.sh / --mcp write — ask or run? the knob decides | `bin/exec-approval.sh` | mdl,gate | 5,6 | baseline | +| Dispatching a build step to a helper — generate its one-file pack (the brief's Build steps row, mxcli brain brief, live DESCRIBEs, example, folder) and hand over the path instead of a reading list. Measured: ~9% fewer tokens, ~40% fewer tool calls, same quality | `project-bin/context-pack.sh` | mdl | 5 | ondemand | | Building a module with mxcli — verified, iterative, coverage-checklist gated | `skills/iterative-build-loop.md` | mdl,gate | 5 | ondemand | | After marking a module done, or any time "how much is built vs proven" is asked — renders build-plan.html from done- prefixes and verify-module.sh/improvement-register.md, kept as two honestly separate views; --json writes architecture/build-plan.json parsed from build-plan.md's Phase headings (a plan with no Phase headings gets no file) | `project-bin/build-plan-status.sh` | architect,gate,review | 4,5,6 | ondemand | | Turning a client-derived Mendix app into a clean, shareable demo with zero client fingerprint — branding, data, custom widgets | `skills/anonymize-client-app-for-demo.md` | mdl,review | 6 | ondemand | diff --git a/agents/mdl-agent.md b/agents/mdl-agent.md index 999bbce..5552d99 100644 --- a/agents/mdl-agent.md +++ b/agents/mdl-agent.md @@ -68,6 +68,7 @@ Open a file when its When cell happens in your task, not all of them at the star | `skills/learned-stylegallery.md` | Building or using the in-app design gallery | | `project-bin/check-design-portability.sh` | Before porting ds.css into SCSS, and at the Stage-3 gate — greps the stylesheet for rules that cannot match the HTML Mendix emits (rem against the real root, table/th/td selectors, positional row selectors). mx check, mxcli check and mxcli lint are all blind to CSS | | `skills/learned-mcp-patterns.md` | Before the first MCP write in a session (Studio Pro open: `mxcli --mcp` exec, or pg_*/ped_* calls) — save after every write, the handoff sequence, confirmed JSON payloads. Choosing the write mode itself is Step 0 of learned-mdl-preflight.md | +| `project-bin/context-pack.sh` | Dispatching a build step to a helper — generate its one-file pack (the brief's Build steps row, mxcli brain brief, live DESCRIBEs, example, folder) and hand over the path instead of a reading list. Measured: ~9% fewer tokens, ~40% fewer tool calls, same quality | | `bug-logs/mxcli-bugs.md` | Reading a whole class of tool defects (a retest, a new mxcli release, an audit) — for one CE code or symptom use bin/bug-lookup.sh instead; the ledger is 32k words | | `skills/cloud-dev-environment.md` | Setting up or resuming an mxcli project in a cloud/ephemeral container — the one-time setup order (mxcli download → mxcli init → init-project.sh → sources decision → push) and the commit-and-push loop that survives container reclaim | | `skills/microflow-loop-antipatterns.md` | Reading what loop bodies do (LOOP_TQ, deferred commit, nested loop, REST in loop, transaction control per item, scheduled-event reachability) from described MDL; the catalog holds top-level activities only and cannot see inside a loop | @@ -125,6 +126,12 @@ this summary. The hard STOPs below are inline on purpose; never route around the the Wiring block; format in `module-brief.md`): roles/access, screens, validation, write-mode plan, and pointers to wireframes/domain MDL. **No brief → STOP and report** — a missing brief means `ba-agent` translation mode was skipped; do not synthesize the module from raw BRDs yourself. +- **Got a context pack?** If the dispatch names one (`.mxcli/packs/-.md`), read it + right after this file: it already holds the brief's row for this step, the brain brief, live + DESCRIBEs of what the step reads, the app's example to copy and the target folder. Don't + re-DESCRIBE what it shows; open the brief or BRD only for what it lacks. +- **Learned a *why* worth keeping?** (a pattern chosen, a trap hit) `mxcli brain capture "" -a + @Module.Element` — capture only and name the id in your report; a person promotes it. - **An unchecked open question in the brief is a stop sign.** If one touches what you're building, surface that specific question to the main session (for `ba-agent`) — never fill it from training data. - Business rules come from the brief and {{BUSINESS_RULES_SOURCE}}; read the domain-model script diff --git a/bin/lib/install-manifest.sh b/bin/lib/install-manifest.sh index 23cedd6..784eb5e 100644 --- a/bin/lib/install-manifest.sh +++ b/bin/lib/install-manifest.sh @@ -106,7 +106,7 @@ MXTK_AGENTS_STAGE_BUILD="mdl-agent.md gate-agent.md test-agent.md review-agent.m # directory. prototype-route.js is the one reader and writer of design/prototype.html's section # format, so page-fidelity.js, check-page-shell.sh and check-prototype-links.js resolve it as a # sibling too. -MXTK_PROJECT_BIN="assemble-prototype.js prototype-route.js check-prototype-links.js wf-add-path-terminators.py wf-set-call-captions.py mxunit_bson.py _common.sh app-facts.sh _claims.sh snapshot-mpr.sh restore-mpr.sh exec.sh save-sp.sh restart-sp.sh check-sp-health.sh verify-module.sh test-stack-up.sh fixture-manifest.sh check-root-clean.sh lint-gate.sh close-task.sh conformance-check.sh coverage-preflight.sh graph-sweep.sh review-module.sh coherence-cadence.sh build-plan-status.sh done-drift-check.sh page-scope.sh render-improvement-register.sh check-design-portability.sh check-design-reaches-app.sh check-page-shell.sh page-fidelity.js model-stamp.sh verify-model.sh install-project-hooks.sh session-check.sh constants-audit.sh" +MXTK_PROJECT_BIN="assemble-prototype.js prototype-route.js check-prototype-links.js wf-add-path-terminators.py wf-set-call-captions.py mxunit_bson.py _common.sh app-facts.sh _claims.sh snapshot-mpr.sh restore-mpr.sh exec.sh save-sp.sh restart-sp.sh check-sp-health.sh verify-module.sh test-stack-up.sh fixture-manifest.sh check-root-clean.sh lint-gate.sh close-task.sh conformance-check.sh coverage-preflight.sh graph-sweep.sh review-module.sh coherence-cadence.sh build-plan-status.sh done-drift-check.sh page-scope.sh render-improvement-register.sh check-design-portability.sh check-design-reaches-app.sh check-page-shell.sh page-fidelity.js model-stamp.sh verify-model.sh install-project-hooks.sh session-check.sh constants-audit.sh context-pack.sh" # Files in project-bin/ that are deliberately NOT installed into projects. The reverse check # below flags anything named by NEITHER list, so a new file in project-bin/ has to be either diff --git a/bin/lib/skill-routing.tsv b/bin/lib/skill-routing.tsv index fad34c6..4591281 100644 --- a/bin/lib/skill-routing.tsv +++ b/bin/lib/skill-routing.tsv @@ -108,6 +108,7 @@ check-prototype-links project-bin/check-prototype-links.js Before wireframes pas learned-mcp-patterns skills/learned-mcp-patterns.md Before the first MCP write in a session (Studio Pro open: `mxcli --mcp` exec, or pg_*/ped_* calls) — save after every write, the handoff sequence, confirmed JSON payloads. Choosing the write mode itself is Step 0 of learned-mdl-preflight.md mdl 5 ondemand build/mdl module-review skills/module-review.md Reviewing any module before calling it done — the ONE pass: build, gate, prove, LOOK (is it logical, does it look right, does it match our design, over every page not just the tested ones), confirm with the denominator stated mdl,review,test 5,6 baseline build exec-approval bin/exec-approval.sh Before any mxcli exec / exec.sh / --mcp write — ask or run? the knob decides mdl,gate 5,6 baseline build +context-pack project-bin/context-pack.sh Dispatching a build step to a helper — generate its one-file pack (the brief's Build steps row, mxcli brain brief, live DESCRIBEs, example, folder) and hand over the path instead of a reading list. Measured: ~9% fewer tokens, ~40% fewer tool calls, same quality mdl 5 ondemand build testing-shape skills/testing-shape.md Before calling any module tested — what testing a module means, and the false-green register of confirmed ways a test reports green over a broken feature test,gate,review 5,6 baseline verify verify-module project-bin/verify-module.sh Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh mdl,gate,test,review 5,6 baseline verify mxcli-bugs bug-logs/mxcli-bugs.md Reading a whole class of tool defects (a retest, a new mxcli release, an audit) — for one CE code or symptom use bin/bug-lookup.sh instead; the ledger is 32k words mdl,gate 5,6 ondemand diagnose diff --git a/process/context-pack-bench-2026-10-01.md b/process/context-pack-bench-2026-10-01.md new file mode 100644 index 0000000..032c61a --- /dev/null +++ b/process/context-pack-bench-2026-10-01.md @@ -0,0 +1,134 @@ +# Context pack — controlled A/B bench, 2026-10-01 + +**Question:** when a build helper gets one generated file for its step (`project-bin/context-pack.sh`) +instead of a reading list, does it build as well or better, and does it cost less? + +**Method.** All runs used a scratch copy of one real Mendix app (mxcli v0.24.0). The real model was +never touched. The app's module brief got a `### Build steps` table (five rows) and a folder plan. +Each run was a fresh drafting agent with the same model and one build step to do. It got either: + +- **pack**: the step's `context-pack.sh` output (brief row, `mxcli brain brief` for the WHY, live + DESCRIBE of every name the step reads or copies, folder plan), or +- **nopack**: the same brief row plus the usual reading list (brief, skills, "DESCRIBE what you need"). + +Each run was scored mechanically: + +1. `mxcli check --references` on the script. +2. `check-page-shell` for page steps. +3. Exec into a **fresh** copy of the model, exit code. +4. `mx check` on that copy, counting only errors that were **not** there before (baseline error list). +5. A folder and access-rights check against the folder plan. + +Token, tool-call and wall-time figures come from each agent's own usage report. + +Two rounds: + +- **r2**: microflow step, 3 pack and 3 nopack runs. +- **r3**: five step kinds on the pack. These were a new microflow (big), a change to an existing + microflow (move it and keep its rights), a brief row with a typo'd name, a run with no brain + available, and a page. The page step also got 2 nopack runs. After the first page runs, the + fixes below were made and the page step was run twice more ("pack v2"). + +## Results + +**Correctness.** All 16 runs passed `mxcli check` and exec with rc 0. Every run put its element in the +planned folder. The change run kept the original access rights. The typo run reported the bad name +instead of guessing (the pack exits 1 and names the missing element). + +The one quality failure was on pages: + +| Page runs | New `mx check` errors | +|---|---| +| pack (first version), run 1 | 1 × CE2421 | +| pack (first version), run 2 | 3 × CE2421 | +| nopack, runs 1–2 | 0 | +| pack v2 (with widget watch-out), runs 1–2 | 0 | + +CE2421 means a `TEXTBOX` is bound to a DateTime or enumeration attribute. `mxcli check` passes it; +only `mx check` fails it. The pack showed the attribute types, but the helpers still reached for +`TEXTBOX`. The nopack runs happened to read a page skill that steers to `DYNAMICTEXT`. + +**Cost.** Averages per run: + +| Step | Arm | Runs | Tokens | Tool calls | Seconds | +|---|---|---|---|---|---| +| Microflow (r2) | nopack | 3 | 144k | — | — | +| Microflow (r2) | pack | 3 | 130k (−9%) | −38% | −41% | +| Page (r3) | nopack | 2 | 101.7k | 40 | 306 | +| Page (r3) | pack, all versions | 4 | 99.7k (−2%) | 29.5 (−25%) | 214 (−30%) | + +**Tokens are noisy.** Pack page runs ranged from 69k to 173k. Without the 173k run, the pack average +is −18%. With two to four runs per arm, the token saving is **not proven**. Fewer tool calls and less +time held in every pair. + +The other r3 pack runs (one each) used 113k–144k tokens, 12–19 tool calls, and 60–87 s. + +## Conclusions acted on + +- **Page widget watch-out** (`dfbb16a`). For a page step, the pack now lists each read entity's + DateTime, Boolean and enumeration attributes under "not TEXTBOX-bindable", with the replacement + widget. Result: 4 CE2421 errors over 2 runs before, 0 over 2 runs after. Microflow packs are + byte-identical to before. +- **Impact list for changed elements.** For a step that changes an existing element, the pack now + includes `mxcli impact` ("depends on — keep their contract"). The change helpers no longer search + for callers by hand. +- **Brain headings nested under "## Why"**, so the pack reads as one document. + +## Open items + +- **Tokens not proven.** It needs more runs per arm, or a larger task where context reading is a + bigger share. +- **Module-scale run** (next). It covers five dependent steps building one unbuilt slice: action + microflow, detail page, open-detail microflow, data source, overview page. Each step runs on the + model the previous step left. That is two arms, two lanes each, so 20 helper runs. It tests what + this bench cannot: does the pack stay right when it is regenerated from a model that earlier + helpers changed? +- **`page-fidelity.js` returned null (0/0) on this wireframe in every run.** This is an instrument + gap and needs a separate fix. +- **Windows not run.** `context-pack.sh` is Bash 3.2 and uses no mac-only tools, but it has no Git Bash + run yet. + +## Where it belongs: CLI or toolkit + +The pack does two different jobs. They belong in different places. + +**1. Reading the model: belongs in mxcli.** Most of the pack is mechanical: + +- DESCRIBE each name; +- `impact` for changed elements; +- `brain brief` for the slice; +- type-based warnings such as "these attributes can't take a TEXTBOX". + +Today the script makes about 6–12 separate mxcli calls for this. A helper without the pack makes the +same calls itself, one tool call each. That is where its extra 10+ tool calls go. + +mxcli R&D is already folding `mx check` and `mxcli check` into one call. The read side is the natural +twin: + +| Today | One call | +|---|---| +| `mxcli check` + `mx check` (+ `check-page-shell`) | `mxcli check` that also runs the mx rules, e.g. CE2421, which `mxcli check` misses today | +| DESCRIBE × n + `impact` + `brain brief` + `context` | something like `mxcli context --for-step --example --slice `: definitions, dependents and WHY in one document | + +In mxcli the type rules would live next to the checker that enforces them. The watch-out list would +then be generated from the same rule table as the CE2421 check, and could not drift from it. +`mxcli context` today gives relationships but no definitions, so it is the obvious command to extend. + +**2. Deciding what a step needs: stays in the toolkit.** These parts carry project and process +knowledge that mxcli should not own: + +- the brief's `### Build steps` row (what to build, what it reads, which element to copy); +- the folder plan; +- the dispatch wiring (`iterative-build-loop.md`, the `mdl-agent` stub); +- this bench. + +**Split.** The toolkit decides *which* names go in. mxcli turns names into one context document. Then +`context-pack.sh` shrinks to about 30 lines: read the brief row, call mxcli once, add the folder plan. +Until mxcli has the command, the script stays as it is. The bench above is the acceptance test for +the CLI version: same scores, same or fewer tool calls. + +**Upstream asks (drafted, for the maintainer to file):** + +- `mxcli check` should flag CE2421 (TEXTBOX on a non-string attribute) and CE1571. +- `mxcli check` should validate enumeration values. +- A "context for a set of names" command, or `mxcli context` extended with definitions. diff --git a/project-bin/context-pack.sh b/project-bin/context-pack.sh new file mode 100755 index 0000000..a813594 --- /dev/null +++ b/project-bin/context-pack.sh @@ -0,0 +1,236 @@ +#!/usr/bin/env bash +# context-pack.sh — one file with everything a helper needs for ONE build step. +# +# WHAT THIS IS. A dispatched helper (mdl-agent) used to assemble its own context: read the +# brief, grep the BRD, DESCRIBE the entities one call at a time, hunt for an example, guess the +# folder. Every one of those reads is a round trip that re-sends the whole conversation, and +# the guesses are where folder drift and wrong association names came from. This script does +# the assembly once, mechanically, from three sources that each own one kind of fact: +# +# the module brief's "Build steps" row WHAT this step builds, reads, copies (the plan) +# mxcli brain brief WHY — decisions and the slice's requirements +# mxcli DESCRIBE, run now WHAT IS — the model as it stands, never a copy +# +# Nothing in the pack is written down twice: the brief names elements, the model describes +# them, the brain explains them. A pack is regenerated per dispatch and never committed. +# +# PRODUCER FOR EVERY CONSUMER. Reads the "### Build steps" table in +# architecture/modules//module-brief.md (or the flat -brief.md), which architect-agent writes at Stage 4 per +# skills/module-brief.md (Ready-check: "Build steps covers every document to be built"). Reads +# docs/brain/ if `mxcli brain init` has run (iterative-build-loop.md, build start); without it +# the pack says so and carries on — the brain section is the only optional one. +# +# Read-only. Runs `mxcli -c "DESCRIBE ..."` and `mxcli brain brief`, never exec. +# +# Usage: +# bin/context-pack.sh # pack to stdout, size line to stderr +# bin/context-pack.sh --out FILE +# bin/context-pack.sh --brief PATH # brief somewhere else +# +# Exit: 0 pack complete · 1 pack written, but an element it names is not in the model +# (listed under "Not found" — a typo in the brief, or a Reads element not built yet) +# · 2 instrument fault (no brief, no row for , no mxcli, no .mpr) +# +# Bash 3.2 compatible: no mapfile, no associative arrays. + +. "$(dirname "$0")/_common.sh" + +MODULE="" STEP="" OUT="" BRIEF="" +while [ $# -gt 0 ]; do + case "$1" in + --out) OUT="$2"; shift 2 ;; + --brief) BRIEF="$2"; shift 2 ;; + -h|--help) sed -n '2,32p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;; + -*) echo "context-pack: unknown flag $1" >&2; exit 2 ;; + *) if [ -z "$MODULE" ]; then MODULE="$1"; elif [ -z "$STEP" ]; then STEP="$1"; else echo "context-pack: extra argument $1" >&2; exit 2; fi; shift ;; + esac +done +[ -n "$MODULE" ] && [ -n "$STEP" ] || { echo "usage: context-pack.sh [--out FILE] [--brief PATH]" >&2; exit 2; } + +# Both brief layouts are in use: /module-brief.md (iterative-build-loop.md) and the flat +# -brief.md. The folder form wins when both exist. +if [ -z "$BRIEF" ]; then + BRIEF="$PROJECT_ROOT/architecture/modules/$MODULE/module-brief.md" + [ -f "$BRIEF" ] || BRIEF="$PROJECT_ROOT/architecture/modules/$MODULE-brief.md" +fi +[ -f "$BRIEF" ] || { echo "context-pack: no brief at architecture/modules/$MODULE/module-brief.md or $MODULE-brief.md" >&2; exit 2; } +MPR="$(find_mpr)" || exit 2 +MODEL_DIR="$(cd "$(dirname "$MPR")" && pwd)" +MPR_BASE="$(basename "$MPR")" + +MXCLI="${MXCLI:-}" +if [ -z "$MXCLI" ]; then + if [ -x "$MODEL_DIR/mxcli" ]; then MXCLI="$MODEL_DIR/mxcli" + elif [ -x "$PROJECT_ROOT/mxcli" ]; then MXCLI="$PROJECT_ROOT/mxcli" + elif command -v mxcli >/dev/null 2>&1; then MXCLI="mxcli" + else echo "context-pack: mxcli not found (./mxcli or on PATH); set MXCLI=/path/to/mxcli" >&2; exit 2; fi +fi + +# --- the brief: one table row, two sections ------------------------------------------------- +# section — the body under a "### ..." heading, up to the next +# heading. Prefix match, because briefs add a parenthetical after the name. +section() { + awk -v h="### $1" 'index($0,h)==1 {on=1; next} on && /^##/ {exit} on {print}' "$BRIEF" +} +# cells of the Build steps row whose first cell is $STEP: "builds|reads|example|slice" +ROW="$(section "Build steps" | awk -F'|' -v s="$STEP" ' + function t(x) { gsub(/`/,"",x); gsub(/^[ \t]+|[ \t]+$/,"",x); return x } + NF>=6 && t($2)==s { print t($3) "|" t($4) "|" t($5) "|" t($6); exit }')" +[ -n "$ROW" ] || { echo "context-pack: no row '$STEP' in the '### Build steps' table of $BRIEF" >&2; exit 2; } +BUILDS="$(printf '%s' "$ROW" | cut -d'|' -f1)" +READS="$(printf '%s' "$ROW" | cut -d'|' -f2)" +EXAMPLE="$(printf '%s' "$ROW" | cut -d'|' -f3)" +SLICE="$(printf '%s' "$ROW" | cut -d'|' -f4)" +case "$SLICE" in -|—|none) SLICE="" ;; esac + +# a cell is a comma-separated list of qualified names; "—" or "-" means none +names() { printf '%s\n' "$1" | tr ',' '\n' | sed 's/^[ \t]*//; s/[ \t]*$//' | grep -vE '^(-|—|none)?$'; } + +# folder for a document: its row in the "Document folder plan" table, by full or short name +folder_of() { + section "Document folder plan" | awk -F'|' -v full="$1" ' + function t(x) { gsub(/`/,"",x); gsub(/^[ \t]+|[ \t]+$/,"",x); return x } + BEGIN { short=full; sub(/^[^.]*\./,"",short) } + NF>=4 { d=t($2); sub(/ *\(.*$/,"",d); if (d==full || d==short) { print t($3); exit } }' +} + +# --- the model ------------------------------------------------------------------------------ +MISSING="" +# describe — try each document kind; mxcli says "not found" for the wrong one +describe() { + local kind out + for kind in ENTITY ASSOCIATION ENUMERATION MICROFLOW NANOFLOW PAGE SNIPPET CONSTANT "JAVA ACTION" WORKFLOW; do + if out="$(cd "$MODEL_DIR" && "$MXCLI" -p "$MPR_BASE" -c "DESCRIBE $kind $1" 2>/dev/null)"; then + printf '### %s (%s)\n```\n%s\n```\n\n' "$1" "$(printf '%s' "$kind" | tr 'A-Z' 'a-z')" "$out" + return 0 + fi + done + return 1 +} + +pack() { + local n f out + printf '# Context pack — %s step %s\n\n' "$MODULE" "$STEP" + printf 'Generated %s from `%s`. Regenerate, never edit: the model sections are live.\n\n' \ + "$(date '+%Y-%m-%d %H:%M')" "${BRIEF#$PROJECT_ROOT/}" + + printf '## This step builds\n\n| Document | Folder |\n|---|---|\n' + # entities, associations and enumerations live in the domain model, not in a folder; the + # name alone cannot say which kind a not-yet-built element is, so the cell says both + # (field run: an entity step's pack said "escalate" and the helper stopped to ask) + names "$BUILDS" | while IFS= read -r n; do + f="$(folder_of "$n")" + printf '| %s | %s |\n' "$n" "${f:-not in the folder plan — fine for an entity, association or enumeration (domain model); any other document: escalate, do not invent a folder}" + done + printf '\n' + + # the rules a helper must apply, by step kind. Field run 2026-10-01: helpers given a pack + # skipped the skill reads the pack did not mention, and one reported the preflight skill + # "not available" although it was installed. A citation is not a read: name the files here. + local kinds="" tk + for n in $(names "$BUILDS"); do + case "$(folder_of "$n")" in + *Pages*|*Snippets*) kinds="$kinds page" ;; + "") kinds="$kinds domain" ;; + *) kinds="$kinds logic" ;; + esac + done + tk="$(find_toolkit_root 2>/dev/null || true)"; [ -n "$tk" ] && tk="$tk/" + # mxcli's bundled skills: flat .md in older versions, /SKILL.md in newer ones + mxskill() { + if [ -f "$MODEL_DIR/.ai-context/skills/$1/SKILL.md" ]; then printf '.ai-context/skills/%s/SKILL.md' "$1" + else printf '.ai-context/skills/%s.md' "$1"; fi + } + printf '## Rules — read these before drafting (the pack does not replace them)\n\n' + printf -- '- `%sskills/learned-mdl-preflight.md` — STOP table; check every planned operation\n' "$tk" + case "$kinds" in *logic*) printf -- '- `%sskills/learned-microflow-patterns.md` — microflow patterns\n' "$tk" ;; esac + case "$kinds" in *page*) + printf -- '- `%sskills/ui-preflight-pages.md` — page pre-flight (wireframe, tokens, StyleGallery)\n' "$tk" + printf -- '- `%s` — widget syntax\n' "$(mxskill create-page)" ;; + esac + case "$kinds" in *domain*) printf -- '- `%s` — entity, association, enumeration syntax\n' "$(mxskill generate-domain-model)" ;; esac + printf '\n' + + printf '## Why (mxcli brain brief)\n\n' + # the brain's own headings start at "#"; push them two levels down so they nest under this one + if [ -d "$MODEL_DIR/docs/brain" ]; then + if [ -n "$SLICE" ]; then + (set -o pipefail; cd "$MODEL_DIR" && "$MXCLI" brain brief --slice "$SLICE" -p "$MPR_BASE" 2>/dev/null | sed -E 's/^(#+) /##\1 /') || printf '_brain brief --slice %s failed_\n' "$SLICE" + else + (set -o pipefail; cd "$MODEL_DIR" && "$MXCLI" brain brief --module "$MODULE" -p "$MPR_BASE" 2>/dev/null | sed -E 's/^(#+) /##\1 /') || printf '_brain brief --module %s failed_\n' "$MODULE" + fi + else + printf '_No docs/brain/ yet — `mxcli brain init` at build start (iterative-build-loop.md)._\n' + fi + printf '\n' + + n="$(section "Arch constraints")" + [ -n "$n" ] && printf '## Arch constraints (module brief)\n%s\n\n' "$n" + + printf '## Model now — what this step reads (live DESCRIBE)\n\n' + local watch="" ui="" a + for n in $(names "$READS"); do + if out="$(describe "$n")"; then + printf '%s\n\n' "$out" + case "$out" in "### $n (entity)"*) + a="$(printf '%s\n' "$out" | sed -nE 's/^[[:space:]]+"?([A-Za-z0-9_]+)"?: (DateTime|Boolean|Enumeration)([^A-Za-z].*)?$/\1/p' | paste -s -d, - | sed 's/,/, /g')" + [ -n "$a" ] && watch="$watch- $n: $a +" ;; + esac + else MISSING="$MISSING $n"; fi + done + + # a page step: its folder-plan folder holds pages, or the thing it changes / copies is a page + for n in $(names "$BUILDS"); do + case "$(folder_of "$n")" in *Pages*|*Snippets*) ui=1 ;; esac + done + + if [ -n "$(names "$BUILDS")" ]; then + for n in $(names "$BUILDS"); do + if out="$(describe "$n")"; then + printf '## Already in the model — you are changing this, not creating it\n\n%s\n' "$out" + # who depends on it: keep their contract (field run: the change helper looked this up by hand) + a="$(cd "$MODEL_DIR" && "$MXCLI" impact -p "$MPR_BASE" "$n" 2>/dev/null | sed -n -e '/^| SourceType/,$p' -e '/^(no impact/p')" + [ -n "$a" ] && printf '### Depends on %s (mxcli impact) — keep their contract\n\n%s\n\n' "$n" "$a" + case "$out" in "### $n (page)"*|"### $n (snippet)"*) ui=1 ;; esac + fi + done + fi + + if [ -n "$(names "$EXAMPLE")" ]; then + printf '## Example to copy — the app'"'"'s own way of doing this\n\n' + for n in $(names "$EXAMPLE"); do + if out="$(describe "$n")"; then + printf '%s\n\n' "$out" + case "$out" in "### $n (page)"*|"### $n (snippet)"*) ui=1 ;; esac + else MISSING="$MISSING $n"; fi + done + fi + + # field run 2026-10-01: 2 of 2 page helpers bound a TEXTBOX to DateTime/enum attributes they + # had in the pack above; `mxcli check` passed, mx check failed with CE2421. Name them up front. + if [ -n "$ui" ] && [ -n "$watch" ]; then + printf '## Widget watch-out — not TEXTBOX-bindable\n\n' + printf 'TEXTBOX takes String and number attributes only. These fail mx check with CE2421 (`mxcli check` passes them): read-only → DYNAMICTEXT with ContentParams; editable → DATEPICKER, CHECKBOX, or RADIOBUTTONS for an enum.\n\n%s\n' "$watch" + fi + + if [ -n "$MISSING" ]; then + printf '## Not found in the model\n\n' + for n in $MISSING; do printf -- '- %s — typo in the brief, or not built yet. Ask; do not guess its shape.\n' "$n"; done + printf '\n' + fi + # the MISSING list is built in this function's shell; hand it to the caller via a file + printf '%s' "$MISSING" > "$TMP_MISSING" +} + +TMP_MISSING="$(mktemp "${TMPDIR:-/tmp}/ctxpack.XXXXXX")" || exit 2 +trap 'rm -f "$TMP_MISSING"' EXIT +if [ -n "$OUT" ]; then + mkdir -p "$(dirname "$OUT")" || exit 2 + pack > "$OUT" || exit 2 + echo "context-pack: $(wc -l < "$OUT" | tr -d ' ') lines, $(wc -c < "$OUT" | tr -d ' ') bytes -> $OUT" >&2 +else + pack +fi +[ -s "$TMP_MISSING" ] && { echo "context-pack: not found in the model:$(cat "$TMP_MISSING")" >&2; exit 1; } +exit 0 diff --git a/skills/iterative-build-loop.md b/skills/iterative-build-loop.md index d5c4e47..6e8b501 100644 --- a/skills/iterative-build-loop.md +++ b/skills/iterative-build-loop.md @@ -134,6 +134,8 @@ still had the old (wrong) description and two already-resolved open questions st Run this before scripting each module: - [ ] **Module brief exists and passes its ready-check.** `architecture/modules//module-brief.md` must exist (authored by `ba-agent` translation mode, per `module-brief.md`) with every ready-check box ticked: every screen has a wireframe, the access table covers every element, no open business question blocks this phase, write mode chosen for every STOP-row element. **No brief, or an unchecked ready-check item touching this phase → STOP.** Produce/complete the brief first — do not let the `mdl-agent` synthesize the module from raw BRDs. This is the just-in-time gate. `gate-check.sh` cannot require all briefs at Stage 4 (they don't all exist yet), so the ordering guarantee lives where the build actually gets its orders: every phase opens with a **`BRIEF` row** (`brd-to-build-plan.md` Step 5) — a check-then-create carrying its own `State` cell: does the brief exist and cover this phase's rows, and if not, write or extend it. `project-bin/exec.sh` **warns** if a script writes to a module with no brief — satisfied by either `architecture/modules//module-brief.md` or a `## Module brief — ` heading in the build plan (the single-module merged form) — but it does not refuse, and says nothing at all in a project with no build plan. À-la-carte use of this toolkit is a supported choice; the warning is a signal that a planned row was skipped, not a gate. +- [ ] **The project brain is on.** `docs/brain/` exists beside the `.mpr` (`mxcli brain init`, once per app). The module's requirements are captured into their slice with FORWARD anchors at what will be built (`mxcli brain capture "" --slice -a @Module.Element`, then `promote`), so `mxcli brain plan` reports BUILT/PLANNED from the model — progress nobody self-reports. A *why* learned while building (pattern chosen, trap hit) is captured with a BACKWARD anchor; a person promotes it (`close-the-loop.md`). +- [ ] **Each dispatch gets a pack.** Before handing a step to `mdl-agent`: `bin/context-pack.sh --out .mxcli/packs/-.md`, and give the helper that path instead of a reading list. Exit 1 means a name in the brief's Build steps row is not in the model — fix the brief before dispatching. - [ ] Read source screenshots for this module top-to-bottom - [ ] Read the feature doc (F-doc or BRD) for this module - [ ] Extract the build checklist from the feature doc: diff --git a/skills/module-brief.md b/skills/module-brief.md index 5bbe0a5..c558a16 100644 --- a/skills/module-brief.md +++ b/skills/module-brief.md @@ -251,6 +251,24 @@ rather than wondering whether they missed a file. the `folder` property and never invents one. A document with no row and no derivable feature (module-folder-convention.md, "Deciding the path") is escalated, NOT swept into Common/. --> +### Build steps (one row per dispatch — `bin/context-pack.sh ` turns a row into the helper's pack) +| Step | Builds | Reads | Example | Slice | +|------|--------|-------|---------|-------| +| e.g. 5.1 | Sales.ACT_Order_ApproveAll | Sales.Order, Sales.Order_Customer, Sales.ENUM_OrderStatus | Sales.ACT_Order_Approve | 02-approvals | + + ### Cross-module dependencies & integrations - Depends on: · Integrations: @@ -264,6 +282,8 @@ rather than wondering whether they missed a file. - [ ] No open business question blocks the elements in this build phase - [ ] Write mode chosen for every element that hits a learned-mdl-preflight STOP row - [ ] Folder plan names the module's feature groups and covers every document to be built +- [ ] Build steps has a row for every document to be built in this phase, and + `bin/context-pack.sh ` exits 0 for each (exit 1 = a Reads name is wrong) - [ ] Test plan complete: shape declared, base set named and covering the coverage checklist, at least one journey with its data effect - [ ] Build skills to read first is filled in — or explicitly says "none detected" — for every build group this module touches (integration, pages, mdl, workflow, agents), not just @@ -284,6 +304,9 @@ never covered) is fed back here, not silently fixed in the model with the brief 1. **Step 1 of every build task:** read the module brief. It replaces "read the task spec and hunt for context" — the brief *is* the context, pre-synthesized. + **Dispatching a helper?** Run `bin/context-pack.sh --out + .mxcli/packs/-.md` and hand it the path: the step's Build steps row, the brain + brief, live DESCRIBEs of every Reads element, the example and the folder, in one file. 2. **Gap escalation, not guessing:** if a business rule the agent needs is missing or an open question is unresolved, the agent surfaces the specific question to `ba-agent` (which updates the brief) — it never fills the gap from training data. This is why the brief has an explicit "open