Skip to content

journey --positive-control proves the harness can fail, not that the journey's own business claim can: add named controls #159

Description

@MendixMau

What happens

project-tests/e2e/journey-runner.js --positive-control mutates the generic rungs (landing, text, trace, data, outcome) and reports "N of M rungs proven". That proves the runner can fail. It does not prove that the claim the journey exists for can fail.

A journey can pass every rung while its point is only true by accident, and every generic mutant is still caught.

Evidence (card-disbursement requirements-driven build, 2026-09-26)

  • One journey's first version passed every rung, and its positive control proved every rung.
  • Its point was "a case waits while an upstream system is offline". That was true only by accident: the case was waiting for a different reason.
  • The project added named-controls.js, and the named control for that journey came back NOT PROVEN. That is how the defect was found.
  • Five journeys, each with a named control: all five PROVEN after the fix.

The shape that worked

  • A control file is a copy of its journey with one claim the app must not satisfy, plus a named check:

    { "control": { "of": "J-01", "mustFail": "<check-name text>" } }
  • PROVEN only when all three hold:

    1. the runner exits 1;
    2. every check whose name contains mustFail is FAIL, and there is at least one;
    3. nothing else is FAIL or INVALID.
  • Rule 3 is the point. A control that goes red somewhere else, such as a login fault or a moved landing, proves nothing about the named claim, and reads NOT PROVEN.

  • The control's persona must equal TEST_USER; a mismatch is a FAULT and the control does not run.

  • Findings go to artifacts/control-<id>.json via the runner's --out=, so they never overwrite the real walk's findings.

Suggested fix

  • Port named-controls.js into project-tests/e2e/. It is about 60 lines; it spawns the runner and reads its findings JSON.
  • skills/journey-proof.md / testing-shape.md §6: every journey names its business claim and ships a control for it.
  • The generic rungs stay, because they prove the harness.
  • Optionally count "journeys with a PROVEN named control" as a denominator in the journeys obligation.

Depends on --out= from #148. Filed rather than ported because it needs a field run on a scratch app with a journey and its control.

Files

  • project-tests/e2e/named-controls.js (new)
  • project-tests/e2e/journey-runner.js (--out=)
  • skills/journey-proof.md
  • skills/testing-shape.md
  • bin/lib/obligations.tsv: optional

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions