Repository navigation
feat: accept x- extension keys in native documents and run the OMG PSSM 1.0 test suite - #697
Conversation
…t suite Add a converter from the PSSM 1.0 test suite XMI (ptc/18-11-06) to native YAML cases, the 103 generated cases, the overlays for behaviors with logic and CallEvent tests, and a harness that runs every case on the sync and async engines. Cases the library cannot pass yet are expected failures with a reason code. A drift test regenerates the cases when PSSM_XMI points to the XMI, and a separate CI job downloads it to run that test. Refs #692 Signed-off-by: Fernando Macedo <fgmacedo@gmail.com>
| - name: Compare the committed cases with the converter output | ||
| env: | ||
| PSSM_REQUIRED: "1" | ||
| run: uv run pytest tests/uml/pssm/test_pssm_drift.py --timeout 300 |
There was a problem hiding this comment.
Not applied yet: PyYAML 6.0.3 publishes no cp315 wheels, so --no-build fails on the 3.15 jobs. Tracked in #684, which adds the flag once the wheels land.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #697 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 53 53
Lines 5897 5901 +4
Branches 940 940
=========================================
+ Hits 5897 5901 +4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
… force HTTPS on the XMI download Signed-off-by: Fernando Macedo <fgmacedo@gmail.com>
…ed accepts it Signed-off-by: Fernando Macedo <fgmacedo@gmail.com>
| cache-suffix: "python${{ matrix.python-version }}" | ||
| - name: Install the project | ||
| run: uv sync --all-extras --dev | ||
| run: uv sync --locked --all-extras --dev |
There was a problem hiding this comment.
Not applied yet: PyYAML 6.0.3 publishes no cp315 wheels, so --no-build fails on the 3.15 jobs. Tracked in #684.
| enable-cache: true | ||
| cache-suffix: "pssm-drift" | ||
| - name: Install the project | ||
| run: uv sync --locked --all-extras --dev |
There was a problem hiding this comment.
Not applied yet: PyYAML 6.0.3 publishes no cp315 wheels, so --no-build fails on the 3.15 jobs. Tracked in #684.
|
|
||
| - name: Install the project | ||
| run: uv sync --all-extras --dev | ||
| run: uv sync --locked --all-extras --dev |
There was a problem hiding this comment.
Not applied yet: PyYAML 6.0.3 publishes no cp315 wheels, so --no-build fails on the 3.15 jobs. Tracked in #684.
Signed-off-by: Fernando Macedo <fgmacedo@gmail.com>
Signed-off-by: Fernando Macedo <fgmacedo@gmail.com>
…-pssm Signed-off-by: Fernando Macedo <fgmacedo@gmail.com>
|




Why
The UML work tracked in #692 needs a machine-checked reference: the OMG PSSM 1.0 test suite has 103 tests with an executable oracle in its XMI (ptc/18-11-06), and none of them run against the library today. This PR runs all 103 on both engines and records each divergence with a reason code. Refs #692.
What changes
tests/uml/pssm/convert.pyturns the PSSM XMI into one native YAML case per test (cases/<Category>/<id>.yaml, 103 files, header "Do not edit"). It checks the sha256, is deterministic, and reads compiled activities only (Alf is read only for guard bodies). Each case is a native statechart thatstatemachine.io.load(path)loads as is; the test (stimulus, behaviors, expected traces,requires) lives under thex-pssmextension key. Constructs without a native spelling are listed inrequires.x-extension keys: the parser and the JSON Schema ignore their content, any other unknown key is still rejected, and anx-key below the root too. Documented indocs/io/formats.mdand in new 3.3.0 release notes.trace()behavior becomes a bare callback reference. A harness listener appends the token, and event data reaches it by dependency injection. doActivities use theon_invoke_<state>convention; a doActivity that waits for an event parks until its state is left.test_pssm_cases.pyruns every case on the sync and async engines (206 runs). Any error fails the case loudly, including errors the async engine would otherwise drop after a doActivity completes.conftest.pylists 44 expected failures with codes (requires:<construct>,conflict:K2/K10/K13/K14,bug:B1/B2, oneunclassified). It also lists the 25 cases that pass only through the converter's lowering inLOWEREDwithlowered:<construct>reasons; those passes do not show native support. A test enforces that every case with a non-emptyrequiresis classified.test_pssm_drift.pyregenerates the cases whenPSSM_XMIis set and compares them byte for byte. The newpssm-driftCI job downloads the XMI with curl into a temp dir and runs that test withPSSM_REQUIRED=1; the XMI is never cached or committed.Results, identical on both engines:
Evidence
Under
statemachine/, only thex-extension keys change (parser and schema). ruff, ruff format and mypy (pre-commit flags) are clean.Adversarial review: four rounds, each by a fresh agent with no shared context.
acceptno longer completes early (it parks). Async errors are caught. Reasons now list every blocker. A cleanup replaced document-order-changing YAML round-tripping with JSON, which turned two supposedbug:B5cases into passes.strict=Falseon the lowered cases hid regressions. The coordinator decided to unmark them and list them inLOWERED.Not verified
pssm-driftCI job has not run on GitHub Actions yet; it was emulated locally with the same curl command and env.unclassified(History_002-B): completion transitions in two orthogonal regions, lowered to eventless, run in one microstep, while PSSM runs each completion in its own step. Not in the plan's K table.InvokeManager._run_async_handlerdrops an exception raised while processingdone.invokewhen that same step exits the doActivity's state (ctx.cancelledis already set). The harness works around it; no PSSM case hits it today.bug:B5is not reproduced by any PSSM case after the key-order fix.tests/uml/pssm/README.mdis generic (OMG plus the organizations named in the specification), written without quoting the PDF; the maintainer should check the wording.target, while the parser accepts space-separated targets; the converter uses the latter for forks and entry points with several regions.sm._engine._invoke_managerstate to know when doActivities have settled.