feat: accept x- extension keys in native documents and run the OMG PSSM 1.0 test suite

.github/workflows/python-package.yml

@@ -12,6 +12,10 @@
 permissions:
   contents: read
 
+env:
+  # Must match the exclude-newer span recorded in uv.lock, or `uv sync --locked` fails.
+  UV_EXCLUDE_NEWER: "7 days"
+
 jobs:
   build:
     runs-on: ubuntu-latest
@@ -27,14 +31,14 @@
       with:
         python-version: ${{ matrix.python-version }}
     - name: Setup Graphviz
-      uses: ts-graphviz/setup-graphviz@v2
+      uses: ts-graphviz/setup-graphviz@b1de5da23ed0a6d14e0aeee8ed52fdd87af2363c # v2
     - name: Install uv
-      uses: astral-sh/setup-uv@v10.2.0
+      uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
       with:
         enable-cache: true
         cache-suffix: "python${{ matrix.python-version }}"
     - name: Install the project
-      run: uv sync --all-extras --dev
+      run: uv sync --locked --all-extras --dev
       #----------------------------------------------
       #              run ruff
       #----------------------------------------------
@@ -54,7 +58,7 @@
       #          upload coverage
       #----------------------------------------------
     - name: Upload coverage to Codecov
-      uses: codecov/codecov-action@v7
+      uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7
       if: matrix.python-version == 3.15
       with:
         token: ${{ secrets.CODECOV_TOKEN }} # not required for public repos
@@ -64,3 +68,32 @@
         flags: unittests
         name: codecov-umbrella
         verbose: true
+
+  pssm-drift:
+    # The XMI may not be stored or cached anywhere, see tests/uml/pssm/README.md.
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+    - uses: actions/checkout@v7
+    - name: Set up Python
+      uses: actions/setup-python@v7
+      with:
+        python-version: "3.15"
+    - name: Install uv
+      uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
+      with:
+        enable-cache: true
+        cache-suffix: "pssm-drift"
+    - name: Install the project
+      run: uv sync --locked --all-extras --dev
+    - name: Download the PSSM test suite XMI
+      run: |
+        xmi_dir="$(mktemp -d)"
+        curl --proto "=https" --fail --silent --show-error --location --retry 3 \
+          --output "$xmi_dir/PSSM_TestSuite.xmi" \
+          https://www.omg.org/spec/PSSM/20181101/PSSM_TestSuite.xmi
+        echo "PSSM_XMI=$xmi_dir/PSSM_TestSuite.xmi" >> "$GITHUB_ENV"
+    - name: Compare the committed cases with the converter output
+      env:
+        PSSM_REQUIRED: "1"
+      run: uv run --locked pytest tests/uml/pssm/test_pssm_drift.py --timeout 300

.github/workflows/release.yml

@@ -3,6 +3,10 @@
     tags: [ 'v?*.*.*' ]
 name: release
 
+env:
+  # Must match the exclude-newer span recorded in uv.lock, or `uv sync --locked` fails.
+  UV_EXCLUDE_NEWER: "7 days"
+
 jobs:
   release-build:
     name: Build release artifacts
@@ -19,15 +23,15 @@
           python-version: '3.15'
 
       - name: Setup Graphviz
-        uses: ts-graphviz/setup-graphviz@v2
+        uses: ts-graphviz/setup-graphviz@b1de5da23ed0a6d14e0aeee8ed52fdd87af2363c # v2
 
       - name: Install uv
-        uses: astral-sh/setup-uv@v10.2.0
+        uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
         with:
           enable-cache: true
 
       - name: Install the project
-        run: uv sync --all-extras --dev
+        run: uv sync --locked --all-extras --dev
 
       - name: Test
         run: |
@@ -63,4 +67,4 @@
           path: dist/
 
       - name: Publish package distributions to PyPI
-        uses: pypa/gh-action-pypi-publish@release/v1
+        uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # release/v1

.gitignore

@@ -86,6 +86,9 @@ tmp/
 # Local specs
 specs/
 
+# Local copy of the OMG PSSM test suite, never committed (see tests/uml/pssm/README.md)
+PSSM_TestSuite.xmi
+
 # Environment files (secrets)
 .env
 .env.*

docs/contributing.md

@@ -163,6 +163,24 @@ Before you submit a pull request, check that it meets these guidelines:
 3. Consider adding yourself to the contributor's list.
 4. The pull request should work for all supported Python versions.
 
+## UML cases based on PSSM
+
+`tests/uml/pssm/` runs cases generated from the test suite of the OMG Precise Semantics of
+UML State Machines (PSSM) 1.0 specification, on both engines. Each case is a native YAML
+statechart plus the stimulus and the traces the suite expects. Cases the library cannot
+pass yet are listed in `tests/uml/pssm/conftest.py`, each with a reason code; since
+`xfail_strict` is on, a change that makes one of them pass must remove it from that list.
+The same file lists in `LOWERED` the cases that pass only because the converter lowers a
+construct the library cannot spell yet; when that construct gets a native spelling, the
+converter stops lowering it and those cases are reclassified.
+
+The cases are generated by `tests/uml/pssm/convert.py` and must not be edited by hand. The
+OMG XMI is never committed: to regenerate the cases, download it, point `PSSM_XMI` at it
+and run the converter, as described in
+[tests/uml/pssm/README.md](https://github.com/fgmacedo/python-statemachine/blob/main/tests/uml/pssm/README.md).
+The `pssm-drift` CI job downloads the XMI and fails if the committed cases differ from the
+converter output.
+
 ## Releasing a New Version
 
 This project follows [trunk based development](https://trunkbaseddevelopment.com/): `main` is

docs/io/formats.md

@@ -6,9 +6,9 @@ the same document in either format compiles to the same state machine. Pick YAML
 readability, or JSON when a tool emits or consumes it. (SCXML, the W3C XML standard, loads
 the same way and is covered at the [end of this page](#scxml).)
 
-A document has an optional top-level envelope (`name`, `description`, `datamodel`) and a
-`states` mapping keyed by state id. Here is the same toggle written both ways, proving the
-result is identical:
+A document has an optional top-level envelope (`name`, `description`, `datamodel` and
+[extension keys](#extension-keys)) and a `states` mapping keyed by state id. Here is the
+same toggle written both ways, proving the result is identical:
 
 ```py
 >>> from statemachine.io import load
@@ -446,6 +446,28 @@ Invoke spawns a child machine that runs concurrently, so the parent reaches `fin
 once the child signals back. Because that resolution is not synchronous, the example above
 is shown for structure rather than as a doctest.
 
+## Extension keys
+
+Top-level keys that start with `x-` are reserved for other tools: the loader and the
+[JSON Schema](json_schema.md) accept them and ignore their content, so a document can carry
+test fixtures, editor hints or provenance alongside the statechart. Any other unknown key is
+still an error, and so is an `x-` key below the document root.
+
+```py
+>>> sc = load(
+...     """
+...     x-review: {owner: ops, ticket: OPS-12}
+...     states:
+...       a: {initial: true}
+...     """,
+...     format="yaml",
+...     validate=True,
+... )
+>>> sorted(sc().configuration_values)
+['a']
+
+```
+
 ## SCXML
 
 [SCXML](https://www.w3.org/TR/scxml/) (State Chart XML) is the W3C standard this library's

docs/releases/3.3.0.md

@@ -0,0 +1,12 @@
+# python-statemachine 3.3.0
+
+*Unreleased*
+
+## What's new in 3.3.0
+
+### Extension keys in native documents
+
+A JSON or YAML document can now carry top-level keys prefixed with `x-`, such as
+`x-review: {owner: ops}`. The loader and the JSON Schema accept them and ignore their
+content, so other tools can keep their data in the same file as the statechart. Other
+unknown keys are still rejected. See [](../io/formats.md#extension-keys).

docs/releases/index.md

@@ -16,6 +16,7 @@ Requires Python 3.10+.
 ```{toctree}
 :maxdepth: 2
 
+3.3.0
 3.2.2
 3.2.1
 3.2.0

statemachine/io/native.py

@@ -46,6 +46,8 @@
 # accepted vocabulary and are asserted equal to the JSON Schema in tests/io/test_validation.py,
 # so the schema and the parser can never silently drift apart.
 _DOCUMENT_KEYS = frozenset({"name", "description", "datamodel", "states"})
+_EXTENSION_PREFIX = "x-"
+"""Document-root keys with this prefix are extensions for other tools; the parser ignores them."""
 _STATE_KEYS = frozenset(
     {
         "initial",
@@ -95,9 +97,17 @@ def _flag(value) -> bool:
     return bool(value)
 
 
-def _check_keys(node: Mapping, allowed: "frozenset[str]", kind: str) -> None:
+def _is_extension_key(key) -> bool:
+    return isinstance(key, str) and key.startswith(_EXTENSION_PREFIX)
+
+
+def _check_keys(
+    node: Mapping, allowed: "frozenset[str]", kind: str, *, extensions: bool = False
+) -> None:
     """Reject keys outside the declared vocabulary, mirroring the schema's strictness."""
-    unknown = sorted(set(node) - allowed)
+    unknown = sorted(
+        key for key in node if key not in allowed and not (extensions and _is_extension_key(key))
+    )
     if unknown:
         raise InvalidDefinition(
             f"Unknown {kind} key(s) {unknown}; allowed keys are {sorted(allowed)}."
@@ -112,7 +122,7 @@ def native_dict_to_definition(
         raise InvalidDefinition(
             f"Statechart definition must be a mapping, got {type(doc).__name__}."
         )
-    _check_keys(doc, _DOCUMENT_KEYS, "document")
+    _check_keys(doc, _DOCUMENT_KEYS, "document", extensions=True)
     states_doc = doc.get("states")
     if not isinstance(states_doc, Mapping) or not states_doc:
         raise InvalidDefinition("Statechart definition must have a non-empty 'states' mapping.")

statemachine/io/schemas/statechart.schema.json

@@ -20,6 +20,9 @@
       "additionalProperties": { "$ref": "#/$defs/state" }
     }
   },
+  "patternProperties": {
+    "^x-": {}
+  },
   "additionalProperties": false,
   "$defs": {
     "flag": {

tests/io/test_native.py

@@ -323,6 +323,22 @@ def test_unknown_document_key(self):
         with pytest.raises(InvalidDefinition, match="Unknown document key"):
             defn({"states": {"a": {"initial": True}}, "bogus": 1})
 
+    def test_extension_keys_are_ignored_at_the_document_root(self):
+        definition = defn({"states": {"a": {"initial": True}}, "x-tool": {"any": [1, 2]}})
+        assert list(definition.states) == ["a"]
+
+    def test_extension_keys_are_ignored_in_inline_invoke_content(self):
+        child = {"states": {"r": {"initial": True}}, "x-tool": 1}
+        defn({"states": {"a": {"initial": True, "invoke": {"content": child}}}})
+
+    def test_extension_prefix_is_case_sensitive(self):
+        with pytest.raises(InvalidDefinition, match="Unknown document key"):
+            defn({"states": {"a": {"initial": True}}, "X-tool": 1})
+
+    def test_extension_keys_are_rejected_below_the_root(self):
+        with pytest.raises(InvalidDefinition, match="Unknown state 'a' key"):
+            defn({"states": {"a": {"initial": True, "x-tool": 1}}})
+
     def test_unknown_state_key(self):
         with pytest.raises(InvalidDefinition, match="Unknown state 'a' key"):
             defn({"states": {"a": {"initial": True, "bogus": 1}}})

tests/io/test_validation.py

@@ -51,11 +51,21 @@ def test_node_vocabulary_matches_parser(self, parser_keys, schema_pointer):
             node = node[key]
         assert set(parser_keys) == set(node)
 
+    def test_extension_prefix_matches_parser(self):
+        assert load_schema()["patternProperties"] == {f"^{native._EXTENSION_PREFIX}": {}}
+
 
 class TestValidateDocument:
     def test_valid(self):
         validate_document({"name": "M", "states": {"a": {"initial": True, "transitions": []}}})
 
+    def test_valid_with_extension_key(self):
+        validate_document({"states": {"a": {"initial": True}}, "x-tool": {"any": [1]}})
+
+    def test_extension_key_below_the_root_is_invalid(self):
+        with pytest.raises(InvalidDefinition):
+            validate_document({"states": {"a": {"initial": True, "x-tool": 1}}})
+
     def test_valid_with_actions(self):
         validate_document(
             {

tests/uml/pssm/README.md

@@ -0,0 +1,81 @@
+# UML state machine cases based on PSSM
+
+The cases in `cases/` are generated by `convert.py` from the test suite of the OMG
+Precise Semantics of UML State Machines (PSSM) 1.0 specification:
+
+- Source: <https://www.omg.org/spec/PSSM/20181101/PSSM_TestSuite.xmi>
+- OMG document number: ptc/18-11-06
+- sha256: `c355b249c356774377a46b60345019d827af1ce417bde88e533aa5f39206ae07`
+
+The PSSM specification and its test suite are copyright © Object Management Group, Inc.
+and the organizations named in the specification's copyright notice. UML and OMG are
+trademarks of the Object Management Group, used here only to name the source.
+
+The repository holds the converter, the harness and the generated cases, never the XMI
+itself: the specification's license does not allow posting copies of it on a network
+computer, so the XMI is downloaded when needed and never committed, stored or cached. The
+cases are *based on* PSSM; passing them is not a conformance claim.
+
+## Layout
+
+- `convert.py`: XMI to native YAML, standard library XML parser, deterministic output.
+- `cases/<Category>/<id>.yaml`: one generated case per PSSM test. Do not edit.
+- `overlays/<id>.yaml`: hand-written patches the converter applies, for behaviors with logic
+  (counters, signals sent from inside the machine) and for CallEvent tests. `cases/` is
+  always exactly the converter output for the XMI plus these overlays. In the generated
+  statechart, an overlay's native actions run before its trace callback.
+- `conftest.py`: the expected failures, per engine, with a reason code, and `LOWERED`: the
+  cases that pass only because the converter lowers a construct the library cannot spell
+  yet (for example, a junction flattened into guarded transitions). Those passes do not
+  show native support for the construct. Every case with a non-empty `requires` is in one
+  of the two.
+- `test_pssm_cases.py`: runs every case on the sync and the async engine.
+- `test_pssm_drift.py`: regenerates the cases and compares them with `cases/`.
+
+## Case format
+
+Each case is a native statechart document: `statemachine.io.load(path)` loads it as is. The
+test lives in the document's `x-pssm` [extension key](../../../docs/io/formats.md#extension-keys),
+which the loader ignores:
+
+- `requires`: UML constructs the case uses that the native format cannot spell yet
+  (`choice`, `junction`, `join`, `fork`, `terminate`, `entry_point`, `exit_point`,
+  `defer`, `call_event`, `redefinition`, `do_activity_accept`). The statechart is the
+  closest loadable approximation.
+- `expected_traces`: the accepted traces, taken from the suite's executable oracle.
+- `stimulus`: what the tester does, in order, starting with `Start`: `{event, kwargs}`
+  sends a signal, `{call, kwargs}` calls an operation, `{trace}` is a token the tester
+  traces itself, `trace_result` formats the value a call returned.
+- `behaviors`: the trace tokens each callback appends; `{name}` placeholders are filled
+  with the event data. `on_invoke_<state>` entries are doActivities; in them,
+  `{send: {event}}` sends an event to the machine and `{accept}` parks the doActivity until
+  its state is left.
+
+The callbacks the statechart names come from the harness listener, which reads
+`behaviors`; loaded without it, the machine runs but traces nothing. The harness loads the
+cases untrusted and validated against the JSON Schema.
+
+The converter lowers UML completion transitions to what the library has today: from a
+composite state to `done.state.<id>`, from a state with a doActivity to
+`done.invoke.<id>`, from a simple state to an eventless transition.
+
+## Running
+
+```bash
+timeout 120 uv run pytest tests/uml/pssm -n 4
+```
+
+## Regenerating
+
+Download the XMI to a git-ignored location (any directory outside the repository, or the
+repository root, where `PSSM_TestSuite.xmi` is ignored), then:
+
+```bash
+export PSSM_XMI=/path/to/PSSM_TestSuite.xmi
+uv run python -m tests.uml.pssm.convert --out tests/uml/pssm/cases
+uv run pytest tests/uml/pssm/test_pssm_drift.py
+```
+
+The converter checks the sha256 above before reading the file. Without `PSSM_XMI` the
+drift test is skipped; `PSSM_REQUIRED=1` turns the skip into a failure, which is how the
+`pssm-drift` CI job runs it.