test: formalize real PLC acceptance testing

.github/ISSUE_TEMPLATE/plc-test-result.yml

@@ -0,0 +1,78 @@
+name: Real PLC test result
+description: Record one reproducible python-snap7 acceptance run against a real PLC
+title: "[hardware test]: "
+labels: ["hardware-test"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Attach the generated JSON and JUnit XML. Review all attachments before uploading. Never include PLC addresses, credentials, private keys, plant names, or unreviewed packet captures.
+  - type: input
+    id: source
+    attributes:
+      label: Source revision
+      description: Immutable 40-character commit SHA and, if applicable, release version.
+      placeholder: abcdef0123456789abcdef0123456789abcdef01
+    validations:
+      required: true
+  - type: dropdown
+    id: result
+    attributes:
+      label: Overall result
+      options: [pass, fail, partial]
+    validations:
+      required: true
+  - type: dropdown
+    id: protocol
+    attributes:
+      label: Protocol path
+      options: [legacy-s7, s7commplus]
+    validations:
+      required: true
+  - type: input
+    id: plc
+    attributes:
+      label: PLC family, model, and order code
+      placeholder: S7-1500, CPU 1511-1 PN, 6ES7 ...
+    validations:
+      required: true
+  - type: input
+    id: firmware
+    attributes:
+      label: Firmware and security mode
+      placeholder: V2.9, PUT/GET enabled, non-optimized fixture DBs
+    validations:
+      required: true
+  - type: input
+    id: host
+    attributes:
+      label: Host environment
+      placeholder: Python 3.13, Linux x86_64
+    validations:
+      required: true
+  - type: input
+    id: started
+    attributes:
+      label: Run time (UTC)
+      placeholder: 2026-09-11T12:34:56Z
+    validations:
+      required: true
+  - type: textarea
+    id: artifacts
+    attributes:
+      label: Sanitized report artifacts
+      description: Drag the generated JSON and JUnit XML here. State any manually reviewed capture separately.
+    validations:
+      required: true
+  - type: textarea
+    id: skips
+    attributes:
+      label: Skips, failures, and notes
+      description: Include machine-readable skip reasons and whether this result supersedes an earlier issue.
+  - type: checkboxes
+    id: safety
+    attributes:
+      label: Safety confirmation
+      options:
+        - label: I reviewed the attachments and they contain no PLC address, credentials, private keys, plant/site identifiers, or proprietary PLC program.
+          required: true

CHANGES.md

@@ -6,6 +6,9 @@ CHANGES
 
 Major release: new `s7commplus` package with S7CommPlus protocol support.
 
+* Add a safe, Gherkin-based real-PLC acceptance program with restored scratch
+  writes and sanitized JSON/JUnit reports (#840).
+
 * Decode corroborating CPU execution attributes so S7CommPlus `get_cpu_state()`
   distinguishes RUN from STOP on S7-1500 and returns UNKNOWN for absent or
   inconsistent state attributes, including S7-1200 responses that omit them.

README.rst

@@ -27,6 +27,8 @@ As of version 3.0, the C library is no longer used, but the name is kept for
 backwards compatibility.
 
 The full documentation is available on `Read The Docs <https://python-snap7.readthedocs.io/en/latest/>`_.
+The safe volunteer hardware-test procedure is in
+`Real-PLC acceptance testing <doc/real-plc-testing.rst>`_.
 
 
 Quick Start

doc/index.rst

@@ -34,6 +34,7 @@ Welcome to python-snap7's documentation!
    :caption: Development
 
    development
+   real-plc-testing
 
 .. toctree::
    :maxdepth: 2

doc/real-plc-testing.rst

@@ -0,0 +1,90 @@
+Real-PLC acceptance testing
+===========================
+
+The versioned Gherkin specifications in ``tests/features/real_plc`` exercise the
+same client APIs and canonical 37-byte fixture used by the diagnostic end-to-end
+tests. ``pytest-bdd`` was selected because it keeps pytest as the runner, reuses
+fixtures, maps tags to markers, and supports the existing JUnit workflow.
+
+Safety model
+------------
+
+Use only a dedicated, non-production test PLC. The default runner selects
+``@smoke`` and is read-only. It never enables writes or administrative actions.
+Scratch writes require ``--allow-write``, a separate DB (DB2 by default), and are
+wrapped in a guard that saves, restores, and reads back the original bytes from
+fixture teardown even when a step fails or pytest handles an interruption.
+Hard process termination or power loss cannot run cleanup, so inspect the scratch
+DB before reuse after either event.
+
+Administrative actions (CPU start/stop, clock changes, program transfers, or
+protection changes) require ``--allow-admin`` and must never target an in-service
+PLC. No administrative scenarios are included in the initial suite.
+
+PLC preparation
+---------------
+
+Import ``tests/plc_setup/e2e_test_dbs.scl`` into TIA Portal, or recreate its exact
+layout. It defines read-only DB1 and scratch DB2. On S7-1200/1500, disable
+``Optimized block access`` for both fixture DBs. Enable PUT/GET only when the
+chosen legacy protocol path and the PLC security policy require it. Defaults are
+rack 0, slot 1, and TCP port 102. Apply the least privilege that permits the
+selected scenarios; do not weaken a PLC that is in service.
+
+Running and reporting
+---------------------
+
+Install the test extras, then run the safe profile. Values supplied to reportable
+metadata must not contain addresses, credentials, certificate paths, plant names,
+or other site identifiers::
+
+  uv sync --extra test --extra s7commplus
+  uv run python tools/run_real_plc_acceptance.py \
+    --plc-ip YOUR_PRIVATE_ADDRESS \
+    --protocol legacy_s7 \
+    --tester @YOUR_HANDLE \
+    --plc-family S7-1500 \
+    --plc-model "CPU 1511-1 PN" \
+    --plc-firmware V2.9 \
+    --plc-security-mode "PUT/GET, non-optimized fixture DBs"
+
+The runner prints a Gherkin-aware terminal summary and writes JUnit XML plus a
+schema-versioned, sanitized JSON report beneath ``real-plc-results/``. The JSON
+records exact source state, host environment, PLC metadata, scenario tags,
+pass/fail/skip results, and bounded diagnostics. It deliberately never reads the
+PLC address into the report. Review both artifacts before publishing them.
+
+Add ``--allow-write`` only after confirming DB2 is disposable scratch space.
+``--allow-admin`` remains separate so write permission never implies permission
+for operational actions.
+
+Result policy
+-------------
+
+File one ``Real PLC test result`` issue per tester, PLC configuration, source
+revision, and run. Attach both generated artifacts and apply exactly one result
+label and one protocol label:
+
+* ``hardware-test``
+* ``test-result: pass``, ``test-result: fail``, or ``test-result: partial``
+* ``protocol: legacy-s7`` or ``protocol: s7commplus``
+
+``pass`` means every selected applicable scenario passed. ``fail`` means at least
+one selected scenario failed. ``partial`` means the run completed with capability
+skips or an incomplete selection; unsupported capabilities are not failures.
+Reruns get a new issue and link the earlier result. Close the earlier issue as
+superseded only after the replacement artifacts exist. A result becomes stale
+when the tested code, feature schema, relevant protocol implementation, PLC
+firmware, or PLC configuration changes—not merely with age.
+
+Useful searches:
+
+* `Open hardware failures <https://github.com/gijzelaerr/python-snap7/issues?q=is%3Aissue+is%3Aopen+label%3Ahardware-test+label%3A%22test-result%3A+fail%22>`_
+* `S7CommPlus hardware runs <https://github.com/gijzelaerr/python-snap7/issues?q=is%3Aissue+label%3Ahardware-test+label%3A%22protocol%3A+s7commplus%22>`_
+* Search a release or commit by adding its version or SHA to ``is:issue label:hardware-test``.
+
+For a release candidate, prioritize one supported legacy S7 PLC and one
+S7CommPlus-capable S7-1200/1500 before expanding the host matrix; hosted CI already
+covers supported Python versions and operating systems without hardware. Test
+evidence stays in issues. Volunteer coordination may use Discussions, but is not
+a substitute for an attached structured result.

pyproject.toml

@@ -32,7 +32,7 @@ Homepage = "https://github.com/gijzelaerr/python-snap7"
 Documentation = "https://python-snap7.readthedocs.io/en/latest/"
 
 [project.optional-dependencies]
-test = ["pytest", "pytest-asyncio", "pytest-cov", "pytest-html", "hypothesis", "mypy", "types-setuptools", "ruff", "tox", "tox-uv", "types-click", "uv"]
+test = ["pytest", "pytest-asyncio", "pytest-bdd>=8.1,<9", "pytest-cov", "pytest-html", "hypothesis", "mypy", "types-setuptools", "ruff", "tox", "tox-uv", "types-click", "uv"]
 s7commplus = ["cryptography"]
 cli = ["rich", "click" ]
 demo = ["psutil", "rich", "click"]
@@ -56,6 +56,13 @@ markers =[
     "client",
     "common",
     "e2e: end-to-end tests requiring a real PLC connection",
+    "real_plc: versioned real-PLC acceptance scenario",
+    "smoke: read-only real-PLC acceptance scenario",
+    "write: real-PLC scenario that changes and restores scratch data",
+    "plc_write: legacy alias for real-PLC tests that modify scratch data",
+    "administrative: disruptive real-PLC scenario requiring separate opt-in",
+    "legacy_s7: scenario using the legacy S7 protocol path",
+    "s7commplus: scenario using the S7CommPlus protocol path",
     "hypothesis: property-based tests using Hypothesis",
     "logo",
     "mainloop",

tests/conftest.py

@@ -2,9 +2,15 @@
 
 import socket
 import sys
+from pathlib import Path
+from typing import Any
 
 import pytest
 
+from tests.real_plc.reporting import RealPLCReport, report_metadata
+
+_REAL_PLC_REPORT = RealPLCReport()
+
 
 def get_free_tcp_port() -> int:
     """Return a TCP port that is free *right now* on 127.0.0.1.
@@ -76,6 +82,27 @@ def pytest_addoption(parser: pytest.Parser) -> None:
         default=2,
         help="Read-write DB number for e2e tests (default: 2)",
     )
+    parser.addoption(
+        "--allow-plc-write",
+        action="store_true",
+        default=False,
+        help="Allow tests that modify and restore the dedicated scratch DB",
+    )
+    parser.addoption(
+        "--allow-plc-admin",
+        action="store_true",
+        default=False,
+        help="Allow disruptive administrative tests on a dedicated non-production PLC",
+    )
+    parser.addoption("--plc-protocol", choices=("legacy_s7", "s7commplus"), default="legacy_s7")
+    parser.addoption("--plc-report-json", default="", help="Write a sanitized real-PLC JSON report")
+    parser.addoption("--tester", default="", help="GitHub handle of the volunteer running the test")
+    parser.addoption("--plc-family", default="", help="Reportable PLC family, such as S7-1500")
+    parser.addoption("--plc-model", default="", help="Reportable PLC model")
+    parser.addoption("--plc-order-code", default="", help="Reportable PLC order code")
+    parser.addoption("--plc-firmware", default="", help="Reportable PLC firmware version")
+    parser.addoption("--plc-security-mode", default="", help="Reportable access/TLS mode; never enter credentials")
+    parser.addoption("--plc-tia-configuration", default="", help="Reportable TIA settings, without site details")
 
 
 def pytest_configure(config: pytest.Config) -> None:
@@ -84,6 +111,9 @@ def pytest_configure(config: pytest.Config) -> None:
         "markers",
         "e2e: mark test as end-to-end test requiring real PLC connection",
     )
+    global _REAL_PLC_REPORT
+    _REAL_PLC_REPORT = RealPLCReport()
+    config._real_plc_report = _REAL_PLC_REPORT
 
 
 def pytest_collection_modifyitems(config: pytest.Config, items: list[pytest.Item]) -> None:
@@ -104,14 +134,26 @@ def pytest_collection_modifyitems(config: pytest.Config, items: list[pytest.Item
             e2e.DB_READ_ONLY = int(config.getoption("--plc-db-read"))
             e2e.DB_READ_WRITE = int(config.getoption("--plc-db-write"))
 
-    # Skip e2e tests if flag not provided
-    if config.getoption("--e2e"):
-        return
-
-    skip_e2e = pytest.mark.skip(reason="Need --e2e option to run end-to-end tests")
     for item in items:
-        if "e2e" in item.keywords:
-            item.add_marker(skip_e2e)
+        if "e2e" in item.keywords and not config.getoption("--e2e"):
+            item.add_marker(pytest.mark.skip(reason="Need --e2e option to run end-to-end tests"))
+        if ("write" in item.keywords or "plc_write" in item.keywords) and not config.getoption("--allow-plc-write"):
+            item.add_marker(pytest.mark.skip(reason="WRITE_OPT_IN_REQUIRED: pass --allow-plc-write"))
+        if "administrative" in item.keywords and not config.getoption("--allow-plc-admin"):
+            item.add_marker(pytest.mark.skip(reason="ADMIN_OPT_IN_REQUIRED: pass --allow-plc-admin"))
+
+
+def pytest_runtest_logreport(report: Any) -> None:
+    """Capture final BDD outcomes without recording connection secrets."""
+    _REAL_PLC_REPORT.record(report)
+
+
+def pytest_sessionfinish(session: pytest.Session, exitstatus: int) -> None:
+    """Write the optional structured report after all scenario outcomes are known."""
+    del exitstatus
+    report_path = session.config.getoption("--plc-report-json")
+    if report_path:
+        session.config._real_plc_report.write(Path(report_path), report_metadata(session.config))
 
 
 @pytest.fixture(scope="session")

tests/features/real_plc/connection.feature

@@ -0,0 +1,26 @@
+@real_plc
+Feature: Connect to and identify a real PLC
+
+  Background:
+    Given I am using a dedicated test PLC
+    And the test configuration contains no secrets in reportable fields
+
+  @smoke
+  Scenario Outline: Establish and close a session
+    Given the client uses the "<protocol>" protocol path
+    When I connect to the configured PLC
+    Then the client reports that it is connected
+    And the negotiated protocol and security mode are recorded
+    And the PLC identity and CPU state are recorded when available
+    When I disconnect
+    Then the client reports that it is disconnected
+
+    @legacy_s7
+    Examples: legacy S7
+      | protocol   |
+      | legacy_s7  |
+
+    @s7commplus
+    Examples: S7CommPlus
+      | protocol   |
+      | s7commplus |

tests/features/real_plc/lifecycle.feature

@@ -0,0 +1,37 @@
+@real_plc
+Feature: Recover from normal connection lifecycle events
+
+  @smoke
+  Scenario Outline: Reconnect after a clean disconnect
+    Given the client uses the "<protocol>" protocol path
+    And I connected to and disconnected from the PLC
+    When I reconnect with the same configuration
+    Then a known read succeeds
+
+    @legacy_s7
+    Examples: legacy S7
+      | protocol   |
+      | legacy_s7  |
+
+    @s7commplus
+    Examples: S7CommPlus
+      | protocol   |
+      | s7commplus |
+
+  @smoke
+  Scenario Outline: Repeated operations do not corrupt the session
+    Given the client uses the "<protocol>" protocol path
+    And I am connected to the PLC
+    When I read the canonical fixture repeatedly
+    Then every read succeeds with the expected value
+    And disconnect completes cleanly
+
+    @legacy_s7
+    Examples: legacy S7
+      | protocol   |
+      | legacy_s7  |
+
+    @s7commplus
+    Examples: S7CommPlus
+      | protocol   |
+      | s7commplus |