Skip to content

Letter the tests 3.1r1 doesn't number - #69

Merged
jadamcrain merged 1 commit into
mainfrom
lettered-ids
Oct 14, 2026
Merged

jadamcrain merged 1 commit into
mainfrom
lettered-ids

Conversation

@jadamcrain

Copy link
Copy Markdown
Member

New ids

3.1r1 leaves 103 tests unnumbered. Each one now gets its section's number followed by a letter for its place among the section's unnumbered tests, in document order:

Before After
8.6.5-2 8.6.5b
8.14.2-3 8.14.2c
8.21-8.13.2-4 8.21d
8.22-8.14.2-16 8.22p
  • No cross-references. An id no longer refers to another section. 8.21 and 8.22, which rerun 8.13's and 8.14's tests with double-bit inputs, are lettered in the same order as the tests they rerun.
  • Can't be mistaken for 3.1r1's own numbers. 3.1r1 numbers only with digits, so 8.22a sits beside 8.22.1. There's no dot before the letter, so an id doesn't look like a step label such as TB2017-001's "1.a".
  • Enough letters. The largest section needs 16 letters, a to p. A new test checks that each section's letters run a, b, c… with no gap.

New titles for the double-bit reruns

The 20 double-bit reruns now say which test each one runs. For example, "Double-bit Input Change, Qualifier 0x06 (as 8.14.2c)" replaces "Binary Input Change, Qualifier 0x06, with double-bit inputs".

Other mentions of the ids

Comments, test names, option descriptions, the README and docs/manual-testing.md all use the new ids, and their explanations of the scheme describe the letters.

Checks

I compared a dump of both revisions' trees, every procedure's -describe output, the disabled lists and Revision.validate, before and after, with the new ids mapped back to the old ones. It covered every subset level in three configurations. The only differences are the 20 new titles. mvn test passes.

Saved results

Results saved under the old ids don't count in a report under the new ones. No release has 3.1r1 yet, so this only affects local runs, which need running again. stepfunc/dnp3's conformance device.ini names some of these ids in a comment; that gets updated with its next dnp3c pin.

A test 3.1r1 leaves unnumbered was named after its section and its place
with a dash, 8.6.5-2, and 8.21's and 8.22's reruns of 8.13's and 8.14's
tests after both sections, 8.22-8.14.2-3. Now each has its section's
number followed by a letter for its place among the section's unnumbered
tests, in document order: 8.6.5b, 8.22c. Ids no longer refer to other
sections, and a letter can't be mistaken for one of 3.1r1's own numbers:
8.22a sits beside 8.22.1. No section has more than 16 such tests, and a
test checks each section's letters run a, b, c and on without a gap.

The double-bit reruns' titles say which test each runs, e.g.
"Double-bit Input Change, Qualifier 0x06 (as 8.14.2c)", in place of 8.14's
title with ", with double-bit inputs".

Every procedure's steps are unchanged.
@jadamcrain
jadamcrain merged commit 1a54223 into main Oct 14, 2026
5 checks passed
@jadamcrain
jadamcrain deleted the lettered-ids branch October 14, 2026 23:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant