Repository navigation
How should specVersion evolve while the schema remains 0.0.0? #69
Description
Producer context
Microsoft.Testing.Platform validates its CTRF mapping against the upstream specification and currently emits a hard-coded "specVersion": "0.0.0":
- https://github.com/microsoft/testfx/blob/main/src/Platform/Microsoft.Testing.Extensions.CtrfReport/CtrfReportEngine.cs#L6-L11
- https://github.com/microsoft/testfx/blob/main/src/Platform/Microsoft.Testing.Extensions.CtrfReport/CtrfReportEngine.JsonSerializer.cs#L59-L70
The current specification says:
- its version is
0.0.0and this corresponds directly tospecVersion; specVersionfollows SemVer;- producers must use the highest CTRF version they fully support;
- consumers should support PATCH changes within a MAJOR.MINOR version.
However, normative schema changes such as the identity model (PR #57, merged July 2026) and clarified retry history (PR #62, merged August 2026) were added while the specification and schema still identify themselves as 0.0.0. The specification header date is also still 2025-11-24. Discussion #30 asks about the v1 roadmap but has no answer.
Questions
- Does
0.0.0mean “the latest revision onmain”, or does it identify a fixed schema revision? - When a normative field or constraint is added before 1.0, should
specVersionreceive a PATCH/MINOR bump (for example0.0.1or0.1.0)? - If all pre-1.0 revisions intentionally remain
0.0.0, how can a producer truthfully identify which schema revision it fully supports, and how can a consumer select the matching validator? - Is there or will there be a tagged/released schema artifact, immutable schema URL, or separate revision identifier that producers should record?
- Until that exists, should producers pin a commit and still emit
0.0.0, or is there another recommended compatibility policy?
Why this matters
Two documents can both claim specVersion: 0.0.0 while conforming to materially different versions of the normative schema. MTP needs a stable contract for deciding when to update its mapping and tests, and CTRF consumers need a reliable way to interpret and validate reports. A brief pre-1.0 release/versioning policy would resolve this without requiring the project to commit to v1 immediately.
Additional conflicting version signals
The current repository also contains a stronger contradiction than the unchanged 0.0.0 header alone:
- the standalone examples (
examples/minimal.json,examples/with-retries.json, andexamples/comprehensive.json) currently emit"specVersion": "1.0.0"; - normative conformance tests also use
"specVersion": "1.0.0"; - the written specification and its embedded examples still identify the working draft as
0.0.0; - the roadmap still shows the v1 release as incomplete, and the repository has no tags or releases.
That leaves a producer unable to tell whether 1.0.0 is already the intended interoperable value, whether those examples are premature, or whether the specification header is stale. Clarifying the currently valid value would be especially helpful in addition to defining the future bump policy.
Activity
gregweb commentedon Oct 2, 2026
It also poses a problem when releasing a new version of a producers.
What to write in the release notes?
I would have prefered to write:
Proposal
Tag with incrementing
xoryin0.x.yafter each group of changes that were commited.Ma11hewThomas commentedon Oct 3, 2026
Thanks both for raising this and for the suggestions!
I've addressed the versioning concerns in #71 , as well as fixing the inconsistencies identified here. The PR has now been merged.
The spec, examples, and conformance fixtures are now aligned on 0.1.0, and the pre-1.0 versioning and tagging strategy is documented to make it clear what producers should emit and how consumers can pin to a specific revision.
See RELEASE.md and CHANGELOG.md for more details.
We're also assessing whether to introduce GitHub Releases ahead of v1.
Please take a look and let me know if this addresses the concerns raised here.
gregweb commentedon Oct 3, 2026
Many thanks, Mathew.
Speaking for myself, with releasing version 0.1.0, you addressed my concerns.
Evangelink commentedon Oct 3, 2026
Thanks, this fully addresses the questions I raised.
The new release policy gives producers and consumers the missing stable contract:
specVersionidentifies a published tagged release, not the moving contents ofmain.vMAJOR.MINOR.PATCHtags are the canonical release record and provide stable specification and schema URLs.v0.1.0and emit"specVersion": "0.1.0".I also verified that the tagged
v0.0.4andv0.1.0schema files are byte-for-byte identical, so adopting0.1.0is a metadata correction rather than a report-shape change.The TestFx follow-up is tracked in microsoft/testfx#11731, which updates the reporter to emit
0.1.0and pins its documentation to the tagged schema and specification.One TestFx-specific follow-up remains, but it is not an upstream specification gap and does not block closing this issue: before our merger supports reports from different pre-1.0 MINOR lines, it should enforce that compatibility boundary instead of inheriting the first input's version. The current same-toolchain flow does not require that change.
With the specification questions resolved, I am closing this issue.