test(qa): hold the DOCX fidelity corpus to its baseline whichever way a line moved

.github/workflows/ci.yml

@@ -538,9 +538,13 @@ jobs:
     # Every template preset, and the documents built for shapes no preset has, exported to
     # DOCX, set by LibreOffice, and held line by line to
     # qa/src/test/resources/docx-fidelity/libreoffice-linux*.tsv
-    # (DocxFidelityCorpusTest): a change that sets any line of any document further
-    # from the engine's PDF, down or across, at other words, or on more pages fails here. Gated on
-    # `docx` — what an export of the corpus is made from.
+    # (DocxFidelityCorpusTest): a change that moves any line of any document more than 0.5pt from
+    # where the baseline holds it — down or across, nearer the engine's PDF or further — or sets a
+    # document at other words, on another number of pages, with another set of its lines found, or
+    # one the engine draws on another number of pages or lines, fails here. A move nearer the page is the export doing better and fails all the same, until
+    # the change records it by rewriting the baseline: held one-sidedly, the Word baseline drifted
+    # stale in 1609 of its 5165 rows over 74 commits without a single red run (FidelityBaseline's
+    # javadoc has the breakdown). Gated on `docx` — what an export of the corpus is made from.
     #
     # The image is pinned, not ubuntu-latest: the baseline is LibreOffice's setting
     # of the text, and a new image's LibreOffice sets some lines a little apart. A

CHANGELOG.md

@@ -4345,6 +4345,60 @@ follow semantic versioning; release dates are ISO 8601.
 
 ### Tests
 
+- **The DOCX fidelity gate holds each document where its baseline puts it, whichever way it
+  moved, instead of only catching lines set further from the page.**
+  - A document failed when a line stood more than 0.5pt further from the engine's PDF than its
+    baseline held it, when a line the baseline found was no longer found, or when the editor set
+    it on a page count further from the engine's. A line that moved nearer the page, a page count
+    that came nearer, and a line found that the baseline did not hold all passed, at any size.
+  - Each is now measured as a step from where the baseline holds it: a line fails when its drift
+    moves more than 0.5pt, down or across, nearer the page or further, and a document fails when
+    the editor sets it on another number of pages, when the engine draws it on another number of
+    pages or another number of lines, or
+    when the set of its lines found is not the set the baseline holds. The failure names each
+    line's step — `+1.10 -> +0.00pt (-1.10)` — so the direction is in the message.
+  - Measured on the gate as it stood: a line moving 1.10pt nearer the page, one moving 5pt
+    nearer, a page count coming onto the engine's own, and a line newly found were each let
+    through in silence, as was `+3.00 -> -3.00pt` — six points of movement, which a rule comparing
+    which of two drifts is further from the page cannot see at all, since both stand three points
+    away.
+  - A change that moves documents, nearer the page included, now rewrites the baseline in the
+    same commit (`-Dgraphcompose.docxFidelity.update=true`), and that diff is the record of what
+    moved. A rewrite now also keeps the notes the file it replaces carried, since those are where
+    a row's reckoning is written and a rewrite is the ordinary answer to a document moving.
+  - Held one-sidedly, the Word baseline went stale in 1609 of its 5165 rows over 74 commits
+    without one red run: 1388 rows moved down the page — 717 nearer it, 635 further, and 36
+    crossing it to stand the same distance on the other side — and 229 moved across, 228 of them
+    nearer by as much as 1.10pt. Of all that, 228 rows had moved more than the slack, 226 across
+    and 2 down. And yet no row's *distance* from the page had grown by more than 0.48pt — 21 rows
+    of `cv-serif_headline`, `cv-panel` and `invoice-obsidian` by that much, the rest by 0.36pt or
+    less — which against half a point of slack is exactly why a rule reading distances stayed
+    quiet, and those three documents are the ones that ended 0.48pt further.
+  - A baseline line row whose drift or place across is not a number is now refused. Every
+    comparison against `NaN` is false, so such a row held its line to nothing however far it
+    moved — and these files are patched by hand, so one mistyped cell was enough.
+  - The slack stays one number for both directions: it is rounding, and rounding is the same size
+    whichever way a line rounds. A step of exactly 0.5pt is still rounding, for every drift from
+    0 to 5pt and in both directions. The aggregates a row carries — its median, its p90, its
+    lines past 2pt — are held to nothing: they follow from the lines, which are held one by one.
+  - The export does not change: the 71-document corpus exports byte for byte as before, and the
+    Word baseline needs no rewrite — Word 16.0.20527 measures all 71 documents and 5165 line rows
+    exactly as `word-windows.tsv` holds them, not a row differing.
+  - `libreoffice-linux.tsv` is rewritten here, from the run of this change. Not shot whole since
+    #898, it had gone stale in 1778 of its 5149 rows, of which 230 stood more than half a point
+    from where it held them — 226 across and 4 down, in 8 documents — and **every one of the 230
+    stood nearer the engine's PDF, not one further**, on the LibreOffice build the file names. So
+    the two-sided rule's first real run recorded 230 improvements the one-sided rule had swallowed,
+    which is what it is for; the file's note records the run and the reckoning.
+  - `libreoffice-windows.tsv` is left as it stands and marked stale in the file instead, because
+    it is the opposite case. Nothing in CI runs it, and it is red already under the rule it was
+    written for: 2 documents, 8 lines no longer found, 2 page counts and 65 lines standing more
+    than half a point further. `proposal-indigo` accounts for the worst, 8 lines 10.25pt lower
+    than it holds them on the build it names, so the move is the export's and not the editor's.
+    Rewriting it alongside the Linux file would bury a 10pt move in a baseline diff; a move
+    further from the page wants attributing to the commit that made it first, as the Word
+    baseline's drift was attributed before #975 rewrote it.
+
 - **The DOCX fidelity corpus holds an icon laid over a card's corner, as the export writes it
   today.**
   - `SyntheticDocxCorpus` takes `icon_over_card`: a layer stack laying an 18pt icon over the top

CONTRIBUTING.md

@@ -349,34 +349,56 @@ Choose the smallest tests that match the change:
   [DocxFidelityCorpusTest.java](qa/src/test/java/com/demcha/compose/document/templates/fidelity/DocxFidelityCorpusTest.java).
   It exports every template preset to DOCX, with documents built for shapes no preset has on
   its own fixture, has LibreOffice set each one, and holds every
-  line of every document to the drift its baseline records, down and across: a line set further
-  from the page either way, a line set at other words, or a page more fails it. It needs
+  line of every document to the drift its baseline records, down and across: a line that
+  moved more than 0.5pt from where the baseline holds it, nearer the page or further, a line
+  set at other words, a line found that the baseline does not hold, another number of pages
+  from the editor, or another number of pages or lines from the engine fails it. It needs
   LibreOffice and runs only when asked for — install first, as for any standalone `qa` run:
   `./mvnw -B -ntp test -f qa/pom.xml -Dtest=DocxFidelityCorpusTest -Dgraphcompose.docxFidelity=libreoffice`.
-  A change that moves documents nearer the page rewrites the baseline with
+  A change that moves documents, nearer the page included, rewrites the baseline with
   `-Dgraphcompose.docxFidelity.update=true`, and commits it with the change: its diff shows
-  which documents moved. A new preset joins the corpus in its family's `*DocxCorpus` class,
-  and so does a preset on a longer fixture; a shape no preset has — a page break inside a
+  which documents moved, and which way. A new preset joins the corpus in its family's
+  `*DocxCorpus` class, and so does a preset on a longer fixture; a shape no preset has
+  — a page break inside a
   panel, a break in a row's column, a shape container narrower than the block it stands in
   or clipped short of its text, an icon a layer stack lays over a card's corner — goes in
   `SyntheticDocxCorpus`. Its baseline holds it as the editors set it, a defect of the export
   included, and the change fixing that rewrites its rows nearer the page.
   CI runs it on Linux (the `DOCX Fidelity` job) against `libreoffice-linux.tsv`, and uploads
   what it measured: LibreOffice sets text a little differently per platform, so a change that
-  moves documents nearer the page takes that artifact's files as the Linux baseline, and its own
-  run's as the Windows one. Microsoft Word, the editor the export answers to, is not in CI: on
+  moves documents takes that artifact's files as the Linux baseline, and its own run's as the
+  Windows one. Microsoft Word, the editor the export answers to, is not in CI: on
   Windows with Word, `scripts/docx-visual/word-fidelity.ps1` (`-Update` to rewrite) exports the
   corpus, has a private Word instance convert it, and holds it to `word-windows.tsv` — run it
   from PowerShell (pwsh), after the install above, before a DOCX export change is opened.
-  Rewrite the baseline in the change that moves the documents: the gate cannot remind you, as
-  it fails a line only once it stands more than 0.5pt further from the page than the baseline
-  holds it, so a change that moves documents nearer — or less than that further — passes with
-  the baseline left as it was. Between #899 and #974 that left 1609 of the Word baseline's 5165
-  rows stale over 74 commits, 220 of them standing about a point nearer the page, until three
-  documents stood 0.48pt further and the next change to touch them would have failed the gate
-  for drift it did not cause. Each baseline's header records the build it was shot on, and a
+  Rewrite the baseline in the change that moves the documents, and the gate will remind you —
+  as long as you run it, which for Word means running the script above: it fails a line that has
+  moved more than 0.5pt from where the baseline holds it, nearer the page or further, a document
+  the editor sets on another number of pages, one the engine draws on another number of pages or
+  lines, and one whose lines the editor finds a different set of. Until the gate was made
+  two-sided it only looked for lines set further, which let a change move documents nearer, or
+  less than 0.5pt further, with the baseline left as it was: between #899 and #974 that left
+  1609 of the Word baseline's 5165 rows stale over 74 commits, 226 of them standing more than
+  half a point nearer the page across, until three documents stood 0.48pt further and the next
+  change to touch them would have failed the gate for drift it did not cause. The slack is read
+  afresh against each rewrite, so it bounds no total: what two-sidedness buys is that every move
+  past rounding is seen and written down, not that drift cannot accumulate over many changes.
+  A failure naming lines that moved nearer the page is not a defect to fix — it is the export
+  doing better, and the baseline is how it gets recorded. A rewrite keeps the `#` notes the file
+  it replaces carried, by either path, and leaves it to you to prune the ones it has made stale;
+  a note is a block under the header, not an annotation of the rows below it, and comes back
+  above them. Each baseline's header records the build it was shot on, and a
   differing build is rarely the cause: Word 16.0.20430 and 16.0.20527 measure the corpus as
   #899 exported it identically, row for row.
+  `libreoffice-windows.tsv` is known stale, and the local LibreOffice run fails on it: 2
+  documents fail even one-sidedly — 8 lines no longer found, 2 page counts and 65 lines more
+  than half a point further, 8 of them `proposal-indigo`'s, 10.25pt lower than it holds them
+  on the LibreOffice build the baseline names. It has not been shot whole since #898, where
+  `libreoffice-linux.tsv` now has. Do not rewrite it as part of an unrelated change: a move
+  further from the page wants the commit that made it found first, as the Word baseline's drift
+  was attributed before #975 rewrote it, and the file rewritten with that reckoning. Until then,
+  read a local LibreOffice run against the documents your change touches rather than against a
+  green run.
   Each run also writes `measured-<editor>-acceptance.md`: which documents the editor sets as the
   page does — the same pages, 95% of the lines found, a median of 1pt or less, no more than 10%
   past 2pt — and what keeps the rest short of it. CI shows LibreOffice's in the job summary.

qa/src/test/java/com/demcha/compose/document/templates/fidelity/DocxFidelityCorpusTest.java

@@ -24,14 +24,14 @@
 
 /**
  * Every template preset's DOCX, and that of each document built for a shape no preset has
- * ({@link SyntheticDocxCorpus}), set by an editor, stands no further from the engine's PDF than
- * its baseline holds it.
+ * ({@link SyntheticDocxCorpus}), set by an editor, stands where its baseline holds it.
  *
  * <p>The engine draws each corpus document to PDF and exports it to DOCX; an editor converts the
  * DOCX to PDF; each document is measured line by line against the page
  * ({@link FidelityMeasurement}) and held to the baseline for this editor and operating system
- * ({@link FidelityBaseline}). A change to the export that sets any document further from the
- * page fails here, wherever the document is.</p>
+ * ({@link FidelityBaseline}). A change to the export that moves any document fails here,
+ * wherever the document is and whichever way it moved: a move nearer the page is the export
+ * doing better and is recorded the same way, by rewriting the baseline in the same commit.</p>
  *
  * <p>It runs only when asked for, with {@code -Dgraphcompose.docxFidelity}, and takes a few
  * minutes; a value it does not know fails. {@code libreoffice} exports, converts with
@@ -41,9 +41,9 @@
  * Word converts them, and {@code word} measures Word's PDFs against {@code word-windows.tsv} —
  * each only when the DOCX Word converted, by the SHA-256 its conversion records, is the one
  * this tree exports ({@link WordConversion}). {@code -Dgraphcompose.docxFidelity.update=true}
- * writes what it measured as the baseline, for a change that moves documents nearer the page;
- * the baseline's diff then shows the reviewer which documents moved. The documents, their PDFs
- * and the report are left in {@code target/docx-fidelity}.</p>
+ * writes what it measured as the baseline, which is how a change that moves documents records
+ * the move; the baseline's diff then shows the reviewer which documents moved, and which way.
+ * The documents, their PDFs and the report are left in {@code target/docx-fidelity}.</p>
  */
 class DocxFidelityCorpusTest {
 
@@ -52,7 +52,7 @@ class DocxFidelityCorpusTest {
     private static final String EXPORT = "export";
 
     @Test
-    void everyDocumentStandsNoFurtherFromThePageThanItsBaseline() throws Exception {
+    void everyDocumentStandsWhereItsBaselineHoldsIt() throws Exception {
         String mode = System.getProperty("graphcompose.docxFidelity", "");
         Assumptions.assumeTrue(!mode.isEmpty(), "NOT_RUN: the DOCX fidelity corpus runs with "
                 + "-Dgraphcompose.docxFidelity=libreoffice, or with Word through scripts/docx-visual/word-fidelity.ps1");
@@ -104,20 +104,23 @@ void everyDocumentStandsNoFurtherFromThePageThanItsBaseline() throws Exception {
             measurements.add(FidelityMeasurement.of(document.stem(),
                     engine.resolve(document.stem() + ".pdf"), converted));
         }
-        FidelityBaseline.write(work.resolve("measured-" + mode + ".tsv"), note, measurements);
-        FidelityAcceptance.write(work.resolve("measured-" + mode + "-acceptance.md"), note, measurements);
-
         Path baseline = Path.of("src", "test", "resources", "docx-fidelity",
                 mode + "-" + LibreOfficeConverter.platform() + ".tsv");
-        List<String> regressions = FidelityBaseline.read(baseline).regressions(measurements);
+        // Written with the baseline's notes, so that copying this file over the baseline — what a
+        // run on a machine the baseline cannot be shot on, CI's, is for — keeps them.
+        FidelityBaseline.write(work.resolve("measured-" + mode + ".tsv"), note, measurements, baseline);
+        FidelityAcceptance.write(work.resolve("measured-" + mode + "-acceptance.md"), note, measurements);
+        List<String> divergences = FidelityBaseline.read(baseline).divergences(measurements);
         if (Boolean.getBoolean("graphcompose.docxFidelity.update")) {
             // Written all the same: what it lets through is the baseline's diff, shown here first.
-            regressions.forEach(regression -> System.err.println("baseline rewritten over: " + regression));
+            divergences.forEach(divergence -> System.err.println("baseline rewritten over: " + divergence));
             FidelityBaseline.write(baseline, note, measurements);
             return;
         }
-        assertThat(regressions)
-                .as("documents set further from the page than %s holds them (measured: %s)",
+        assertThat(divergences)
+                .as("documents standing apart from %s, whichever way they moved: a change that moves "
+                    + "them — nearer the page included — rewrites the baseline in the same commit, with "
+                    + "-Dgraphcompose.docxFidelity.update=true, so its diff shows what moved (measured: %s)",
                         baseline, work.resolve("measured-" + mode + ".tsv"))
                 .isEmpty();
     }