fix(docx): stack drawings in Word as the page paints them

CHANGELOG.md

@@ -8,6 +8,27 @@ follow semantic versioning; release dates are ISO 8601.
 
 ### Public API
 
+- **A DOCX export stacks its drawings as the page paints them.** Word stacks drawings by the
+  place each is given (`wp:anchor/@relativeHeight`), a later one over an earlier one. The export
+  gave them their places in the order it drew them. In the flow it draws a stack's or a shape
+  container's layers in the order given, and columns' marks after the columns' own content. So
+  a layer the page stacks under another by its `zIndex` but given after it stood over it.
+  - Measured in Word and LibreOffice on Windows, a white dot (`zIndex` 0) given after a disc
+    (`zIndex` 1) stood over the disc in both. The page hides the dot. This held in a stack that
+    only draws, in a shape container, in a band of layers with text, and in a stack nested in
+    such a band.
+  - A shape container whose layers the page stacks in another order was named for it. Every
+    other case went unsaid.
+  - Each drawing now takes its place from where the page paints the fragment it is drawn from.
+    Both editors draw the dot under the disc in all four cases. The text keeps the order it is
+    written in.
+  - The places order the drawings behind the text among themselves, and those in front of it.
+    Word draws every one behind under every one in front: on a painted panel a layer framing
+    text stays behind while one that does not stands in front, whatever their `zIndex`.
+  - The shape container is no longer named for its order.
+  - In the fidelity corpus, 34 of the 62 files change only in those places. 26 of them stack as
+    before. The other 8 change the order of some drawings, and their pages render pixel for
+    pixel as before at 96 dpi in Word and in LibreOffice.
 - **A DOCX export names what a text header or footer shows otherwise in Word.** Measured in
   Word and LibreOffice on Windows, four things went unsaid:
   - **A band too shallow for its line.** Its line is written from the page's edge.

docs/architecture/backend-capability-matrix.md

@@ -123,7 +123,7 @@ honour an option ignores it (documented contract).
 | Viewer preferences | ✅ `applyViewerPreferences` in `PdfFixedLayoutBackend` | ❌ (ignored with a one-time warning — PDF-viewer concept) | n/a (not written; reported `DROPPED`, `viewer preferences`) |
 | Debug guide lines / node labels | ✅ `PdfGuideLinesRenderer`, `PdfNodeLabelRenderer` | ❌ (ignored with a one-time warning — render through the PDF backend to see overlays) | n/a |
 | Keep a block on one page (`keepTogether()`, `keepWithNext()`) | ✅ resolved by `LayoutCompiler` before any backend runs | ✅ same — the slides are the laid-out pages | ✅ `DocxSemanticBackend.keepOnOnePage` — Word re-paginates, so a block the layout placed on one page is told to stay there: `w:keepLines` on each of its paragraphs and `w:keepNext` on every one but the last (on the last too for `keepWithNext`), a table inside it chained row by row. A block that ran over a page break in the layout is taller than a page and is left to flow, as the layout left it. A body table of one row kept with what follows, its cells holding paragraphs alone, is written with a row that may break (`letTheOneRowTablesKeptWithTheNextBreak`): LibreOffice reads a one-row table whose row may not break as one that may not split, and keeps such a table with nothing |
-| Layer stack — layers drawn over one another (`addLayerStack`) | ✅ each layer's fragments at the place the layout gave it | ✅ same | ⚠️ `DocxSemanticBackend` — Word has no layers, so a stack's children are written one after the other, without their positions. The exception is a stack whose layers are side-by-side columns (`DocxLayerColumns`): every layer a plain container at the stack's top-left corner, the bands their padding leaves either the same or apart. That is written as one table row, a cell per band, the way a two-column CV lays its columns out as layers to draw the name first. Layers sharing a band follow one another in its cell. A spacer the layout placed level with content of another layer of the same band only keeps that content's place, and is not written. The space above a later layer's first block is the gap the page shows below the content before it. A layer the page starts above where those before it end — a caption over its photo, a label in a card's corner — is written under them, in a column's cell as in a band, and the report names it; a stack composed in a table cell, or exported with no layout, has no place of its own, and one writing more than one of its layers is named with them not measured. What the page draws inside another layer's fill comes after that fill. Measured in LibreOffice: `CharcoalGold`, `SidebarPortrait` and `SlateOrange` fit on one page, as on the page; `NavySidebar` runs one line onto a second page. A stack or shape container the page gives no room — its margins taking back its whole height, as `LumaStudioInvoice`'s sidebar over the page's top margin — is laid over the flow (`laidOverTheFlow`): its drawings where the page draws them, each paragraph in a text box in front of the text where the page sets it, nothing in the flow. Not in a table cell or a panel, and not when it holds anything but drawing and plain paragraphs — a link, an anchor, a picture, a list or a table keeps it in the flow as above. A canvas is written as its contents — what it writes one block after another, its drawings where it places them — and holds its height: the room under what it writes is the space below it, and one that only draws holds its whole box where nothing round it — a band before its layer's first block, drawings held whole, an overlay it is a layer of — measures that room already. The report names the places and the width it loses, as it does a column's fixed width narrower than its band, and what it writes running past its height, a drawing in what it writes that takes no room in Word, or a height not measured |
+| Layer stack — layers drawn over one another (`addLayerStack`) | ✅ each layer's fragments at the place the layout gave it | ✅ same | ⚠️ `DocxSemanticBackend` — Word has no layers, so a stack's children are written one after the other, without their positions. The exception is a stack whose layers are side-by-side columns (`DocxLayerColumns`): every layer a plain container at the stack's top-left corner, the bands their padding leaves either the same or apart. That is written as one table row, a cell per band, the way a two-column CV lays its columns out as layers to draw the name first. Layers sharing a band follow one another in its cell. A spacer the layout placed level with content of another layer of the same band only keeps that content's place, and is not written. The space above a later layer's first block is the gap the page shows below the content before it. A layer the page starts above where those before it end — a caption over its photo, a label in a card's corner — is written under them, in a column's cell as in a band, and the report names it; a stack composed in a table cell, or exported with no layout, has no place of its own, and one writing more than one of its layers is named with them not measured. What the page draws inside another layer's fill comes after that fill. Drawings stack as the page paints them, by their z-index, each given its place (`wp:anchor/@relativeHeight`) from where the page paints its fragment, whatever order the layers are written in — among the drawings behind the text, and among those in front of it, as Word draws every one behind under every one in front. Measured in LibreOffice: `CharcoalGold`, `SidebarPortrait` and `SlateOrange` fit on one page, as on the page; `NavySidebar` runs one line onto a second page. A stack or shape container the page gives no room — its margins taking back its whole height, as `LumaStudioInvoice`'s sidebar over the page's top margin — is laid over the flow (`laidOverTheFlow`): its drawings where the page draws them, each paragraph in a text box in front of the text where the page sets it, nothing in the flow. Not in a table cell or a panel, and not when it holds anything but drawing and plain paragraphs — a link, an anchor, a picture, a list or a table keeps it in the flow as above. A canvas is written as its contents — what it writes one block after another, its drawings where it places them — and holds its height: the room under what it writes is the space below it, and one that only draws holds its whole box where nothing round it — a band before its layer's first block, drawings held whole, an overlay it is a layer of — measures that room already. The report names the places and the width it loses, as it does a column's fixed width narrower than its band, and what it writes running past its height, a drawing in what it writes that takes no room in Word, or a height not measured |
 
 ## Output surface and lifecycle
 

docs/recipes/docx-export.md

@@ -894,8 +894,16 @@ recolour what is under it in Word and it keeps the colour it was flattened to.
   export, and a note on each container that writes a layer in the flow. A
   container whose layers are all drawn — a tile, an icon, a badge holding
   its glyph — writes none: each layer is drawn where the page draws it, in
-  source order, and it is not noted unless the page stacks its layers in
-  another order, by their z-index. The report names the clip — a shape container's, and a layer
+  the order the page stacks them, by their z-index. Word stacks drawings by
+  the place each is given, a later one over an earlier one, and every drawing
+  takes its place from where the page paints it, wherever and whenever the
+  export draws it: a stack's layers, a band's and columns' marks, a table's
+  cell drawings. Text is written in the order given. The places order the
+  drawings behind the text among themselves, and those in front of it: Word
+  draws every one behind under every one in front, and on a painted panel a
+  layer framing text stays behind while one that does not stands in front,
+  whatever their z-index.
+  The report names the clip — a shape container's, and a layer
   stack's that clips to its bounds — wherever it cuts what the layers paint:
   a square tile's corners in a disc, a label run past its chip, an ornament
   set past a sidebar's side. One composed in a table's cell, which has no

render-docx/src/main/java/com/demcha/compose/document/backend/semantic/docx/DocxDrawingAnchors.java

@@ -57,8 +57,10 @@
  * in that cell's paragraph, placed from the paragraph and the cell's text column, so it moves
  * with its row; the cell holds it whole, so Word's clipping cuts nothing.</p>
  *
- * <p>Every shape takes its place in the paint order when it is drawn ({@link #ordered}), not
- * when it finds its paragraph.</p>
+ * <p>Every shape takes its place in the paint order from where the page paints the fragment it
+ * is drawn from ({@link #ordered}), not when it is drawn or when it finds its paragraph: Word
+ * stacks drawings by those places, a later one over an earlier one, and the export draws a
+ * stack's layers, a band's and a table's cells in the order it writes them.</p>
  */
 final class DocxDrawingAnchors {
 
@@ -78,8 +80,21 @@ final class DocxDrawingAnchors {
     private final List<Offer> offers = new ArrayList<>();
     // Identifiers unique in the document, shared with the other drawings the export writes.
     private final LongSupplier ids;
-    // The order the shapes are painted in, across the document.
-    private int order;
+    // The places given each fragment's shapes in the section so far, by the fragment's place in
+    // the page's paint order.
+    private final Map<Integer, Integer> placesTaken = new HashMap<>();
+    // The latest place given: a shape drawn from no fragment the page paints is drawn as the
+    // fragment that place is of.
+    private int latest = -1;
+    // How many shapes have been drawn in the document, which numbers their names.
+    private int drawn;
+
+    /**
+     * How many places in the paint order a fragment's shapes take, at most: one drawn as more
+     * shapes than that stacks the rest at its last place. A fragment is drawn as five shapes at
+     * most ({@link DocxDrawings#of}).
+     */
+    static final int PLACES_PER_FRAGMENT = 8;
 
     DocxDrawingAnchors(LongSupplier ids) {
         this.ids = ids;
@@ -94,29 +109,43 @@ void reset() {
         breakParagraph.clear();
         seats.clear();
         offers.clear();
-        order = 0;
+        placesTaken.clear();
+        latest = -1;
+        drawn = 0;
     }
 
     /**
      * A shape and its place in the paint order: a later one is drawn over an earlier one.
      *
      * @param shape the shape
-     * @param order its place, given when it was drawn, whenever it is anchored
+     * @param order its place, from where the page paints the fragment it is drawn from, whenever
+     *              it is anchored
+     * @param drawn how many shapes were drawn in the document before it, which numbers its name
      */
-    record Ordered(DocxDrawings.Shape shape, int order) {
+    record Ordered(DocxDrawings.Shape shape, int order, int drawn) {
     }
 
     /**
-     * Gives shapes their places in the paint order, in the order they come, as drawn now.
+     * Gives shapes drawn from one fragment their places in the paint order: after every shape of
+     * a fragment the page paints before it, and in the order they come among that fragment's.
+     *
+     * <p>A shape is painted where the page paints it, not in the order it is drawn or finds a
+     * paragraph: a disc a dot is laid under, its layers given dot last, hides the dot as on the
+     * page; one waiting for its page's first paragraph is not painted over a shape anchored in a
+     * cell meanwhile, though painted before it.</p>
      *
-     * <p>A shape is painted in the order it is drawn, not the order it finds a paragraph: one
-     * waiting for its page's first paragraph would otherwise be painted over a shape anchored
-     * in a cell meanwhile, though drawn before it.</p>
+     * @param painted where the page paints the fragment they are drawn from, from 0, as the layout
+     *                lists its fragments; -1 where it is none the layout holds, and they are drawn
+     *                as the fragment the latest place given is of, after its shapes so far
      */
-    List<Ordered> ordered(List<DocxDrawings.Shape> shapes) {
+    List<Ordered> ordered(List<DocxDrawings.Shape> shapes, int painted) {
         List<Ordered> ordered = new ArrayList<>(shapes.size());
+        int fragment = painted >= 0 ? painted : Math.max(0, latest) / PLACES_PER_FRAGMENT;
         for (DocxDrawings.Shape shape : shapes) {
-            ordered.add(new Ordered(shape, order++));
+            int taken = placesTaken.merge(fragment, 1, Integer::sum) - 1;
+            int place = fragment * PLACES_PER_FRAGMENT + Math.min(taken, PLACES_PER_FRAGMENT - 1);
+            latest = Math.max(latest, place);
+            ordered.add(new Ordered(shape, place, drawn++));
         }
         return ordered;
     }
@@ -135,10 +164,11 @@ record Owner(String subject, String path) {
      * Gives shapes their places in the paint order and keeps them for the section's end, which
      * anchors them (see {@link #endSection}).
      *
-     * @param owner what they are drawn for, or {@code null} where no note names them
+     * @param painted where the page paints the fragment they are drawn from ({@link #ordered})
+     * @param owner   what they are drawn for, or {@code null} where no note names them
      */
-    void queue(List<DocxDrawings.Shape> shapes, Owner owner) {
-        queueOrdered(ordered(shapes), owner);
+    void queue(List<DocxDrawings.Shape> shapes, int painted, Owner owner) {
+        queueOrdered(ordered(shapes, painted), owner);
     }
 
     /** Keeps shapes given their places in the paint order for the section's end, as {@link #queue} does. */
@@ -317,6 +347,9 @@ Leftovers endSection(int lastPage, Supplier<XWPFParagraph> opening, Supplier<XWP
         cellParagraph.clear();
         breakParagraph.clear();
         seats.clear();
+        // The next section's layout counts its fragments from its own first one.
+        placesTaken.clear();
+        latest = -1;
         Map<Owner, Boolean> droppedWhole = new java.util.LinkedHashMap<>();
         droppedFor.forEach((owner, count) -> droppedWhole.put(owner, count.equals(queuedFor.get(owner))));
         return new Leftovers(inCells, dropped, droppedWhole);
@@ -347,7 +380,7 @@ void anchorInCell(XWPFParagraph carrier, List<Ordered> shapes, DocxDrawings.Cell
         XWPFRun run = carrier.insertNewRun(0);
         for (Ordered shape : shapes) {
             run.getCTR().addNewDrawing().set(
-                    DocxDrawings.drawingInCell(shape.shape(), ids.getAsLong(), shape.order(), origin));
+                    DocxDrawings.drawingInCell(shape.shape(), ids.getAsLong(), shape.order(), shape.drawn(), origin));
         }
     }
 
@@ -395,8 +428,8 @@ private void anchorBeside(XWPFParagraph paragraph, List<Placed> shapes) {
             double rise = placed.place().top() - ordered.shape().top();
             double top = rise > 0 && rise <= SLACK ? ordered.shape().top() : placed.place().top();
             run.getCTR().addNewDrawing().set(Double.isInfinite(column)
-                    ? DocxDrawings.drawingInParagraph(ordered.shape(), ids.getAsLong(), ordered.order(), top)
-                    : DocxDrawings.drawingInCell(ordered.shape(), ids.getAsLong(), ordered.order(),
+                    ? DocxDrawings.drawingInParagraph(ordered.shape(), ids.getAsLong(), ordered.order(), ordered.drawn(), top)
+                    : DocxDrawings.drawingInCell(ordered.shape(), ids.getAsLong(), ordered.order(), ordered.drawn(),
                             new DocxDrawings.CellOrigin(column, top)));
         }
     }
@@ -699,7 +732,7 @@ private record Placed(Ordered ordered, Place place) {
     private void anchor(XWPFParagraph carrier, List<Ordered> shapes) {
         XWPFRun run = carrier.insertNewRun(0);
         for (Ordered shape : shapes) {
-            run.getCTR().addNewDrawing().set(DocxDrawings.drawing(shape.shape(), ids.getAsLong(), shape.order()));
+            run.getCTR().addNewDrawing().set(DocxDrawings.drawing(shape.shape(), ids.getAsLong(), shape.order(), shape.drawn()));
         }
     }
 }