Skip to content

Integrate fourteen filesystem and format maintenance branches - #325

Merged
Hawkynt merged 180 commits into
mainfrom
integrate/cw-round1b
Sep 11, 2026
Merged

Hawkynt merged 180 commits into
mainfrom
integrate/cw-round1b

Conversation

@Hawkynt

@Hawkynt Hawkynt commented Sep 11, 2026

Copy link
Copy Markdown
Owner

Fourteen open branches merged onto current main, repaired where they collided, and verified as a set.

Closes #289
Closes #291
Closes #292
Closes #293
Closes #295
Closes #298
Closes #299
Closes #300
Closes #302
Closes #312
Closes #315
Closes #317
Closes #319
Closes #323

Verified

main is green at 28,866 passed / 0 failed, so every failure below was introduced by the branches, not inherited. After repair this branch runs the required tier at 29,039 passed, 0 failed, 1,509 skipped.

The collisions worth knowing about

The shared maintenance probe encrypted archives the verbs under test cannot open. #319 made the probe credential-aware, so every format merely advertising SupportsPassword got an encrypted probe — but Defragment(Stream) and Purge take no credentials. That broke the generic round-trips for Zip, 7z, RAR, ACE, ARJ, SQX, CB7 and CBR: fifteen failures from one helper. The probe now creates unencrypted and falls back to a password only where creation refuses without one.

Two branches each added an IArchiveModifiable overload, which made every bare <see cref="IArchiveModifiable.Add"/> and .Remove in the tree ambiguous (CS0419) — a build break neither branch could see alone.

RebuildVerb.CreateScratchStream was internal, so #296's NRG could not reach it from its own assembly. RebuildVerb is public and this is the scratch stream its own rebuild path hands out, so the helper is public now.

Plus the usual integration debris: a ReadOnlySpan captured by an Assert.Multiple lambda in five places, a stackalloc inside a loop (CA2014), a default interface member called on a concrete type, Math.Max(int, ushort) matching several overloads, an unused System.ComponentModel import making [Category] ambiguous against NUnit's, and two fixed-capacity MemoryStreams handed to verbs that grow the image.

Four branches are deliberately not in this round

They fail in ways that need a decision about the format, not a merge repair:

PR Why
#238 Office OPC parts .ppt/.docx detection: an existing descriptor already claims those extensions and wins, so the new Office views never surface. That is an ordering/priority decision.
#286 bcachefs bucket generations In-place commit leaves allocation discrepancies — "the freespace tree offers bucket 113, which the alloc tree says is in use".
#296 NRG Writes CUEX entry 1 at LBA −150 where its own fixture expects 0. One of the two has the Red Book pregap convention wrong.
#301 T64 Its own wild-image fixture reads back "entry 0 extends into the next payload or past EOF".

Each stays open with its branch intact.

Also excluded

#297, #304, #305, #306, #307, #314, #324 conflict against current main and need rebasing first. #304 additionally does not compile: its AKB remux uses _ => return false as a switch-expression arm, and it calls AudioStreamFormat.CodecPrivateData, which does not exist.

Collisions that only appear once the branches sit side by side; no behaviour
change intended.

- Two branches each added an IArchiveModifiable overload, making every bare
  <see cref="IArchiveModifiable.Add"/> and .Remove ambiguous (CS0419).
- UefiFvReader and four fixtures captured a ReadOnlySpan in a lambda.
- NrgWriter allocated inside a loop (CA2014).
- NRG reached RebuildVerb.CreateScratchStream from another assembly, where it
  was internal.
- An external-oracle fixture imported System.ComponentModel for Win32Exception,
  making [Category] ambiguous against NUnit's.
- CSO fixtures called a default interface member on the concrete descriptor.
- A Lustre fixture's Math.Max(int, ushort) matched several overloads.
- Two fixtures built fixed-capacity MemoryStreams for verbs that grow the image.

# the shared maintenance probe encrypted archives the verbs cannot open

Making the probe credential-aware meant every password-capable format got an
encrypted probe -- but Defragment(Stream) and Purge take no credentials, so
Zip, 7z, Rar, Ace, Arj, Sqx, Cb7 and Cbr all failed their round trips. The probe
is unencrypted, falling back to a password only where creation refuses without
one.

* regenerate the filesystem support matrix from the live registry
…into integrate/cw-round1b"

This reverts commit b9aefe5, reversing
changes made to 1e7aaa3.
Both fail their own tests once integrated, in ways that need a format decision
rather than a merge repair:

- NRG writes CUEX entry 1 (track 1, index 0) at LBA -150 where its own fixture
  expects 0. One of the two states the Red Book pregap convention wrongly, and
  which is a question about the format, not about the merge.
- T64 reads back "entry 0 extends into the next payload or past EOF" on the
  wild-image fixture the same branch adds.

They return to their own pull requests; the other fourteen land here.
The fourteen branches changed the public surface of Compression.Core,
FileFormats.Archives and FileFormats.FileSystems.
@Hawkynt
Hawkynt merged commit 0293423 into main Sep 11, 2026
4 checks passed
@Hawkynt
Hawkynt deleted the integrate/cw-round1b branch September 12, 2026 06:57
Hawkynt added a commit that referenced this pull request Sep 21, 2026
.github/workflows/refs-diagnostic.yml triggered only on pushes to the branch
refs-block-refcount-clone, which no longer exists: its content was absorbed into
#325 and the remote branch has since been deleted, so the workflow can never
fire again.

It was never a gate in the first place. The job cloned the PNGCrushCS sibling,
restored and built the solution with continue-on-error, uploaded the build log
and re-raised the build result - a debugging aid for one red branch. That branch
was red three times on

  error CS0104: 'Category' is an ambiguous reference between
  System.ComponentModel.CategoryAttribute and NUnit.Framework.CategoryAttribute

which #325 fixed with a using alias, leaving the workflow with nothing to
diagnose. No other workflow or script references the dead branch.
Hawkynt added a commit that referenced this pull request Sep 21, 2026
* + executing coverage for the ReFS offline block clone

RefsOfflineBlockCloner and RefsCowBlockRefcountEditor had no test that ever
ran. Their only caller was RefsBlockCloneExternalTests, which sits in the
ExternalFsInterop tier the core CI filter excludes, needs CWB_REFS_CLONE_IMAGE
and CWB_REFS_FOREFST that nothing sets, generates no fixture, and runs under
continue-on-error - so it always reached Assert.Ignore. A 263-line primitive
whose failure mode is silent cross-file extent corruption was therefore shipped
on the strength of a placeholder.

Root cause of the gap is structural, not neglect: this package deliberately has
no ReFS creator - the writer edits existing volumes only - so there was no image
to drive the cloner against, and nobody wrote one.

RefsSyntheticVolume now builds a byte-exact unmounted ReFS 3.14 image: VBR, SUPB,
two CHKP slots, Container Table, Object Table, Schema Table, Medium Allocator,
Block Refcount and a root-directory object, every tree a single leaf page so no
B+ split, separator-key convention or Upcase Table is involved. RefsImageProbe
decodes the result straight from raw bytes and shares no code with the reader,
writer or CoW engine, so the assertions are not our reader agreeing with our
writer.

Covered, running in the core tier: whole-file clone of two equal-sized
cluster-aligned extent-backed files; first-clone materialisation both of an
absent Block Refcount row and of a zero slot in an existing one; paired
publication of Object Table root #0 and Block Refcount root #6 through the
alternate checkpoint with the superseded slot left intact; allocator roots
accounting for the newly reserved CoW pages; reclamation of the destination's
former allocation only after the checkpoint is durable; an abort injected into
the checkpoint write leaving the volume byte-identical; copy-on-write isolation
when one clone is written afterwards; fragmented sources carrying every run; and
refusal of unequal sizes, non-aligned sizes, resident streams, empty streams,
repeated sharing, self-clone and unknown destinations.

Every assertion was verified to bite by perturbing the code it covers: removing
the implicit-owner baseline from AddCloneReferences, skipping the sparse-row
insertion, publishing root #0 without root #6, reclaiming before the commit, and
an off-by-one on the cloned extent each fail the tests that name them.

The cloner itself turned out to be correct on its stated profile.

* - the ReFS diagnostic workflow pinned to a deleted branch

.github/workflows/refs-diagnostic.yml triggered only on pushes to the branch
refs-block-refcount-clone, which no longer exists: its content was absorbed into
#325 and the remote branch has since been deleted, so the workflow can never
fire again.

It was never a gate in the first place. The job cloned the PNGCrushCS sibling,
restored and built the solution with continue-on-error, uploaded the build log
and re-raised the build result - a debugging aid for one red branch. That branch
was red three times on

  error CS0104: 'Category' is an ambiguous reference between
  System.ComponentModel.CategoryAttribute and NUnit.Framework.CategoryAttribute

which #325 fixed with a using alias, leaving the workflow with nothing to
diagnose. No other workflow or script references the dead branch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant