Repository navigation
Conversation
Hawkynt
force-pushed
the
feat/zpaq-real
branch
2 times, most recently
from
October 23, 2026 12:06
2b6f816 to
4823a8e
Compare
Symptom: after any merge, every open pull request went DIRTY on screenshots/main-window.png (and docs/screenshots/*). The Generate and Branch screenshots workflows re-captured the window on every push to every branch and committed the result back, so branches that only touch a codec or a format each carried their own slightly different PNG, and the first merge turned all the others into binary conflicts. Each resolution re-ran the capture, restarted CI and needed a new approval. Both workflows now run only for pushes that touch the shell (Compression.NativeUI, Compression.Shell), the NativeForms pin, or the capture scripts and workflows themselves. workflow_dispatch still refreshes any branch on demand.
ZIP entries compressed with method 95 threw NotSupportedException, so every archive 7-Zip writes with -mm=xz (and WinZip .zipx XZ entries) was unreadable. APPNOTE 4.4.5 defines the payload as one complete .xz stream, so the entry is decoded and encoded with the existing XzStream (LZMA2, CRC-64 check); the writer falls back to Store when XZ would not shrink the data, like every other method. "xz" is exposed in the method lists (descriptor schema, CLI method spec, UI). Oracle: 7-Zip 26.03 tests and extracts our archives byte for byte and reports "Method = xz"; we decode archives 7-Zip writes with -mm=xz. A 7-Zip-written reference archive is committed under Compression.Tests/Zip/ReferenceVectors so the read direction is pinned without the tool. Sourcing: rung 3 (APPNOTE.TXT); the codec is our own XZ implementation.
# XAR: bzip2, lzma and xz members decoded and checked against the TOC Symptom: lzma and xz members came back still compressed, bzip2 members threw, and any other encoding style was returned raw. Root cause: Extract only knew zlib and "none"; every lzma/xz style was folded into one name and fell through to the raw-bytes default. Change: encoding styles map one-to-one (octet-stream, x-gzip, x-bzip2, x-lzma, x-xz). bzip2 and xz go through our own stream decoders; x-lzma is sniffed, because xar's own encoder writes .xz there while libarchive writes the legacy .lzma format. Unknown styles are refused by name. Every member is checked against its declared size and its SHA-1, MD5 or SHA-256 extracted-checksum. Evidence: none/gzip/bzip2/lzma/xz archives written by bsdtar 3.7.2 (libarchive 3.7.2), committed with provenance; every member passes its SHA-1, all five decode to identical bytes, inputs match the generator's SHA-256s. XarOracleTests re-runs bsdtar live. Sourcing: rung 3 (xar format documentation) for the encoding styles; codecs are our own. * regenerate package references
…refused # LHA: -lh2- and -lh3- decoded with their own decoders, writing them refused Symptom: -lh2- and -lh3- members were decoded with the -lh5- decoder, and the writer emitted -lh5- data labelled -lh2-/-lh3-, which no real LHA reads. Root cause: both are distinct LHarc 2.x methods. -lh2- is LZSS over an 8 KiB window with two adaptive Huffman trees (the position tree grows a leaf every 64 output bytes); -lh3- sends static trees per block, with a "ready made" position table when none is sent. Neither shares -lh5-'s bit stream. Change: Lh2Decoder and Lh3Decoder (with the LHarc fillbuf/getbits bit reader) decode them; LhaWriter refuses both methods and NativeUI no longer offers them. Evidence: lh2.lzh and lh3.lzh from lhasa's test suite (ISC), written by UNLHA32.DLL 2.67a, committed with provenance. Both decode to the CRC-16 UNLHA32 recorded, to 2,853 bytes of MIT licence text, and to identical bytes as each other. Sourcing: rung 1, converted from XADMaster's XADLZHOldHandles.m (LGPL-2.1-or-later; descended from LHa for UNIX via libxad), notice in THIRD-PARTY-NOTICE.XADMaster.txt. XADMaster sizes the -lh3- position alphabet as 1 << 10; the ready-made table only satisfies Kraft's equality with 128 symbols (8 KiB / 64), which is what the real files need. * regenerate package references
…ir archivers wrote them # LHA: level-1 extended headers, level-3 headers and directory names read Symptom: on lhasa's test suite, members behind level-1 headers with extended headers came out a few bytes off (CRC mismatch or end of stream), level-3 archives were refused, directory names were dropped, and the modifier and layout map walked level-2 chains with the level-0 size byte. Root cause: a level-1 member's compressed size includes its extended headers, which the reader did not subtract; level 3 (32-bit sizes) was not implemented; extended header 0x02 (directory, 0xFF-separated) was ignored; each of the three walkers had its own partial parser. Change: one LhaHeaderParser for levels 0-3 (header checksum, the level-1 size correction, the OS-9/68k level-2 quirk, 0x01 file and 0x02 directory headers) used by the reader, the modifier and the layout map. A truncated last member is listed and fails on extraction instead of failing the whole archive. Evidence: archives from lhasa's test suite (LHA for Amiga 1.22, LHA OS/2 2.08, ...) committed with provenance; every member passes the CRC-16 its archiver recorded. Sourcing: rung 1, converted from lhasa's lha_file_header.c and ext_header.c (ISC), notice in THIRD-PARTY-NOTICE.lhasa.txt. # LHA: -lh1-, -lzs-, -lz5-, -pm1-, -pm2- decoded as their archivers wrote them Symptom: on lhasa's test suite every -lh1-, -lzs-, -lz5-, -pm1- and -pm2- member failed (CRC mismatch, end of stream, divide by zero); our writer's output for these methods was not readable by lhasa; -lhx- was refused. Root cause: the decoders implemented other schemes: -lh1- rebuilt a static tree every 4 KiB instead of LZHUF's adaptive tree; -lzs- was read byte-aligned with relative offsets instead of an MSB-first bit stream of absolute ring positions; -lz5- lacked LArc's predefined initial ring and start position; -pm1-/-pm2- were a PPMd model with no relation to PMarc. The encoders mirrored the same mistakes. Change: decoders converted from lhasa for -lh1- (shared adaptive tree), -lzs-, -lz5-, -pm1- and -pm2- (move-to-front history, PMarc's static and rebuilt trees); new -lh1-, -lzs- and -lz5- encoders built on the same tree and ring rules; writing -pm1-/-pm2- is refused (no encoder) and NativeUI stops offering them; -lhx- reads through the lh5 decoder with a 1 MiB window. Evidence: LHarc 1.13, LArc 3.33, PMarc 1.24/2 and UNLHA32 archives from lhasa's suite pass their CRC-16s, and five methods decode the same GPL-2 text identically. LhaOracleTests: every method x header level (0-2) our writer produces extracts byte-identically in lhasa (lha t / lha x). Remaining: LHARK's lk7 variant of -lh7-. Sourcing: rung 1, decoders converted from lhasa (ISC), tables copied exactly, notice in THIRD-PARTY-NOTICE.lhasa.txt; encoders are our own. * regenerate package references
# PackIt: real on-disk layout, Huffman entries decoded, encrypted ones refused
Symptom: real PackIt archives did not read: "PMa4" Huffman members came
back as compressed bytes, and stored ones were cut at the wrong offsets.
Archives we wrote were not PackIt files at all.
Root cause: reader, writer and modifier shared an invented 87-byte
header. The real header is 94 bytes after the magic (name, type,
creator, Finder flags, fork sizes, dates, CRC-16), the forks are followed
by a CRC-16, the archive ends with "PEnd", and a "PMa4" entry Huffman-
codes the header, both forks and the CRC as one stream with no stored
length.
Change: reader, writer and modifier rewritten to the real layout. PMa4
entries are decoded (StuffIt method-3 style pre-order tree, MSB-first)
and walked to find the next entry; both CRC-16s are checked; PMa5/PMa6
(XOR/DES-encrypted) entries are listed and refused. The writer emits
stored entries with both CRCs and the end marker; defrag carries each
entry over byte for byte so resource forks and Huffman coding survive.
Evidence: nine real PackIt archives (dexvert sample set; seven Huffman-
coded) decode with every header and fork CRC matching and every data
fork identical to unar's; they are not committed (unknown redistribution
terms) and run from test-corpus/packit when present. unar extracts
archives we write, and decodes a test-encoded PMa4 entry to the same
bytes we do. The header CRC (unchecked by XADMaster) is CRC-16/XMODEM
over the 92 bytes before it, measured on every real header.
Sourcing: rung 1, layout and Huffman scheme converted from XADMaster's
XADPackItParser.m and XADStuffItHuffmanHandle.m (LGPL-2.1-or-later),
notice in THIRD-PARTY-NOTICE.XADMaster.txt.
# DiskDoubler: real header, methods 0/1/6/9 decoded, the rest refused
Symptom: compressed DiskDoubler forks were returned as their compressed
bytes, without an error, and real DiskDoubler files did not parse.
Root cause: the reader and writer used an invented 82-byte header with a
name field. A real file opens with the magic 0xABCD0054 and an 84-byte
header (sizes, per-fork method, version bytes, dates, type/creator,
per-fork checksums and delta types, CRC-16), often behind a MacBinary
wrapper; the extract path had no decoder for any method but stored.
Change: reader and writer rewritten to the real layout, with header CRC
check and MacBinary detection; magic-based detection added. Decoded:
0 (stored), 1 (Unix compress with its 0x5A-masked header and byte-sum
check), 6 and 9 ("AD" LZSS blocks with per-block header XOR). Methods
2-5, 7, 8, 10 and delta pre-processing throw NotSupportedException.
The writer emits a stored single file.
Evidence: on the dexvert DiskDoubler sample set (not committed: unknown
redistribution terms; runs from test-corpus/diskdoubler when present)
every method 6/9 fork is identical to unar's output; the one method-1
sample decodes 2 bytes short of its size in unar and in gzip alike and
we refuse it the same way; the Compact Pro (8) samples are refused.
unar extracts files we write.
Sourcing: rung 1, layout and the AD decoder converted from XADMaster's
XADDiskDoublerParser.m and XADDiskDoublerADnHandle.m (LGPL-2.1-or-later),
notice in THIRD-PARTY-NOTICE.XADMaster.txt.
* regenerate package references
* FreeArc: real FreeArc 0.67 archives, read and write
Symptom: FreeArcReader/FreeArcWriter used a private layout of our own
("ArC\1" + flags + typed blocks). Not one real FreeArc archive could be
read, and FreeArc rejected everything we wrote, but the descriptor still
claimed create and modify.
Root cause: the format was never taken from FreeArc. A real archive is
read from its end. The footer descriptor sits in the last 4 KiB and
lists the control blocks. Each directory block lists its solid blocks
with offsets relative to itself and the per-file columns. Every
descriptor and control block is CRC-32 checked. Integers use FreeArc's
own variable-length form, and the directory and footer are normally
lzma-compressed.
Change: the reader and writer are rewritten from docs/FREEARC-ON-DISK.md
and the private layout is removed with no backward compatibility.
- Reading: the full container (SFX prefix, comment, directories, UTF-8
names, several solid blocks). storing and lzma (raw LZMA1, dictionary
and lc/lp/pb from the method string) decode. tornado, ppmd, lzp, grzip,
rep, delta, exe, 4x4 and encryption are listed but refused with
NotSupportedException. Archives whose directory cannot be read
(encrypted headers) list as FULL.arc + metadata.ini
(parse_status=partial).
- Writing: FreeArc's layout with a storing or lzma solid block. For
storing input in FreeArc's order it is byte-identical to
"arc a -m0 -dmstoring".
Oracle: FreeArc 0.67 (2014-03-15) Arc.exe and unarc.exe, run as black
boxes. "unarc t/x" and "arc t/lb/x" extract our storing and lzma
archives byte-identically, and we extract arc's -m0/-mlzma/-mlzma:lc4
archives byte-identically (FreeArcOracleTests). Nine archives Arc.exe
wrote are checked in as fixtures, with provenance. Create/modify stay
because FreeArc itself accepts our output.
Sourcing: rung 2. FreeArc's sources are GPL-2.0 and the 0.67 binaries
freeware, so neither can be absorbed. The sources (svn2github/freearc:
Unarc/ArcStructure.h, ArhiveStructure.hs, ArhiveDirectory.hs,
ByteStream.hs) were read to write docs/FREEARC-ON-DISK.md, every field
was confirmed against hex dumps of Arc.exe's output, and the code was
written from that document. Rung 1 does not apply: there is no
permissively licensed FreeArc implementation.
* FreeArc: regenerate the Archives API reference
The reader and writer used an invented layout: compression taken from the first-header flags, the block compression bit inverted, no script header. Real installers never parsed; the "writer" produced files only our reader accepted. Both are replaced. The reader follows NSIS's fileform.h: first header at a 512-byte boundary, compressor recognised from the data (zlib, bzip2, LZMA, stored), solid streams split on the fly, file names recovered by replaying SetOutPath/File from the script header, NSIS 2 and 3 string escapes, Unicode and ANSI, 32- and 64-bit headers, file times. Deflate gains the NSIS 2 stored-block rule (LEN without NLEN); bzip2 gains NSIS's header-less framing. Detection scans aligned first headers instead of a tail search. Creation, defrag and the fake writer are removed: nothing but makensis can produce a runnable installer. Oracle: 11 installers compiled by makensis 2.51 and 3.10 (zlib/bzip2/LZMA, solid and not, Unicode and ANSI); names, sizes and SHA-256 of every file match what 7-Zip 26.03 extracts. Sourcing: rung 1. NSIS sources (zlib/libpng) for layout and constants; 7-Zip's NSIS handler (LGPL-2.1+) for the signature table and dialect checks.
…esolver descent
+ partitioned disks: 4Kn GPT, GPT backup recovery, BSD/Amiga RDB/Sun/SGI labels
Symptom: a GPT disk with 4096-byte logical blocks was not recognised (the
header was only looked for at byte 512), a damaged primary GPT lost the
whole table, and bare BSD disklabels, Amiga RDB, Sun and SGI disks were not
recognised at all. PartitionEditor's GPT LastUsableLBA overlapped the first
backup entry sector, partitions could be added inside the GPT's own
structures, and every rewrite replaced unique partition GUIDs, attribute
bits and the disk GUID.
Root cause: GptParser, MbrParser and PartitionEditor hard-coded 512-byte
LBAs; the detector knew GPT, MBR and APM only; the editor validated
partitions against the disk length, not the usable LBA range.
Fix: the block size is the one at which LBA 1 holds a CRC-valid header
(512 or 4096); a primary header or entry array failing its CRC falls back
to the backup at AlternateLBA / the last LBA. MBR/EBR parsing takes the
block size; the EBR walk stops at loops, at links leaving the extended
partition and at EBRs without 0x55AA; a lone protective 0xEE entry is no
partition. New readers for the stand-alone BSD disklabel, Amiga RDB
(checksummed RDSK/PART chain), Sun label (magic + XOR checksum) and SGI
volume header (magic + sum checksum), probed after GPT/MBR/APM. The editor
counts in the disk's block size, refuses ranges outside
[FirstUsableLBA, LastUsableLBA] (also before ConvertMbrToGpt writes), puts
LastUsableLBA before the backup array, keeps existing entries' unique
GUIDs and attributes and the disk GUID, and can write an empty GPT/MBR.
Sourcing: rung 3 — UEFI 2.10 §5.2/§5.3 for MBR/GPT; 4.4BSD disklabel.h,
AmigaOS NDK hardblocks.h/filehandler.h, Sun dklabel.h/vtoc.h and IRIX dvh.h
for the label layouts and checksum rules. util-linux sfdisk/fdisk and GNU
parted (GPL) were used only as black-box oracles: fdisk -b 4096 and
sfdisk-written tables are read at the offsets they report, and parted
mklabel amiga/sun/dvh/bsd labels at parted's offsets. parted's BSD labels
do not satisfy the dkcksum rule (measured), so a stand-alone BSD label is
accepted on its two magics and sane geometry instead.
+ PartitionedDisk: edit partition filesystems in place, add/remove partitions, create disks
Symptom: a raw partitioned disk was read-only; files inside a partition
could not be edited, partitions not added or removed, and no disk could be
created from partition images. The partition-aware TryAdd/TryRemove the
virtual-disk descriptors use wrote straight into the partition (a refusal
half-way left a half-edited volume), accepted the generic re-create
default as an "editor", deleted the wrong entry on MBR disks with an
extended partition (detector and editor index differently), and could
place a new partition over the backup GPT.
Root cause: no write path on the descriptor; inner edits had no staging;
whole-partition deletes used list indices instead of the entry itself.
Fix: StagedWindowStream holds a partition edit's writes in 4 KiB pages,
refuses writes or SetLength past the partition, and commits only after
every edit in the call succeeded. TryAdd/TryRemove route
PartitionN_Type/path through the partition filesystem's own Add/Remove
(refused when it only has the interface's re-create default), resolve BSD
slices, plan new partitions (1 MiB-aligned, inside the usable range,
table capacity checked) before writing anything, and delete entries by
their start and size. PartitionedDiskFormatDescriptor implements
IArchiveModifiable, IArchiveCreatable (partition images on a fresh GPT or
MBR, scheme/sectorSize options, loose files refused) and
IArchiveWriteConstraints; purge is not offered. Every refusal leaves the
image byte-identical. The shell evaluates drops against the folder being
browsed, so a file dropped into a partition folder is accepted.
Sourcing: rung 3 — UEFI 2.10 §5.2/§5.3 for the table layouts; the inner
edits are the existing filesystem editors. Verified against GPL tools as
oracles only: sgdisk -v ("No problems found"), sfdisk --verify, parted and
fdisk -b 4096 read the created disks at our offsets; fsck.vfat -n and
e2fsck -fn are clean after in-place adds/removes on an sfdisk-built GPT
disk, mcopy and debugfs read the edits back, and no byte outside the
partitions changes.
# NestedStreamResolver descends into raw partitioned disks
Symptom: ResolveDeepest on a raw MBR/GPT disk image (not wrapped in a
VHD/VMDK/...) found no filesystem, and filesystems whose superblock lies
past 4 KiB were not recognised at any level.
Root cause: partition tables were only tried below a virtual-disk
container, and format detection matched magic bytes in a fixed 4 KiB
header.
Fix: detection goes through FormatDetector.DetectByContent (the disk-image
probe plus signatures as deep as the deepest registered one), after the
fixed-VHD footer check; a PartitionedDisk result descends into its
partitions, BSD slices included.
Sourcing: no format work; reuses the content probe and partition readers.
Symptom: `cwb partition add --start/--length` and the shell's Add Partition dialog multiplied LBAs by 512, so on a 4Kn GPT disk a partition asked for at LBA 256 landed at byte 131072 instead of 1 MiB, and the dialog suggested start sector 2048 (8 MiB) and described 512-byte sectors. Root cause: both used the PartitionEditor.SectorSize constant instead of the block size the disk's GPT validated at. Fix: PartitionOperations.List reports the disk's logical sector size and the new AddBySectors converts LBAs with it; the CLI uses it and says which sector size applied. The dialog takes the editor's LogicalSectorSize: the default start is 1 MiB in that size (2048 or 256) and the hints name it. Sourcing: no format work; UEFI 2.10 §5.3 counts every LBA in the logical block size. Tests: AddBySectors on 512- and 4096-byte GPT disks, the last-usable-LBA boundary on a 4Kn disk (refusal leaves it byte-identical), negative/empty ranges.
… back-references * CvfLzCodec (DS-0-x, JM-0-x, SQ-0-0 and the Stacker dispatch) is replaced by an independent implementation (MsdblLzCodec + ClusterMatchFinder) written from docs/CVF-CODECS-ON-DISK.md and the dmsdos reference vectors. Public API unchanged; CvfLzMethod moves to its own file. Clean-room reimplementation from docs/CVF-CODECS-ON-DISK.md; the implementer did not read the previous code or dmsdos sources (rung 3/4 spec + black-box oracle). * The DS/JM encoder now writes a sync after every 512 decoded bytes and never lets a match cross a 512-byte boundary - measured in all 16 dmsdos DS/JM reference streams and recorded in the spec. A sync followed only by zero padding ends a short cluster (tail reads zero). * Auto in the codec keeps the smaller of DS and JM; GenuineDvr3Writer's Auto also tries SQ. + Sd4Codec rewritten from the spec: the encoder emits short/long matches with every length extension and run-coded code-length tables, codes built by package-merge (optimal, complete, 15-bit limit). Within 10% of the dmsdos SD-4 encoder on the five reference samples; dmsdos reads our SD-4 volumes (and 15-bit-deep main codes) byte-exact. + BB_StackerSd4 building block. # Stacker: a compressed run that needs every sector of its cluster reads as stored (AMAP has no flag for it), so ForceCompress wrote clusters dmsdos and our reader returned verbatim. Such a cluster is now stored raw; verified by dmsdos and our reader. + DmsdosOracle: the dmsdos gates run through WSL on Windows as well as natively on Linux; the existing Stacker/DriveSpace 3/DoubleSpace driver-proof tests use it. + Boundary tests for every CVF codec: empty/1-byte clusters, 512/8K/32K clusters, incompressible data, maximum match lengths and distances, malformed streams. ! Fragmented DriveSpace 3 clusters and fragmented/suballocated Stacker clusters are still refused; they follow in a separate change.
The reader listed nothing real and threw on every extraction; the writer emitted a bare
version string. Both are replaced.
The reader locates the setup loader offset table (0x30 or RCDATA 11111, 32- and 64-bit tables),
parses setup-0 for every known layout (ambiguous tags retried as innoextract does), decodes
setup-1 chunks (stored, zlib, bzip2, LZMA, LZMA2; embedded or setup-N.bin slices), undoes the
x86 call filters, names files the way innoextract does ({app}\x -> app/x, collision rules) and
checks every file against the checksum the compiler stored. Encrypted files and the 16-bit
1.2.x format are refused. Creation is removed.
LZMA2: an LZMA chunk without reset after an uncompressed 0x02 chunk continues the coder state
(xz spec, liblzma, LZMA SDK); our decoder wrongly demanded a reset, which real Inno installers
tripped over. 0x01 now also invalidates properties and state, as those decoders do.
Oracle: installers compiled by ISCC 5.6.1 (ANSI and Unicode) and 6.0.5 with every compressor,
and the official 5.6.1/6.0.5 installers, match innoextract 1.9 file for file; the official 6.4.3
and 7.1.0 installers (beyond any innoextract) extract with every file matching its stored
SHA-256.
Sourcing: rung 1. innoextract (zlib licence) converted for 1.3 to 6.4.0.1; Inno Setup's own
Shared.Struct.pas / Setup.MainFunc.pas / Compression.Base.pas (Inno Setup License) for the
layouts 6.4.2 to 7.0.0.3. Notices beside the code.
Symptom: Zpaq claimed create and modify, but its writer emitted a private
layout ("zPQ" + level + 'c'/'d'/'h' with raw FILETIME/name/size records and
the data verbatim) that zpaq rejects, and its reader only understood that
layout: every archive written by zpaq itself (any method) failed to extract.
Root cause: there was no ZPAQL engine at all. The old ZpaqlVm/ZpaqRangeCoder
were a homebrew model that does not follow the Level 2 specification, so
neither the block format nor the journaling layer on top of it could be real.
Fix: convert libzpaq/zpaq 7.15 (Matt Mahoney, public domain) into the
FileFormat.Zpaq namespace:
- ZPAQL interpreter, predictor (CONS/CM/ICM/MATCH/AVG/MIX2/MIX/ISSE/SSE),
arithmetic coder, PCOMP post-processor, block/segment decompresser,
ZPAQL compiler and compressor;
- method strings 0..5 and explicit x methods (makeConfig/compressBlock),
the LZ77 (packed and byte-aligned, hash and suffix-array match finders),
BWT and E8E9 preprocessors; suffix arrays come from an own SA-IS
implementation instead of the MIT libdivsufsort section;
- ZpaqReader: journaling archives (c/d/h/i blocks, versions, deletions,
deduplicated fragments with SHA-1 verification) and the streaming layout;
encrypted archives are refused;
- ZpaqWriter: one journaling transaction exactly as Jidac::add lays it out
(content-defined fragments, dedup, block packing, per-block method choice).
The private layout is removed without backward compatibility.
BB_Zpaq now emits a real ZPAQ stream (method 4) and decodes any ZPAQ stream.
Verification: zpaq 7.15 lists and extracts byte for byte what we write at
methods 0-5, 14, 36 and explicit x methods; we read zpaq's archives at every
method, with three versions (update + deletion), dedup and the streaming
layout; and with the same input our d blocks equal zpaq 7.15's byte for byte
(transaction date masked). Reference archives written by zpaq64.exe 7.15 are
embedded with a SHA-256 manifest; the live checks become a hard CI gate on
the Linux leg (apt zpaq).
Sourcing: rung 1 (public-domain reference implementation converted, notice
in THIRD-PARTY-NOTICE.libzpaq.txt); tables and generated ZPAQL copied exactly.
Hawkynt
force-pushed
the
feat/zpaq-real
branch
from
October 26, 2026 12:10
16fefa5 to
3b92dff
Compare
The required tier failed because the existing XADMaster conversions for LHA lh2/lh3 and the ARJ prerequisite were absent from the provenance ledger. Record the pinned XADMaster revision, LGPL-2.1-or-later grant and existing notice, and retain the current generated screenshot. Verified: Release solution build; complete Linux and Windows core runs had only the ledger failure; all 72 LHA and provenance tests now pass. Existing implementation sourcing is rung 1.
The legacy-LHA/header PR failed only the provenance check: its ISC-licensed lhasa conversions were missing from the ledger. Record the pinned upstream, all converted decoder/header files and both existing notices, and merge the validated prerequisites. Verified: Release solution build; complete Linux and Windows core runs had only the ledger failure; all 113 LHA and provenance tests pass. Existing implementation sourcing is rung 1.
Required CI failed only because the existing XADMaster conversion was absent from the provenance ledger. Record the pinned source, LGPL-2.1-or-later grant and existing notice, and merge the validated prerequisite updates. Verified: Release solution build; complete Linux and Windows core runs had only that ledger failure; 83 PackIt, DiskDoubler and provenance tests passed, 2 corpus-dependent cases skipped. Existing implementation sourcing is rung 1.
The FreeArc branch failed the required tier only because its prerequisites lacked provenance ledger entries. Merge the validated corrections and retain the current generated screenshots. Verified: Release solution build; complete Linux and Windows core runs had only the provenance failure; 67 FreeArc and provenance tests passed, 9 external-tool cases skipped.
The NSIS branch failed its required tier only because converted-code provenance was missing from its prerequisites. Merge the validated corrections and retain the current generated screenshot. Verified: Release solution build; complete Linux and Windows core runs had only the provenance failure; all 53 NSIS and provenance tests pass.
Required CI failed only because prerequisite conversions lacked provenance ledger entries. Merge the validated corrections and retain the current generated screenshots. Verified: Release solution build; complete Linux and Windows core runs had only the provenance failure; 252 partition and provenance tests passed, 10 tool-gated or format-limited cases skipped.
The sector-size fix failed required CI only because its prerequisite conversions lacked provenance ledger entries. Merge the validated corrections and retain the current generated screenshot. Verified: Release solution build; complete Linux and Windows core runs had only the provenance failure; 257 partition and provenance tests passed, 10 tool-gated or format-limited cases skipped.
Required CI failed because prerequisite conversions lacked provenance ledger entries. The ledger also still described the removed dmsdos-derived bit reader as awaiting a rewrite, although this branch replaces it from the measured format document and black-box reference vectors. Merge the validated prerequisite corrections, record the independent CVF implementation under class R, and retain the current generated screenshots. Verified: Release solution build; complete Linux and Windows core runs had only the provenance failure; 1,043 CVF, compressed-volume and provenance tests passed, 11 tool-gated or format-limited cases skipped.
Required CI failed only because the innoextract conversion and prerequisite conversions were missing from the provenance ledger. Record the existing source, zlib licence and notice, and merge the validated prerequisite updates. Verified: Release solution build; complete Linux and Windows core runs had only the provenance failure; 117 Inno Setup, detection and provenance tests passed, 2 local-corpus cases skipped. Existing conversion sourcing is rung 1.
Required CI failed only because the existing libzpaq conversion and prerequisite conversions were missing from the provenance ledger. Record libzpaq 7.15, its public-domain Unlicense grant and existing notice; merge the validated prerequisite updates without replacing the rebuilt train history. Verified: Release solution build; complete Linux and Windows core runs had only the provenance failure; 178 ZPAQ and provenance tests passed, 31 missing-oracle cases skipped. The standalone ZPAQ integration also passed the full local required tier (34,061 passed, 1,551 skipped). Existing conversion sourcing is rung 1.
Preserve the accumulated history while restoring current main and applying only ZPAQ virtual-machine, journaling and method support with genuine reference archives and its hard-oracle CI step. Retain Zoo LZH documentation, record only the ZPAQ source notice, remove the now-resolved capability warning, and regenerate API documentation and catalogue totals. Verified: Release solution build, 191 focused format/documentation/provenance tests passed with 31 external-tool skips, documentation generation completed without errors. The earlier standalone integration passed the full required local suite: 34,061 tests, 1,551 skipped, zero failures.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Zpaqclaimed create and modify, but its writer produced a private layout that zpaq rejects, and its reader understood only that layout. No archive written by zpaq could be extracted. (Reference-catalogue audit,docs/catalogue/GAPS.md.)ZPAQ is now the real format, converted from libzpaq/zpaq 7.15 (Matt Mahoney, public domain; rung 1, notice in
THIRD-PARTY-NOTICE.libzpaq.txt):0-5, explicitxmethods, and the LZ77/BWT/E8E9 preprocessors. Suffix arrays come from our own SA-IS rather than the MIT divsufsort section.List()falls back toFULL.zpaq+metadata.iniand does not throw.zpaq adddoes it: content-defined fragments, dedup, block packing, a method chosen per block.ZpaqlVm/ZpaqRangeCoderare deleted. Old output stays unreadable on purpose; nothing is kept for backward compatibility.BB_Zpaqnow emits a real ZPAQ stream (method 4) and decodes any ZPAQ stream.reference-formats.csvandGAPS.mdare updated. The row stays PARTIAL because encrypted, multi-part and-indexarchives are refused, and modify rewrites one version instead of appending one.How it was verified
ZpaqExternalOracleTests(live, zpaq 7.15zpaq64.exe, all 31 cases pass locally):l/xrestores byte for byte what we write at methods 0-5,14,36and two explicitxmethods.smethods.BB_Zpaqstream.ZpaqReferenceVectorTests: archives written by zpaq64.exe 7.15 (every method, multi-version, streaming), embedded with a SHA-256 manifest and provenance README.ZpaqEngineTests/ZpaqTests: SA-IS against a naive sort, libzpaq's squash/stretch self-test checksums, compiler output equal to libzpaq's built-in min.cfg bytecode, every transform and model, fragment and block boundaries, dedup, corruption detection.apt-get install zpaqplus a hard ZPAQ oracle step on ubuntu-latest (CWB_REQUIRE_ZPAQ=1).