diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index ca1218294..53444bf50 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -105,15 +105,27 @@ Candidate formats from earlier stages are validated by parsing their headers and |---|----------|--------| | 1 | A signature of ≥ 0.9 confidence and ≥ 4 bytes inside sector 0 | That format (VHDX, QCOW2, VMDK, XFS, SquashFS, gzip, …) | | 2 | ISO 9660 / UDF volume descriptors at 32 KiB | `Iso` / `Udf` — ahead of any partition table, so a hybrid ISO whose system area carries an MBR or GPT browses as the ISO | -| 3 | GPT header at LBA 1 whose CRC32 matches (UEFI 2.10 §5.3.2) | `PartitionedDisk` | +| 3 | GPT header at LBA 1 whose CRC32 matches (UEFI 2.10 §5.3.2), at 512- or 4096-byte logical blocks | `PartitionedDisk` | | 4 | Sector 0 ends in 0x55AA: validated MBR table vs. volume boot record (FAT BPB, NTFS/exFAT OEM ID) | Table only → `PartitionedDisk`; boot record only → that filesystem; both → `PartitionedDisk` only if a partition holds a recognisable filesystem | | 5 | Apple Partition Map | `PartitionedDisk` | -| 6 | A filesystem superblock signature of ≥ 0.75 confidence, up to the deepest one registered (ZFS at 128 KiB) | That filesystem | +| 6 | A checksummed SGI volume header, Sun label or Amiga Rigid Disk Block, or a stand-alone BSD disklabel (both magics, sane geometry) | `PartitionedDisk` | +| 7 | A filesystem superblock signature of ≥ 0.75 confidence, up to the deepest one registered (ZFS at 128 KiB) | That filesystem | -A valid MBR (UEFI 2.10 §5.2.1) has a boot indicator of 0x00 or 0x80 in all four entries, and each used entry starts at LBA ≥ 1 inside the image, is at least one sector long and does not overlap another used entry. A trailing 0x55AA alone does not make a partition table: every FAT, NTFS and exFAT boot sector has one. Bytes that prove nothing stay `Unknown`, so noise named `.img` is no longer handed to the FAT reader. A `.img` path that does not exist yet, or is empty, still resolves to FAT as the create target. The probe reads 40 KiB, and reads further, out to the deepest filesystem signature, only for stage 6. It never scans the image. +A valid MBR (UEFI 2.10 §5.2.1) has a boot indicator of 0x00 or 0x80 in all four entries, and each used entry starts at LBA ≥ 1 inside the image, is at least one sector long and does not overlap another used entry. A trailing 0x55AA alone does not make a partition table: every FAT, NTFS and exFAT boot sector has one. Bytes that prove nothing stay `Unknown`, so noise named `.img` is no longer handed to the FAT reader. A `.img` path that does not exist yet, or is empty, still resolves to FAT as the create target. The probe reads 40 KiB, and reads further, out to the deepest filesystem signature, only for stage 7. It never scans the image. `PartitionedDisk` lists each partition as a `PartitionN_Type/` directory holding the filesystem detected inside it (`InnerFsDetector`, which also reads past 4 KiB to find ISO 9660, UDF, JFS, btrfs and ZFS). A partition it does not recognise becomes a single `PartitionN_Type.raw` entry. Partitions are read through a `PartitionWindowStream` and never copied out. `OpenEntry` hands back the window itself, or the inner filesystem's entry stream over it. Virtual-disk containers (VHD, VHDX, VMDK, QCOW2, VDI, Bochs) are detected by their own signatures and run the same partition-aware listing over their guest disk. +Tables: GPT at 512- or 4096-byte logical blocks (the block size is the one at which LBA 1 holds a CRC-valid header; a damaged primary falls back to the backup header), MBR with its EBR chain of logical partitions, APM, a BSD disklabel inside a BSD slice or standing alone, Amiga RDB, Sun and SGI labels. Writing: + +| Operation | How | Refused when | +|-----------|-----|--------------| +| Add/remove `PartitionN_Type/path` | The partition's own filesystem editor runs in place on a `StagedWindowStream`: writes are staged and reach the disk only when every edit in the call succeeded | No recognised filesystem; its editor is only the generic re-create default; the filesystem refuses (full, unsupported case); it would grow past the partition | +| Add a root-level filesystem or `*.raw`/`*.img` image | New GPT/MBR entry after the last partition, 1 MiB-aligned where room allows, inside the usable LBA range | No free space; table full; not a partition image | +| Remove `PartitionN_Type` | `PartitionEditor.DeletePartition` (GPT, MBR, logical); the sectors stay | BSD/RDB/Sun/SGI/APM tables | +| `Create` | Partition images on a fresh GPT (`scheme=gpt`, default) or MBR, `sectorSize=512`/`4096`, partitions 1 MiB-aligned | A loose file; more than 128 (GPT) / 4 (MBR) images | + +Every refusal leaves the image byte-identical. `sgdisk -v`, `sfdisk --verify`, `parted`, `fdisk -b 4096`, `fsck.vfat` and `e2fsck` check the results (`PartitionedDiskExternalTests`). Resizing a partition is not an archive operation; the partition editor window does it through `PartitionEditor.ResizePartition` and the registered filesystem resizers. + `FormatDetector.DetectByContent(Stream)` runs the same probe over a seekable stream, such as an entry inside a container, without a file on disk. `FormatDetector.DetectCached(path)` is `Detect` with a cache keyed on path, length and last-write time, for UI state that is re-evaluated on every repaint. --- diff --git a/Compression.Core/DiskImage/AmigaRdbParser.cs b/Compression.Core/DiskImage/AmigaRdbParser.cs new file mode 100644 index 000000000..049a15207 --- /dev/null +++ b/Compression.Core/DiskImage/AmigaRdbParser.cs @@ -0,0 +1,153 @@ +using System.Buffers.Binary; +using System.Text; + +namespace Compression.Core.DiskImage; + +/// +/// Parses the Amiga Rigid Disk Block (RDB) partition scheme. Big-endian throughout. +/// +/// +/// +/// The RDSK block may sit in any of the first 16 blocks of the disk +/// (RDB_LOCATION_LIMIT). It links, by block number, a chain of PART blocks; each +/// carries the partition's drive name (a BCPL string) and a DosEnvec whose geometry — +/// low/high cylinder, surfaces, blocks per track, block size in longwords — places the +/// partition on the disk, plus the DosType naming its filesystem (DOS\1 = FFS, +/// PFS\3, LNX\0, …). Every block carries a checksum: the 32-bit sum of its first +/// SummedLongs longwords is zero. +/// +/// +/// A partition's byte range is LowCyl × Surfaces × BlocksPerTrack × SizeBlock × 4 for the +/// start and (HighCyl − LowCyl + 1) × Surfaces × BlocksPerTrack × SizeBlock × 4 for the +/// length. +/// +/// References: +/// +/// AmigaOS NDK, devices/hardblocks.h — struct RigidDiskBlock, struct PartitionBlock, RDB_LOCATION_LIMIT. +/// AmigaOS NDK, dos/filehandler.h — struct DosEnvec field order. +/// +/// +public static class AmigaRdbParser { + + private const uint RdskId = 0x5244534B; // "RDSK" + private const uint PartId = 0x50415254; // "PART" + private const uint EndOfList = 0xFFFFFFFF; + private const int SearchBlockSize = 512; + private const int LocationLimit = 16; + private const int MaxPartitions = 128; + + // RigidDiskBlock field offsets. + private const int RdbSummedLongs = 4; + private const int RdbBlockBytes = 16; + private const int RdbPartitionList = 28; + + // PartitionBlock field offsets. + private const int PbNext = 16; + private const int PbDriveName = 36; + private const int PbEnvironment = 128; + + // DosEnvec longword indices. + private const int DeTableSize = 0; + private const int DeSizeBlock = 1; + private const int DeSurfaces = 3; + private const int DeBlocksPerTrack = 5; + private const int DeLowCyl = 9; + private const int DeHighCyl = 10; + private const int DeDosType = 16; + + /// Reports whether a checksummed RDSK block sits in the first 16 blocks. + /// A seekable stream over the whole disk. + public static bool IsRdb(Stream disk) => FindRdb(disk) >= 0; + + /// Parses every partition the RDB's PART chain describes. + /// A seekable stream over the whole disk. + /// No valid RDB is present. + public static List Parse(Stream disk) { + var rdbBlock = FindRdb(disk); + if (rdbBlock < 0) throw new InvalidDataException("No Amiga Rigid Disk Block (RDSK) in the first 16 blocks."); + + var rdb = ReadBlock(disk, (long)rdbBlock * SearchBlockSize, SearchBlockSize); + var blockBytes = (int)BinaryPrimitives.ReadUInt32BigEndian(rdb.AsSpan(RdbBlockBytes)); + if (blockBytes is < 256 or > 65536 || (blockBytes & (blockBytes - 1)) != 0) blockBytes = SearchBlockSize; + + var result = new List(); + var seen = new HashSet(); + var next = BinaryPrimitives.ReadUInt32BigEndian(rdb.AsSpan(RdbPartitionList)); + while (next != EndOfList && result.Count < MaxPartitions && seen.Add(next)) { + var offset = (long)next * blockBytes; + if (offset + blockBytes > disk.Length) break; + var block = ReadBlock(disk, offset, blockBytes); + if (BinaryPrimitives.ReadUInt32BigEndian(block) != PartId || !ChecksumOk(block)) break; + + uint Env(int i) => BinaryPrimitives.ReadUInt32BigEndian(block.AsSpan(PbEnvironment + i * 4)); + var tableSize = Env(DeTableSize); + if (tableSize >= DeHighCyl) { + var bytesPerBlock = (long)Env(DeSizeBlock) * 4; + var cylinderBytes = (long)Env(DeSurfaces) * Env(DeBlocksPerTrack) * bytesPerBlock; + var low = Env(DeLowCyl); + var high = Env(DeHighCyl); + if (cylinderBytes > 0 && high >= low) { + var dosType = tableSize >= DeDosType ? Env(DeDosType) : 0; + result.Add(new PartitionEntry { + Index = result.Count, + StartOffset = low * cylinderBytes, + Size = (high - low + 1L) * cylinderBytes, + TypeName = DosTypeName(dosType), + TypeCode = $"0x{dosType:X8}", + Name = DriveName(block.AsSpan(PbDriveName, 32)), + Source = "RDB" + }); + } + } + + next = BinaryPrimitives.ReadUInt32BigEndian(block.AsSpan(PbNext)); + } + return result; + } + + private static int FindRdb(Stream disk) { + if (!disk.CanSeek) return -1; + for (var i = 0; i < LocationLimit; ++i) { + var offset = (long)i * SearchBlockSize; + if (offset + SearchBlockSize > disk.Length) break; + var block = ReadBlock(disk, offset, SearchBlockSize); + if (BinaryPrimitives.ReadUInt32BigEndian(block) == RdskId && ChecksumOk(block)) + return i; + } + return -1; + } + + /// The block's first SummedLongs longwords sum to zero (mod 2^32). + private static bool ChecksumOk(ReadOnlySpan block) { + var summed = BinaryPrimitives.ReadUInt32BigEndian(block[RdbSummedLongs..]); + if (summed < 4 || summed * 4L > block.Length) return false; + uint sum = 0; + for (var i = 0; i < summed; ++i) + sum += BinaryPrimitives.ReadUInt32BigEndian(block[(i * 4)..]); + return sum == 0; + } + + /// A BCPL string: a length byte followed by that many characters. + private static string DriveName(ReadOnlySpan bstr) { + var length = Math.Min(bstr[0], (byte)31); + return Encoding.Latin1.GetString(bstr.Slice(1, length)); + } + + /// Renders a DosType as its four characters, the last as a digit (DOS\1 → DOS1). + private static string DosTypeName(uint dosType) { + if (dosType == 0) return "Amiga"; + Span chars = stackalloc char[4]; + for (var i = 0; i < 4; ++i) { + var b = (byte)(dosType >> (24 - i * 8)); + chars[i] = b is >= 0x20 and < 0x7F ? (char)b : (char)('0' + b % 10); + } + return new string(chars); + } + + private static byte[] ReadBlock(Stream disk, long offset, int size) { + var buffer = new byte[size]; + disk.Position = offset; + _ = disk.ReadAtLeast(buffer, size, throwOnEndOfStream: false); + return buffer; + } +} diff --git a/Compression.Core/DiskImage/BsdDisklabelParser.cs b/Compression.Core/DiskImage/BsdDisklabelParser.cs index 87e94fd23..50e573697 100644 --- a/Compression.Core/DiskImage/BsdDisklabelParser.cs +++ b/Compression.Core/DiskImage/BsdDisklabelParser.cs @@ -61,6 +61,43 @@ public static class BsdDisklabelParser { /// true if a disklabel is present. public static bool IsDisklabel(Stream slice) => FindLabelOffset(slice) >= 0; + /// + /// Reports whether carries a disklabel that stands on its own — no MBR + /// slice around it, as a "dangerously dedicated" FreeBSD disk or a NetBSD/OpenBSD disk on a + /// non-PC port writes it. With nothing else vouching for the bytes the label must prove + /// itself structurally: d_magic and d_magic2 both present 132 bytes apart, a + /// sector size that is a power of two in [512, 8192], a partition count within + /// MAXPARTITIONS, and every used partition starting inside the disk. + /// + /// + /// d_checksum is deliberately not required: the labels GNU parted 3.x writes + /// (mklabel bsd) do not satisfy the 4.4BSD dkcksum XOR rule, and neither the + /// Linux kernel nor libblkid checks it — the two magics are what both rely on. + /// + /// A seekable stream over the whole disk. + public static bool IsStandaloneDisklabel(Stream disk) { + var labelOffset = FindLabelOffset(disk); + if (labelOffset < 0) return false; + var label = new byte[PartitionsOffset + MaxPartitions * PartitionRecordSize]; + disk.Position = labelOffset; + var read = disk.ReadAtLeast(label, label.Length, throwOnEndOfStream: false); + if (read < PartitionsOffset) return false; + var span = label.AsSpan(0, read); + if (ReadU32(span, Magic2Offset) != DiskMagic) return false; + var secSize = ReadU32(span, 40); + if (secSize is < 512 or > 8192 || (secSize & (secSize - 1)) != 0) return false; + var npartitions = BinaryPrimitives.ReadUInt16LittleEndian(span[NPartitionsOffset..]); + if (npartitions is 0 or > MaxPartitions) return false; + if (PartitionsOffset + npartitions * PartitionRecordSize > span.Length) return false; + var diskSectors = (ulong)disk.Length / secSize; + for (var i = 0; i < npartitions; ++i) { + var size = ReadU32(span, PartitionsOffset + i * PartitionRecordSize); + var offset = ReadU32(span, PartitionsOffset + i * PartitionRecordSize + 4); + if (size != 0 && offset >= diskSectors) return false; + } + return true; + } + /// /// Parses the BSD disklabel contained in and returns /// its filesystem partitions. diff --git a/Compression.Core/DiskImage/GptParser.cs b/Compression.Core/DiskImage/GptParser.cs index 9ebfb2180..de13f9df4 100644 --- a/Compression.Core/DiskImage/GptParser.cs +++ b/Compression.Core/DiskImage/GptParser.cs @@ -5,18 +5,34 @@ namespace Compression.Core.DiskImage; /// /// Parses GPT (GUID Partition Table) headers and partition entries. -/// GPT header is at LBA 1 (offset 512 for 512-byte sectors), with partition entries starting at LBA 2+. /// +/// +/// +/// The header lives at LBA 1, so its byte offset depends on the disk's logical block size: +/// 512 on a classic or 512e drive, 4096 on a 4Kn drive. Neither the protective MBR nor the +/// header records the block size, so it is found the way the UEFI specification implies: the +/// block size is the one for which LBA 1 holds a header whose CRC32 matches and that names +/// itself as LBA 1 (UEFI 2.10, §5.3.2). Every LBA in the header and the entry array is then +/// counted in that block size. +/// +/// +/// When the primary header or its entry array fails its CRC, the backup header at the last +/// LBA is used instead, as §5.3.2 directs. +/// +/// public static class GptParser { /// GPT signature: "EFI PART". private static ReadOnlySpan Signature => "EFI PART"u8; - /// Standard sector size. - private const int SectorSize = 512; + /// Logical block sizes probed for the header at LBA 1, in probe order. + public static IReadOnlyList SupportedSectorSizes { get; } = [512, 4096]; - /// GPT header offset (LBA 1). - private const int HeaderOffset = SectorSize; + /// Classic sector size, used when nothing else is known. + private const int DefaultSectorSize = 512; + + /// Bytes of the image needed to see LBA 1 at every supported block size. + public const int ProbeBytes = 4096 * 2; /// Minimum GPT header size. private const int MinHeaderSize = 92; @@ -24,28 +40,159 @@ public static class GptParser { /// Size of each GPT partition entry (minimum). private const int MinEntrySize = 128; + /// Upper bound on the entry array the parser will read (UEFI allows any count; real tables hold 128). + private const long MaxEntryArrayBytes = 16L * 1024 * 1024; + + /// Layout facts read from the header that was used to parse the table. + /// Logical block size every LBA is counted in. + /// The LBA the used header sits at (1 for the primary). + /// The LBA of the other header. + /// First LBA a partition may occupy. + /// Last LBA a partition may occupy. + /// The disk GUID. + /// Starting LBA of the entry array the used header points at. + /// Number of entries in the array. + /// Size of one entry in bytes. + /// true when the primary was unusable and the backup was read. + public sealed record HeaderInfo( + int SectorSize, ulong MyLba, ulong AlternateLba, ulong FirstUsableLba, ulong LastUsableLba, + Guid DiskGuid, ulong EntriesLba, uint EntryCount, uint EntrySize, bool FromBackup); + /// - /// Checks whether the given data contains a valid GPT header at LBA 1. + /// Checks whether LBA 1 carries the "EFI PART" signature at any supported block size. /// - /// At least 1024 bytes of disk image data starting at LBA 0. - /// true if a valid GPT signature is found at LBA 1. - public static bool IsGpt(ReadOnlySpan data) - => data.Length >= HeaderOffset + MinHeaderSize - && data.Slice(HeaderOffset, 8).SequenceEqual(Signature); + /// The start of the image (at least 1024 bytes for 512-byte blocks, 8 KiB for 4 KiB blocks). + public static bool IsGpt(ReadOnlySpan data) { + foreach (var size in SupportedSectorSizes) + if (data.Length >= size + MinHeaderSize && data.Slice(size, 8).SequenceEqual(Signature)) + return true; + return false; + } /// /// Checks that LBA 1 holds a GPT header that is internally consistent, not merely the - /// "EFI PART" signature: the header size lies in [92, 512], the header's own CRC32 matches - /// (computed with the CRC field zeroed, UEFI 2.10 §5.3.2), it names itself as LBA 1 and its - /// partition entries are 128 x 2^n bytes each, as the specification requires. + /// "EFI PART" signature, at any supported block size: see . + /// + /// The start of the image. + public static bool HasValidHeader(ReadOnlySpan data) => DetectSectorSize(data) > 0; + + /// + /// The logical block size for which LBA 1 holds a valid GPT header, or 0 when none does. /// - /// At least the first 1024 bytes of the image. - public static bool HasValidHeader(ReadOnlySpan data) { - if (!IsGpt(data)) return false; + /// The start of the image; 8 KiB covers every supported block size. + public static int DetectSectorSize(ReadOnlySpan data) { + foreach (var size in SupportedSectorSizes) + if (HasValidHeader(data, size)) + return size; + return 0; + } + + /// + /// Checks that LBA 1, counted in -byte blocks, holds a GPT header + /// that is internally consistent: the header size lies in [92, block size], the header's own + /// CRC32 matches (computed with the CRC field zeroed, UEFI 2.10 §5.3.2), it names itself as + /// LBA 1 and its partition entries are 128 x 2^n bytes each, as the specification requires. + /// + public static bool HasValidHeader(ReadOnlySpan data, int sectorSize) { + if (data.Length < sectorSize + MinHeaderSize) return false; + var header = data.Slice(sectorSize, Math.Min(sectorSize, data.Length - sectorSize)); + return IsValidHeaderBlock(header, sectorSize, expectedMyLba: 1); + } + + /// + /// The partition table's layout facts, or null when the image carries no usable GPT. + /// Uses the same header would. + /// + public static HeaderInfo? ReadHeader(Stream diskData) { + try { + return Locate(diskData, lenient: true)?.Info; + } catch (Exception ex) when (ex is IOException or InvalidDataException or ArgumentException) { + return null; + } + } + + /// + /// Parses all partitions from a GPT disk image, at whichever block size its header validates. + /// + /// The full disk image as a readable stream. + /// A list of partition entries. + public static List Parse(Stream diskData) => Parse(diskData, out _); + + /// + /// Parses all partitions and reports the header that described them. + /// + /// No GPT header at LBA 1 at any supported block size. + public static List Parse(Stream diskData, out HeaderInfo info) { + var located = Locate(diskData, lenient: true) + ?? throw new InvalidDataException("Invalid GPT: missing 'EFI PART' signature at LBA 1."); + info = located.Info; + return ParseEntries(located.Entries, info); + } + + private sealed record Located(HeaderInfo Info, byte[] Entries); - var header = data.Slice(HeaderOffset, Math.Min(SectorSize, data.Length - HeaderOffset)); + /// + /// Finds the header to trust: a primary whose header and entry-array CRCs both match; else + /// the backup it (or the end of the disk) points at, with the same checks; else, when + /// , a primary that carries only the signature — the reading older + /// images written without CRCs rely on. + /// + private static Located? Locate(Stream disk, bool lenient) { + var head = ReadAt(disk, 0, (int)Math.Min(ProbeBytes, disk.Length)); + foreach (var size in SupportedSectorSizes) { + if (!HasValidHeader(head, size)) continue; + var header = head.AsSpan(size, Math.Min(size, head.Length - size)); + if (TryReadEntries(disk, header, size, out var entries)) + return new Located(ToInfo(header, size, fromBackup: false), entries); + + // Entry array damaged: the backup describes the same table. + if (TryLocateBackup(disk, size, BinaryPrimitives.ReadUInt64LittleEndian(header[32..])) is { } backup) + return backup; + } + + // No primary validates: look for a backup at the end of the disk. + foreach (var size in SupportedSectorSizes) { + if (!IsGpt(head) && !HasProtectiveEntry(head)) break; + if (disk.Length < 2L * size) continue; + if (TryLocateBackup(disk, size, (ulong)(disk.Length / size - 1)) is { } backup) + return backup; + } + + if (!lenient) return null; + foreach (var size in SupportedSectorSizes) { + if (head.Length < size + MinHeaderSize || !head.AsSpan(size, 8).SequenceEqual(Signature)) continue; + var header = head.AsSpan(size, Math.Min(size, head.Length - size)); + var entrySize = BinaryPrimitives.ReadUInt32LittleEndian(header[84..]); + if (entrySize < MinEntrySize) + throw new InvalidDataException($"GPT entry size {entrySize} is smaller than minimum {MinEntrySize}."); + if (!TryReadEntryBytes(disk, header, size, out var entries)) + throw new InvalidDataException("GPT entry array lies outside the image."); + return new Located(ToInfo(header, size, fromBackup: false), entries); + } + return null; + } + + private static Located? TryLocateBackup(Stream disk, int sectorSize, ulong backupLba) { + var offset = (long)backupLba * sectorSize; + if (backupLba < 2 || offset + sectorSize > disk.Length) return null; + var block = ReadAt(disk, offset, sectorSize); + if (block.Length < sectorSize || !IsValidHeaderBlock(block, sectorSize, backupLba)) return null; + return TryReadEntries(disk, block, sectorSize, out var entries) + ? new Located(ToInfo(block, sectorSize, fromBackup: true), entries) + : null; + } + + private static bool HasProtectiveEntry(ReadOnlySpan head) { + if (head.Length < 512 || head[510] != 0x55 || head[511] != 0xAA) return false; + for (var i = 0; i < 4; ++i) + if (head[0x1BE + i * 16 + 4] == 0xEE) return true; + return false; + } + + private static bool IsValidHeaderBlock(ReadOnlySpan header, int sectorSize, ulong expectedMyLba) { + if (header.Length < MinHeaderSize || !header[..8].SequenceEqual(Signature)) return false; var headerSize = BinaryPrimitives.ReadUInt32LittleEndian(header[12..]); - if (headerSize < MinHeaderSize || headerSize > SectorSize || headerSize > header.Length) return false; + if (headerSize < MinHeaderSize || headerSize > sectorSize || headerSize > header.Length) return false; Span copy = stackalloc byte[(int)headerSize]; header[..(int)headerSize].CopyTo(copy); @@ -56,69 +203,71 @@ public static bool HasValidHeader(ReadOnlySpan data) { var myLba = BinaryPrimitives.ReadUInt64LittleEndian(header[24..]); var entrySize = BinaryPrimitives.ReadUInt32LittleEndian(header[84..]); var entryCount = BinaryPrimitives.ReadUInt32LittleEndian(header[80..]); - return myLba == 1 + return myLba == expectedMyLba && entrySize >= MinEntrySize && (entrySize & (entrySize - 1)) == 0 && entryCount is > 0 and <= 65536; } - /// - /// Parses all partitions from a GPT disk image. - /// - /// The full disk image as a readable stream. - /// A list of partition entries. - public static List Parse(Stream diskData) { - // Read GPT header at LBA 1. - var header = new byte[SectorSize]; - diskData.Position = HeaderOffset; - diskData.ReadExactly(header); - - if (!header.AsSpan(0, 8).SequenceEqual(Signature)) - throw new InvalidDataException("Invalid GPT: missing 'EFI PART' signature at LBA 1."); - - var revision = BinaryPrimitives.ReadUInt32LittleEndian(header.AsSpan(8)); - var headerSize = BinaryPrimitives.ReadUInt32LittleEndian(header.AsSpan(12)); - var partitionEntryLba = BinaryPrimitives.ReadUInt64LittleEndian(header.AsSpan(72)); - var numberOfEntries = BinaryPrimitives.ReadUInt32LittleEndian(header.AsSpan(80)); - var entrySize = BinaryPrimitives.ReadUInt32LittleEndian(header.AsSpan(84)); - - if (entrySize < MinEntrySize) - throw new InvalidDataException($"GPT entry size {entrySize} is smaller than minimum {MinEntrySize}."); - - // Read all partition entries. - var totalSize = (long)numberOfEntries * entrySize; - var entryData = new byte[totalSize]; - diskData.Position = (long)partitionEntryLba * SectorSize; - diskData.ReadExactly(entryData); + private static bool TryReadEntries(Stream disk, ReadOnlySpan header, int sectorSize, out byte[] entries) { + if (!TryReadEntryBytes(disk, header, sectorSize, out entries)) return false; + var expected = BinaryPrimitives.ReadUInt32LittleEndian(header[88..]); + return Compression.Core.Checksums.Crc32.Compute(entries) == expected; + } + private static bool TryReadEntryBytes(Stream disk, ReadOnlySpan header, int sectorSize, out byte[] entries) { + entries = []; + var entriesLba = BinaryPrimitives.ReadUInt64LittleEndian(header[72..]); + var count = BinaryPrimitives.ReadUInt32LittleEndian(header[80..]); + var size = BinaryPrimitives.ReadUInt32LittleEndian(header[84..]); + var total = (long)count * size; + if (total <= 0 || total > MaxEntryArrayBytes) return false; + if (entriesLba > (ulong)(long.MaxValue / sectorSize)) return false; + var offset = (long)entriesLba * sectorSize; + if (offset < 0 || offset + total > disk.Length) return false; + entries = ReadAt(disk, offset, (int)total); + return entries.Length == total; + } + + private static HeaderInfo ToInfo(ReadOnlySpan header, int sectorSize, bool fromBackup) => new( + sectorSize, + BinaryPrimitives.ReadUInt64LittleEndian(header[24..]), + BinaryPrimitives.ReadUInt64LittleEndian(header[32..]), + BinaryPrimitives.ReadUInt64LittleEndian(header[40..]), + BinaryPrimitives.ReadUInt64LittleEndian(header[48..]), + ReadMixedEndianGuid(header[56..]), + BinaryPrimitives.ReadUInt64LittleEndian(header[72..]), + BinaryPrimitives.ReadUInt32LittleEndian(header[80..]), + BinaryPrimitives.ReadUInt32LittleEndian(header[84..]), + fromBackup); + + private static List ParseEntries(byte[] entryData, HeaderInfo info) { var result = new List(); var index = 0; + var entrySize = (int)info.EntrySize; + var sectorSize = info.SectorSize; - for (var i = 0; i < (int)numberOfEntries; i++) { - var offset = (int)(i * entrySize); - var entry = entryData.AsSpan(offset, (int)entrySize); + for (var i = 0; i < (int)info.EntryCount; i++) { + var entry = entryData.AsSpan(i * entrySize, entrySize); // Read type GUID (mixed endian: first 3 components LE, last 2 BE). var typeGuid = ReadMixedEndianGuid(entry); if (typeGuid == Guid.Empty) continue; // Unused entry. - // Read unique GUID. - var uniqueGuid = ReadMixedEndianGuid(entry[16..]); - var firstLba = BinaryPrimitives.ReadUInt64LittleEndian(entry[32..]); var lastLba = BinaryPrimitives.ReadUInt64LittleEndian(entry[40..]); var attributes = BinaryPrimitives.ReadUInt64LittleEndian(entry[48..]); + if (lastLba < firstLba || firstLba > (ulong)(long.MaxValue / sectorSize) || lastLba > (ulong)(long.MaxValue / sectorSize)) + continue; // An inverted or impossible range describes no data. // Read UTF-16LE name (up to 36 characters = 72 bytes, starting at offset 56). - var nameLen = Math.Min(72, (int)entrySize - 56); + var nameLen = Math.Min(72, entrySize - 56); var name = Encoding.Unicode.GetString(entry.Slice(56, nameLen)).TrimEnd('\0'); - var sizeBytes = (long)(lastLba - firstLba + 1) * SectorSize; - result.Add(new() { Index = index++, - StartOffset = (long)firstLba * SectorSize, - Size = sizeBytes, + StartOffset = (long)firstLba * sectorSize, + Size = (long)(lastLba - firstLba + 1) * sectorSize, TypeName = PartitionTypeDatabase.GetGptTypeName(typeGuid), IsActive = (attributes & 0x04) != 0, // Legacy BIOS bootable attribute. TypeCode = typeGuid.ToString("D").ToUpperInvariant(), @@ -130,6 +279,14 @@ public static List Parse(Stream diskData) { return result; } + private static byte[] ReadAt(Stream disk, long offset, int count) { + if (count <= 0) return []; + var buffer = new byte[count]; + disk.Position = offset; + var read = disk.ReadAtLeast(buffer, count, throwOnEndOfStream: false); + return read == count ? buffer : buffer[..read]; + } + /// /// Reads a GUID in GPT mixed-endian format: first 3 components are little-endian, /// last 2 are big-endian (stored as-is). diff --git a/Compression.Core/DiskImage/MbrParser.cs b/Compression.Core/DiskImage/MbrParser.cs index 4e6688499..351b5ebe2 100644 --- a/Compression.Core/DiskImage/MbrParser.cs +++ b/Compression.Core/DiskImage/MbrParser.cs @@ -5,6 +5,7 @@ namespace Compression.Core.DiskImage; /// /// Parses MBR (Master Boot Record) partition tables, including extended/logical partition chains. /// The MBR sits at LBA 0 (offset 0) of a disk image, with 4 primary partition entries at offset 0x1BE. +/// The entries count in logical blocks: 512 bytes on a classic or 512e disk, 4096 on a 4Kn disk. /// public static class MbrParser { @@ -20,6 +21,9 @@ public static class MbrParser { /// Standard sector size. private const int SectorSize = 512; + /// Cap on logical partitions walked in one chain, so a damaged or hostile chain cannot spin. + private const int MaxLogicalPartitions = 256; + /// /// Checks whether the given data starts with a valid MBR. /// @@ -44,7 +48,8 @@ public static bool IsMbr(ReadOnlySpan data) /// /// At least the first 512 bytes of the image. /// Length of the whole image in bytes. - public static bool HasValidPartitionTable(ReadOnlySpan sector0, long diskLength) { + /// Logical block size the entries' LBAs are counted in. + public static bool HasValidPartitionTable(ReadOnlySpan sector0, long diskLength, int sectorSize = SectorSize) { if (!IsMbr(sector0)) return false; Span<(ulong Start, ulong End)> used = stackalloc (ulong, ulong)[4]; @@ -57,10 +62,10 @@ public static bool HasValidPartitionTable(ReadOnlySpan sector0, long diskL if (entry.Type == 0x00) continue; if (entry.LbaStart == 0 || entry.SectorCount == 0) return false; - var start = (ulong)entry.LbaStart * SectorSize; + var start = (ulong)entry.LbaStart * (ulong)sectorSize; if (start >= (ulong)diskLength) return false; - var end = start + (ulong)entry.SectorCount * SectorSize; + var end = start + (ulong)entry.SectorCount * (ulong)sectorSize; for (var j = 0; j < count; ++j) if (start < used[j].End && used[j].Start < end) return false; @@ -73,8 +78,9 @@ public static bool HasValidPartitionTable(ReadOnlySpan sector0, long diskL /// Parses all partitions from an MBR, including extended/logical partitions. /// /// The full disk image as a readable stream. + /// Logical block size the entries' LBAs are counted in (512, or 4096 on a 4Kn disk). /// A list of partition entries. - public static List Parse(Stream diskData) { + public static List Parse(Stream diskData, int sectorSize = SectorSize) { var mbr = new byte[SectorSize]; diskData.Position = 0; diskData.ReadExactly(mbr); @@ -90,15 +96,13 @@ public static List Parse(Stream diskData) { if (entry.Type == 0x00) continue; - if (entry.Type is 0x05 or 0x0F or 0x85) { - // Extended partition: walk the chain. - var extStart = entry.LbaStart; - ParseExtendedChain(diskData, extStart, extStart, result, ref index); + if (IsExtended(entry.Type)) { + ParseExtendedChain(diskData, entry.LbaStart, entry.LbaStart, entry.SectorCount, sectorSize, result, ref index); } else { result.Add(new() { Index = index++, - StartOffset = (long)entry.LbaStart * SectorSize, - Size = (long)entry.SectorCount * SectorSize, + StartOffset = (long)entry.LbaStart * sectorSize, + Size = (long)entry.SectorCount * sectorSize, TypeName = PartitionTypeDatabase.GetMbrTypeName(entry.Type), IsActive = entry.IsActive, TypeCode = $"0x{entry.Type:X2}", @@ -115,8 +119,9 @@ public static List Parse(Stream diskData) { /// Operates on a 512-byte buffer (no stream needed). /// /// The first 512 bytes of the disk image. + /// Logical block size the entries' LBAs are counted in. /// A list of primary partition entries (excluding empty entries). - public static List ParsePrimary(ReadOnlySpan mbr) { + public static List ParsePrimary(ReadOnlySpan mbr, int sectorSize = SectorSize) { if (mbr.Length < SectorSize) throw new ArgumentException("MBR data must be at least 512 bytes.", nameof(mbr)); @@ -130,35 +135,50 @@ public static List ParsePrimary(ReadOnlySpan mbr) { result.Add(new() { Index = index++, - StartOffset = (long)entry.LbaStart * SectorSize, - Size = (long)entry.SectorCount * SectorSize, + StartOffset = (long)entry.LbaStart * sectorSize, + Size = (long)entry.SectorCount * sectorSize, TypeName = PartitionTypeDatabase.GetMbrTypeName(entry.Type), IsActive = entry.IsActive, TypeCode = $"0x{entry.Type:X2}", - Source = entry.Type is 0x05 or 0x0F or 0x85 ? "MBR (Extended Container)" : "MBR" + Source = IsExtended(entry.Type) ? "MBR (Extended Container)" : "MBR" }); } return result; } - private static void ParseExtendedChain(Stream diskData, uint ebrLba, uint extStart, List result, ref int index) { + private static bool IsExtended(byte type) => type is 0x05 or 0x0F or 0x85; + + /// + /// Walks the chain of extended boot records. Each EBR's first entry is a logical partition + /// whose start is relative to that EBR; its second entry links to the next EBR, relative to + /// the start of the extended partition. The walk stops at a loop, at an EBR without the boot + /// signature, at a link that leaves the extended partition, and after + /// records. + /// + private static void ParseExtendedChain(Stream diskData, uint ebrLba, uint extStart, uint extSectors, int sectorSize, + List result, ref int index) { var seen = new HashSet(); + var extEnd = (ulong)extStart + extSectors; - while (ebrLba != 0 && seen.Add(ebrLba)) { + while (ebrLba != 0 && seen.Count < MaxLogicalPartitions && seen.Add(ebrLba)) { + if (ebrLba < extStart || ebrLba >= extEnd) + break; var ebr = new byte[SectorSize]; - diskData.Position = (long)ebrLba * SectorSize; - if (diskData.Read(ebr) < SectorSize) + diskData.Position = (long)ebrLba * sectorSize; + if (diskData.ReadAtLeast(ebr, SectorSize, throwOnEndOfStream: false) < SectorSize) + break; + if (BinaryPrimitives.ReadUInt16LittleEndian(ebr.AsSpan(510)) != BootSignature) break; // Entry 0: the logical partition (relative to this EBR's LBA). var logical = ParseEntry(ebr.AsSpan(PartitionTableOffset)); if (logical.Type != 0x00 && logical.SectorCount > 0) { - var logicalLba = ebrLba + logical.LbaStart; + var logicalLba = (long)ebrLba + logical.LbaStart; result.Add(new() { Index = index++, - StartOffset = (long)logicalLba * SectorSize, - Size = (long)logical.SectorCount * SectorSize, + StartOffset = logicalLba * sectorSize, + Size = (long)logical.SectorCount * sectorSize, TypeName = PartitionTypeDatabase.GetMbrTypeName(logical.Type), IsActive = logical.IsActive, TypeCode = $"0x{logical.Type:X2}", @@ -171,7 +191,9 @@ private static void ParseExtendedChain(Stream diskData, uint ebrLba, uint extSta if (next.Type == 0x00 || next.SectorCount == 0) break; - ebrLba = extStart + next.LbaStart; + var nextLba = (ulong)extStart + next.LbaStart; + if (nextLba > uint.MaxValue) break; + ebrLba = (uint)nextLba; } } diff --git a/Compression.Core/DiskImage/PartitionEditor.cs b/Compression.Core/DiskImage/PartitionEditor.cs index 150dc5510..33684d6d8 100644 --- a/Compression.Core/DiskImage/PartitionEditor.cs +++ b/Compression.Core/DiskImage/PartitionEditor.cs @@ -36,9 +36,16 @@ namespace Compression.Core.DiskImage; /// public sealed class PartitionEditor { - /// Standard sector size used by the editor. + /// + /// The classic logical block size, and the one a disk without a GPT is assumed to use unless + /// the caller says otherwise. A 4Kn GPT disk is edited in 4096-byte blocks: see + /// . + /// public const int SectorSize = 512; + /// Logical block sizes the editor supports. + public static IReadOnlyList SupportedSectorSizes => GptParser.SupportedSectorSizes; + private const int MbrSignatureOffset = 510; private const ushort MbrBootSignature = 0xAA55; private const int MbrTableOffset = 0x1BE; @@ -49,10 +56,11 @@ public sealed class PartitionEditor { private const int GptHeaderSize = 92; private const int DefaultGptEntrySize = 128; private const int GptEntryCount = 128; - private const int GptReservedSectors = 33; // 32 sectors of entries + 1 sector of header private static ReadOnlySpan GptSignature => "EFI PART"u8; private readonly Stream _stream; + private readonly int _requestedSectorSize; + private int _sectorSize; private PartitionScheme _scheme; private readonly List _entries = []; @@ -60,25 +68,103 @@ public sealed class PartitionEditor { // defaults applied when initialising a fresh GPT). private uint _gptEntrySize = DefaultGptEntrySize; private uint _gptEntryCount = GptEntryCount; + private Guid? _gptDiskGuid; /// Partition scheme currently in use on the disk image. public PartitionScheme Scheme => this._scheme; + /// + /// The logical block size every LBA on this disk is counted in: the size at which the GPT + /// header validated, or — for an MBR or blank disk — the size the editor was opened with. + /// + public int LogicalSectorSize => this._sectorSize; + /// /// Opens an editor over the given disk-image stream. The stream must be /// readable, writable, and seekable. The scheme (MBR/GPT/None) is /// auto-detected from the on-disk signature. /// - public PartitionEditor(Stream stream) { + public PartitionEditor(Stream stream) : this(stream, SectorSize) { } + + /// + /// Opens an editor over the given disk-image stream, counting LBAs in + /// -byte blocks unless a GPT on the disk validates at another + /// supported block size — the table on the disk decides, since rewriting it at a different + /// size would move every partition. + /// + /// Readable, writable, seekable disk image. + /// 512 or 4096. + public PartitionEditor(Stream stream, int sectorSize) { ArgumentNullException.ThrowIfNull(stream); if (!stream.CanRead) throw new ArgumentException("Stream must be readable.", nameof(stream)); if (!stream.CanWrite) throw new ArgumentException("Stream must be writable.", nameof(stream)); if (!stream.CanSeek) throw new ArgumentException("Stream must be seekable.", nameof(stream)); + if (!SupportedSectorSizes.Contains(sectorSize)) + throw new ArgumentOutOfRangeException(nameof(sectorSize), sectorSize, "Logical sector size must be 512 or 4096."); this._stream = stream; + this._requestedSectorSize = sectorSize; + this._sectorSize = sectorSize; this.Reload(); } + /// + /// Writes an empty partition table of over the start (and, for GPT, + /// the end) of and returns an editor over it. A GPT gets the + /// standard 128 entries of 128 bytes, its backup at the last LBA and a protective MBR. + /// + /// Readable, writable, seekable target, already at the disk's full length. + /// or . + /// 512 or 4096. + public static PartitionEditor CreateEmpty(Stream stream, PartitionScheme scheme, int sectorSize = SectorSize) { + ArgumentNullException.ThrowIfNull(stream); + if (!SupportedSectorSizes.Contains(sectorSize)) + throw new ArgumentOutOfRangeException(nameof(sectorSize), sectorSize, "Logical sector size must be 512 or 4096."); + if (stream.Length % sectorSize != 0) + throw new ArgumentException($"Disk length {stream.Length} is not a whole number of {sectorSize}-byte sectors.", nameof(stream)); + + // Clear whatever table the bytes held, so the editor starts from a blank disk. + var editor = new PartitionEditor(stream, sectorSize); + editor._entries.Clear(); + editor._gptRawSlots.Clear(); + editor._sectorSize = sectorSize; + switch (scheme) { + case PartitionScheme.Gpt: + editor._scheme = PartitionScheme.Gpt; + editor._gptEntrySize = DefaultGptEntrySize; + editor._gptEntryCount = GptEntryCount; + editor._gptDiskGuid = null; + editor.RewriteGpt(); + break; + case PartitionScheme.Mbr: + if (stream.Length / sectorSize > uint.MaxValue) + throw new InvalidOperationException("An MBR cannot address a disk of more than 2^32 sectors; use GPT."); + editor._scheme = PartitionScheme.Mbr; + editor.InitializeEmptyMbr(); + break; + default: + throw new ArgumentOutOfRangeException(nameof(scheme), scheme, "Only GPT and MBR tables can be created."); + } + return editor; + } + + /// Sectors the GPT entry array occupies at the current block size. + private long GptEntrySectors => ((long)this._gptEntrySize * this._gptEntryCount + this._sectorSize - 1) / this._sectorSize; + + /// First LBA a GPT partition may use: after the protective MBR, the header and the entry array. + private long GptFirstUsableLba => 2 + this.GptEntrySectors; + + /// Last LBA a GPT partition may use: before the backup entry array and the backup header. + private long GptLastUsableLba => this._stream.Length / this._sectorSize - this.GptEntrySectors - 2; + + /// + /// The byte range partitions may occupy under the current scheme: everything after LBA 0 for + /// MBR; between the primary and backup GPT structures for GPT. + /// + public (long Start, long End) UsableRange => this._scheme == PartitionScheme.Gpt + ? (this.GptFirstUsableLba * this._sectorSize, (this.GptLastUsableLba + 1) * this._sectorSize) + : (this._sectorSize, this._stream.Length / this._sectorSize * this._sectorSize); + /// /// Convenience factory for an editor backed by an /// container (VHD/VHDX/VMDK/QCOW2/VDI). The caller is responsible for @@ -115,9 +201,10 @@ public IReadOnlyList ListPartitions() /// Range overlaps an existing /// partition or exceeds the disk length. public void AddPartition(long startByteOffset, long lengthBytes, PartitionType type, string? label) { - ValidateRange(startByteOffset, lengthBytes); + this.ValidateRange(startByteOffset, lengthBytes); this.RequireFits(startByteOffset, lengthBytes); this.RequireNoOverlap(startByteOffset, lengthBytes); + this.RequireUsable(startByteOffset, lengthBytes); if (this._scheme == PartitionScheme.None) { // No table yet — promote to MBR. @@ -185,7 +272,7 @@ public void AddPartition(long startByteOffset, long lengthBytes, PartitionType t /// No extended container exists, /// range outside container, or overlap detected. public void AddLogicalPartition(long startByteOffset, long lengthBytes, PartitionType type, string? label) { - ValidateRange(startByteOffset, lengthBytes); + this.ValidateRange(startByteOffset, lengthBytes); if (this._scheme != PartitionScheme.Mbr) throw new InvalidOperationException("Logical partitions only exist in MBR layouts."); @@ -204,13 +291,13 @@ public void AddLogicalPartition(long startByteOffset, long lengthBytes, Partitio var existingLogicals = this._entries.Count(e => e.Source == "EBR"); if (existingLogicals == 0) { // First logical: container_start + 1 sector is the earliest legal data offset. - if (startByteOffset < container.StartOffset + SectorSize) + if (startByteOffset < container.StartOffset + this._sectorSize) throw new InvalidOperationException( - $"First logical must start at or after container_start + 1 sector ({container.StartOffset + SectorSize}); got {startByteOffset}."); + $"First logical must start at or after container_start + 1 sector ({container.StartOffset + this._sectorSize}); got {startByteOffset}."); } else { // Subsequent logical needs an EBR sector before it. - var ebrLba = (startByteOffset / SectorSize) - 1; - var ebrStart = ebrLba * SectorSize; + var ebrLba = (startByteOffset / this._sectorSize) - 1; + var ebrStart = ebrLba * this._sectorSize; if (ebrStart <= container.StartOffset) throw new InvalidOperationException( $"Logical start ({startByteOffset}) leaves no room for an EBR sector inside the extended container [{container.StartOffset}..{container.StartOffset + container.Size})."); @@ -221,8 +308,8 @@ public void AddLogicalPartition(long startByteOffset, long lengthBytes, Partitio $"Logical end ({startByteOffset + lengthBytes}) exceeds extended container end ({container.StartOffset + container.Size})."); // Overlap check: EBR + data for non-first; just data for first (EBR is at container start). - var checkStart = existingLogicals == 0 ? startByteOffset : startByteOffset - SectorSize; - var checkLength = existingLogicals == 0 ? lengthBytes : SectorSize + lengthBytes; + var checkStart = existingLogicals == 0 ? startByteOffset : startByteOffset - this._sectorSize; + var checkLength = existingLogicals == 0 ? lengthBytes : this._sectorSize + lengthBytes; this.RequireNoOverlap(checkStart, checkLength); var entry = new PartitionEntry { @@ -278,9 +365,9 @@ public void PurgePartition(int index) { // For logicals also zero the preceding EBR sector. if (e.Source == "EBR") { - var ebrStart = e.StartOffset - SectorSize; + var ebrStart = e.StartOffset - this._sectorSize; if (ebrStart >= 0) - this.ZeroRange(ebrStart, SectorSize); + this.ZeroRange(ebrStart, this._sectorSize); } this.DeletePartition(index); @@ -319,11 +406,22 @@ public void ConvertMbrToGpt() { }); } + // A partition inside the GPT's own structures would be overwritten by them: refuse before + // writing anything. + this._gptEntrySize = DefaultGptEntrySize; + this._gptEntryCount = GptEntryCount; + var firstUsable = this.GptFirstUsableLba * this._sectorSize; + var lastUsableEnd = (this.GptLastUsableLba + 1) * this._sectorSize; + foreach (var e in translated) + if (e.StartOffset < firstUsable || e.StartOffset + e.Size > lastUsableEnd) + throw new InvalidOperationException( + $"Partition [{e.StartOffset}..{e.StartOffset + e.Size}) overlaps where the GPT's own structures go " + + $"(usable range [{firstUsable}..{lastUsableEnd})); converting would overwrite its data."); + this._entries.Clear(); this._entries.AddRange(translated); this._scheme = PartitionScheme.Gpt; - this._gptEntrySize = DefaultGptEntrySize; - this._gptEntryCount = GptEntryCount; + this._gptDiskGuid = null; this.RewriteGpt(); // Also rewrites the protective MBR. } @@ -359,12 +457,16 @@ public void ConvertGptToMbr() { }); } - // Zero out primary GPT (header + entry array, LBAs 1..33) and backup - // GPT (last 33 sectors) before writing the MBR. - this.ZeroRange(GptHeaderLba * SectorSize, (long)GptReservedSectors * SectorSize); - var backupStart = this._stream.Length - (long)GptReservedSectors * SectorSize; - if (backupStart > 0) - this.ZeroRange(backupStart, (long)GptReservedSectors * SectorSize); + if (translated.Any(e => (e.StartOffset + e.Size) / this._sectorSize > uint.MaxValue)) + throw new InvalidOperationException("A partition ends past 2^32 sectors, which an MBR entry cannot address."); + + // Zero out primary GPT (header + entry array) and backup GPT (entry array + header at the + // end) before writing the MBR. Both lie outside the usable range, so no partition is touched. + var reserved = (this.GptEntrySectors + 1) * this._sectorSize; + this.ZeroRange(GptHeaderLba * (long)this._sectorSize, reserved); + var backupStart = this._stream.Length / this._sectorSize * this._sectorSize - reserved; + if (backupStart > reserved) + this.ZeroRange(backupStart, reserved); this._entries.Clear(); this._entries.AddRange(translated); @@ -508,8 +610,8 @@ private static bool MatchesAtOffset(byte[] probe, MagicSignature sig) { public void ResizePartition(int index, long newSizeBytes) { this.RequireValidIndex(index); if (newSizeBytes <= 0) throw new ArgumentOutOfRangeException(nameof(newSizeBytes)); - if (newSizeBytes % SectorSize != 0) - throw new ArgumentException($"New size must be sector-aligned ({SectorSize} bytes).", nameof(newSizeBytes)); + if (newSizeBytes % this._sectorSize != 0) + throw new ArgumentException($"New size must be sector-aligned ({this._sectorSize} bytes).", nameof(newSizeBytes)); var resizer = InPlaceFilesystemResizer ?? throw new InvalidOperationException( @@ -527,6 +629,9 @@ public void ResizePartition(int index, long newSizeBytes) { if (newEnd > this._stream.Length) throw new InvalidOperationException( $"New partition end ({newEnd}) exceeds disk length ({this._stream.Length})."); + if (newEnd > this.UsableRange.End) + throw new InvalidOperationException( + $"New partition end ({newEnd}) runs into the backup partition table (usable end {this.UsableRange.End})."); if (!isShrink) { foreach (var other in this._entries) { if (other.Index == e.Index) continue; @@ -658,14 +763,17 @@ public PartitionTableVerification Verify() { /// public void Reload() { this._entries.Clear(); + this._gptRawSlots.Clear(); + this._gptDiskGuid = null; + this._sectorSize = this._requestedSectorSize; if (this._stream.Length < SectorSize * 2L) { this._scheme = PartitionScheme.None; return; } - var head = new byte[Math.Min(4096, (int)this._stream.Length)]; + var head = new byte[(int)Math.Min(GptParser.ProbeBytes, this._stream.Length)]; this._stream.Position = 0; - var read = this._stream.Read(head, 0, head.Length); + var read = this._stream.ReadAtLeast(head, head.Length, throwOnEndOfStream: false); if (read < SectorSize) { this._scheme = PartitionScheme.None; return; @@ -673,22 +781,24 @@ public void Reload() { if (GptParser.IsGpt(head)) { try { - var parsed = GptParser.Parse(this._stream); - // Read entry size from the on-disk header for round-trip preservation. - this._stream.Position = (long)GptHeaderLba * SectorSize; - Span hdr = stackalloc byte[GptHeaderSize]; - this._stream.ReadExactly(hdr); - this._gptEntryCount = BinaryPrimitives.ReadUInt32LittleEndian(hdr[80..]); - this._gptEntrySize = BinaryPrimitives.ReadUInt32LittleEndian(hdr[84..]); - if (this._gptEntrySize < DefaultGptEntrySize) this._gptEntrySize = DefaultGptEntrySize; - if (this._gptEntryCount == 0) this._gptEntryCount = GptEntryCount; + var parsed = GptParser.Parse(this._stream, out var info); + // Keep the on-disk layout (block size, entry size/count, disk GUID) for round-trip preservation. + this._sectorSize = info.SectorSize; + this._gptEntryCount = info.EntryCount == 0 ? GptEntryCount : info.EntryCount; + this._gptEntrySize = info.EntrySize < DefaultGptEntrySize ? DefaultGptEntrySize : info.EntrySize; + this._gptDiskGuid = info.DiskGuid == Guid.Empty ? null : info.DiskGuid; + var entryArray = new byte[(long)info.EntryCount * info.EntrySize]; + this._stream.Position = (long)info.EntriesLba * info.SectorSize; + this._stream.ReadExactly(entryArray); + this.CaptureGptSlots(entryArray); this._scheme = PartitionScheme.Gpt; this._entries.AddRange(parsed); this.ReindexAll(); return; - } catch { + } catch (Exception ex) when (ex is IOException or InvalidDataException or ArgumentException or OverflowException) { // Fall through to MBR. + this._sectorSize = this._requestedSectorSize; } } @@ -697,9 +807,9 @@ public void Reload() { // Use the full parser so logical entries are included as well. List parsed; try { - parsed = MbrParser.Parse(this._stream); - } catch { - parsed = [.. MbrParser.ParsePrimary(head)]; + parsed = MbrParser.Parse(this._stream, this._sectorSize); + } catch (Exception ex) when (ex is IOException or InvalidDataException or ArgumentException) { + parsed = [.. MbrParser.ParsePrimary(head, this._sectorSize)]; } // MbrParser.Parse returns primaries (non-extended) plus EBR-discovered @@ -708,7 +818,7 @@ public void Reload() { this._stream.Position = 0; var sector0 = new byte[SectorSize]; this._stream.ReadExactly(sector0); - var primariesFromTable = MbrParser.ParsePrimary(sector0); + var primariesFromTable = MbrParser.ParsePrimary(sector0, this._sectorSize); var entries = new List(); foreach (var p in primariesFromTable) entries.Add(p); @@ -726,13 +836,13 @@ public void Reload() { // ── Overlap / validation helpers ─────────────────────────────────── - private static void ValidateRange(long startByteOffset, long lengthBytes) { + private void ValidateRange(long startByteOffset, long lengthBytes) { if (startByteOffset < 0) throw new ArgumentOutOfRangeException(nameof(startByteOffset)); if (lengthBytes <= 0) throw new ArgumentOutOfRangeException(nameof(lengthBytes)); - if (startByteOffset % SectorSize != 0) - throw new ArgumentException($"Start offset must be sector-aligned ({SectorSize} bytes).", nameof(startByteOffset)); - if (lengthBytes % SectorSize != 0) - throw new ArgumentException($"Length must be a multiple of sector size ({SectorSize} bytes).", nameof(lengthBytes)); + if (startByteOffset % this._sectorSize != 0) + throw new ArgumentException($"Start offset must be sector-aligned ({this._sectorSize} bytes).", nameof(startByteOffset)); + if (lengthBytes % this._sectorSize != 0) + throw new ArgumentException($"Length must be a multiple of sector size ({this._sectorSize} bytes).", nameof(lengthBytes)); } private void RequireFits(long startByteOffset, long lengthBytes) { @@ -741,6 +851,22 @@ private void RequireFits(long startByteOffset, long lengthBytes) { $"Partition end ({startByteOffset + lengthBytes}) exceeds disk length ({this._stream.Length})."); } + /// + /// A partition must stay clear of the table itself: LBA 0 for MBR, and for GPT the primary + /// header and entry array at the front and their backups at the end (UEFI 2.10, §5.3.1 — + /// FirstUsableLBA / LastUsableLBA). An MBR entry also cannot address past 2^32 sectors. + /// + private void RequireUsable(long startByteOffset, long lengthBytes) { + var (usableStart, usableEnd) = this.UsableRange; + if (startByteOffset < usableStart || startByteOffset + lengthBytes > usableEnd) + throw new InvalidOperationException( + $"Partition [{startByteOffset}..{startByteOffset + lengthBytes}) leaves the usable range [{usableStart}..{usableEnd}) " + + "and would overwrite the partition table."); + if (this._scheme != PartitionScheme.Gpt + && ((startByteOffset + lengthBytes) / this._sectorSize > uint.MaxValue)) + throw new InvalidOperationException("An MBR entry cannot address past 2^32 sectors; use GPT."); + } + private void RequireNoOverlap(long startByteOffset, long lengthBytes) { foreach (var existing in this._entries) { // Extended container is allowed to encompass its logicals — skip it @@ -825,8 +951,8 @@ private void RewriteMbr() { entry[1] = 0xFE; entry[2] = 0xFF; entry[3] = 0xFF; entry[4] = mbrByte; entry[5] = 0xFE; entry[6] = 0xFF; entry[7] = 0xFF; - BinaryPrimitives.WriteUInt32LittleEndian(entry[8..], (uint)(e.StartOffset / SectorSize)); - BinaryPrimitives.WriteUInt32LittleEndian(entry[12..], (uint)(e.Size / SectorSize)); + BinaryPrimitives.WriteUInt32LittleEndian(entry[8..], (uint)(e.StartOffset / this._sectorSize)); + BinaryPrimitives.WriteUInt32LittleEndian(entry[12..], (uint)(e.Size / this._sectorSize)); primarySlot++; } @@ -853,12 +979,12 @@ private void RewriteEbrChain() { .OrderBy(e => e.StartOffset) .ToList(); - var containerStartLba = (uint)(container.StartOffset / SectorSize); + var containerStartLba = (uint)(container.StartOffset / this._sectorSize); // Always zero the first sector of the container first — when the chain // shrinks (logicals removed), we don't want a stale EBR lingering. The // active first EBR will be rewritten below if logicals.Count > 0. - this.ZeroRange(container.StartOffset, SectorSize); + this.ZeroRange(container.StartOffset, this._sectorSize); if (logicals.Count == 0) { this._stream.Flush(); @@ -867,14 +993,14 @@ private void RewriteEbrChain() { for (var i = 0; i < logicals.Count; ++i) { var logical = logicals[i]; - var dataLba = (uint)(logical.StartOffset / SectorSize); + var dataLba = (uint)(logical.StartOffset / this._sectorSize); // EBR location: first logical -> container start; subsequent -> data-1. var ebrLba = i == 0 ? containerStartLba : dataLba - 1; - var ebrOffset = (long)ebrLba * SectorSize; + var ebrOffset = (long)ebrLba * this._sectorSize; var sector = new byte[SectorSize]; - if (ebrOffset + SectorSize <= this._stream.Length) { + if (ebrOffset + this._sectorSize <= this._stream.Length) { this._stream.Position = ebrOffset; this._stream.ReadExactly(sector); Array.Clear(sector, MbrTableOffset, 4 * MbrEntrySize); @@ -889,14 +1015,14 @@ private void RewriteEbrChain() { entry[4] = mbrByte; entry[5] = 0xFE; entry[6] = 0xFF; entry[7] = 0xFF; BinaryPrimitives.WriteUInt32LittleEndian(entry[8..], dataLba - ebrLba); - BinaryPrimitives.WriteUInt32LittleEndian(entry[12..], (uint)(logical.Size / SectorSize)); + BinaryPrimitives.WriteUInt32LittleEndian(entry[12..], (uint)(logical.Size / this._sectorSize)); } // Entry 1: pointer to next EBR (relative to extended container start), // or all-zero on the last EBR. if (i + 1 < logicals.Count) { var nextLogical = logicals[i + 1]; - var nextDataLba = (uint)(nextLogical.StartOffset / SectorSize); + var nextDataLba = (uint)(nextLogical.StartOffset / this._sectorSize); var nextEbrLba = nextDataLba - 1; var entry = sector.AsSpan(MbrTableOffset + MbrEntrySize, MbrEntrySize); @@ -905,7 +1031,7 @@ private void RewriteEbrChain() { entry[4] = 0x05; entry[5] = 0xFE; entry[6] = 0xFF; entry[7] = 0xFF; BinaryPrimitives.WriteUInt32LittleEndian(entry[8..], nextEbrLba - containerStartLba); - var nextLastLba = nextDataLba + (uint)(nextLogical.Size / SectorSize) - 1; + var nextLastLba = nextDataLba + (uint)(nextLogical.Size / this._sectorSize) - 1; BinaryPrimitives.WriteUInt32LittleEndian(entry[12..], nextLastLba - nextEbrLba + 1); } @@ -918,15 +1044,24 @@ private void RewriteEbrChain() { // ── GPT writers ──────────────────────────────────────────────────── + /// + /// Writes the protective MBR, the primary header and entry array at LBA 1/2, and the backup + /// entry array and header at the end of the disk (UEFI 2.10, §5.3). LastUsableLBA is the LBA + /// before the backup entry array, so the two never overlap. Entries that already existed keep + /// their unique GUID and attribute bits; the disk keeps its GUID. + /// private void RewriteGpt() { - var totalSectors = this._stream.Length / SectorSize; - if (totalSectors < 34 + GptReservedSectors) - throw new InvalidOperationException( - $"Stream too small for GPT layout ({this._stream.Length} bytes); need at least {(34 + GptReservedSectors) * SectorSize}."); - + var totalSectors = this._stream.Length / this._sectorSize; var entrySize = (int)this._gptEntrySize; var entryCount = (int)this._gptEntryCount; var entryArrayBytes = entrySize * entryCount; + var entrySectors = this.GptEntrySectors; + var reservedSectors = entrySectors + 1; + var firstUsable = 2 + entrySectors; + var lastUsable = totalSectors - reservedSectors - 1; + if (lastUsable < firstUsable) + throw new InvalidOperationException( + $"Stream too small for GPT layout ({this._stream.Length} bytes); need more than {(firstUsable + reservedSectors) * this._sectorSize}."); // Build entry array. var entriesBuf = new byte[entryArrayBytes]; @@ -934,35 +1069,34 @@ private void RewriteGpt() { var e = this._entries[i]; var slot = entriesBuf.AsSpan(i * entrySize, entrySize); var typeGuid = Guid.TryParse(e.TypeCode, out var g) ? g : Guid.Empty; + var known = this._gptRawSlots.TryGetValue(e.StartOffset, out var raw) && raw.Length == entrySize; + if (known) raw.CopyTo(slot); WriteMixedEndianGuid(slot, typeGuid); - WriteMixedEndianGuid(slot[16..], DeriveUniqueGuid(i, e.StartOffset)); - var firstLba = (ulong)(e.StartOffset / SectorSize); - var lastLba = (ulong)((e.StartOffset + e.Size) / SectorSize - 1); + if (!known) WriteMixedEndianGuid(slot[16..], DeriveUniqueGuid(i, e.StartOffset)); + var firstLba = (ulong)(e.StartOffset / this._sectorSize); + var lastLba = (ulong)((e.StartOffset + e.Size) / this._sectorSize - 1); BinaryPrimitives.WriteUInt64LittleEndian(slot[32..], firstLba); BinaryPrimitives.WriteUInt64LittleEndian(slot[40..], lastLba); - BinaryPrimitives.WriteUInt64LittleEndian(slot[48..], e.IsActive ? 0x04UL : 0UL); - var name = e.Name ?? string.Empty; - var nameBytes = Encoding.Unicode.GetBytes(name); + var attributes = known ? BinaryPrimitives.ReadUInt64LittleEndian(slot[48..]) & ~0x04UL : 0UL; + BinaryPrimitives.WriteUInt64LittleEndian(slot[48..], attributes | (e.IsActive ? 0x04UL : 0UL)); var nameMax = Math.Min(72, entrySize - 56); if (nameMax > 0) { - var copyLen = Math.Min(nameMax, nameBytes.Length); - nameBytes.AsSpan(0, copyLen).CopyTo(slot[56..]); + slot.Slice(56, nameMax).Clear(); + var nameBytes = Encoding.Unicode.GetBytes(e.Name ?? string.Empty); + nameBytes.AsSpan(0, Math.Min(nameMax, nameBytes.Length) & ~1).CopyTo(slot[56..]); } } var entriesCrc = Crc32.Compute(entriesBuf); - - // Number of sectors needed for the entry array. - var entrySectors = (entryArrayBytes + SectorSize - 1) / SectorSize; - var reservedSectors = Math.Max(GptReservedSectors, entrySectors + 1); + var diskGuid = this._gptDiskGuid ??= this.DeriveDiskGuid(); // Primary header at LBA 1. var primaryHeader = BuildGptHeader( myLba: 1, altLba: (ulong)(totalSectors - 1), - firstUsableLba: (ulong)(2 + entrySectors), - lastUsableLba: (ulong)(totalSectors - reservedSectors), - diskGuid: this.DeriveDiskGuid(), + firstUsableLba: (ulong)firstUsable, + lastUsableLba: (ulong)lastUsable, + diskGuid: diskGuid, entriesLba: 2, entriesCrc: entriesCrc, entryCount: entryCount, @@ -972,9 +1106,9 @@ private void RewriteGpt() { var backupHeader = BuildGptHeader( myLba: (ulong)(totalSectors - 1), altLba: 1, - firstUsableLba: (ulong)(2 + entrySectors), - lastUsableLba: (ulong)(totalSectors - reservedSectors), - diskGuid: this.DeriveDiskGuid(), + firstUsableLba: (ulong)firstUsable, + lastUsableLba: (ulong)lastUsable, + diskGuid: diskGuid, entriesLba: backupEntriesLba, entriesCrc: entriesCrc, entryCount: entryCount, @@ -983,23 +1117,39 @@ private void RewriteGpt() { // Protective MBR at LBA 0. this.WriteProtectiveMbr(totalSectors); - var headerSector = new byte[SectorSize]; + var headerSector = new byte[this._sectorSize]; primaryHeader.CopyTo(headerSector, 0); - this._stream.Position = SectorSize; + this._stream.Position = this._sectorSize; this._stream.Write(headerSector); - this._stream.Position = 2L * SectorSize; + this._stream.Position = 2L * this._sectorSize; this._stream.Write(entriesBuf); - this._stream.Position = (long)backupEntriesLba * SectorSize; + this._stream.Position = (long)backupEntriesLba * this._sectorSize; this._stream.Write(entriesBuf); Array.Clear(headerSector); backupHeader.CopyTo(headerSector, 0); - this._stream.Position = (totalSectors - 1) * SectorSize; + this._stream.Position = (totalSectors - 1) * this._sectorSize; this._stream.Write(headerSector); this._stream.Flush(); + this.CaptureGptSlots(entriesBuf); + } + + /// Raw entry-array slots by partition start, so a rewrite keeps unique GUIDs and attributes. + private readonly Dictionary _gptRawSlots = []; + + private void CaptureGptSlots(ReadOnlySpan entryArray) { + this._gptRawSlots.Clear(); + var size = (int)this._gptEntrySize; + for (var i = 0; (i + 1) * size <= entryArray.Length; ++i) { + var slot = entryArray.Slice(i * size, size); + if (slot[..16].IndexOfAnyExcept((byte)0) < 0) continue; + var firstLba = BinaryPrimitives.ReadUInt64LittleEndian(slot[32..]); + if (firstLba > (ulong)(long.MaxValue / this._sectorSize)) continue; + this._gptRawSlots[(long)firstLba * this._sectorSize] = slot.ToArray(); + } } private void WriteProtectiveMbr(long totalSectors) { @@ -1092,7 +1242,7 @@ private void VerifyMbr(List issues) { var container = this.FindExtendedContainer(); if (container is not null) { foreach (var logical in this._entries.Where(p => p.Source == "EBR")) { - if (logical.StartOffset < container.StartOffset + SectorSize) + if (logical.StartOffset < container.StartOffset + this._sectorSize) issues.Add($"Logical #{logical.Index} starts before container_start + 1 sector."); if (logical.StartOffset + logical.Size > container.StartOffset + container.Size) issues.Add($"Logical #{logical.Index} data extends past extended container end."); @@ -1101,14 +1251,24 @@ private void VerifyMbr(List issues) { } private void VerifyGpt(List issues) { - if (this._stream.Length < (34 + GptReservedSectors) * SectorSize) { + if (this.GptLastUsableLba < this.GptFirstUsableLba) { issues.Add("Stream too small to hold primary + backup GPT."); return; } + var (usableStart, usableEnd) = this.UsableRange; + var ordered = this._entries.OrderBy(e => e.StartOffset).ToList(); + for (var i = 0; i < ordered.Count; ++i) { + var e = ordered[i]; + if (e.StartOffset < usableStart || e.StartOffset + e.Size > usableEnd) + issues.Add($"Partition #{e.Index} [{e.StartOffset}..{e.StartOffset + e.Size}) lies outside the usable range [{usableStart}..{usableEnd})."); + if (i > 0 && ordered[i - 1].StartOffset + ordered[i - 1].Size > e.StartOffset) + issues.Add($"Partition #{ordered[i - 1].Index} overlaps partition #{e.Index}."); + } + // Primary header. - this._stream.Position = (long)GptHeaderLba * SectorSize; - var headerSector = new byte[SectorSize]; + this._stream.Position = (long)GptHeaderLba * this._sectorSize; + var headerSector = new byte[this._sectorSize]; this._stream.ReadExactly(headerSector); var primaryOk = VerifyGptHeader(headerSector, isPrimary: true, expectedMyLba: 1, issues, out var entriesLba, out var entryCount, out var entrySize, out var entriesCrc); @@ -1118,7 +1278,7 @@ private void VerifyGpt(List issues) { issues.Add($"GPT entry array size {entryBytes} out of range."); } else { var buf = new byte[entryBytes]; - this._stream.Position = (long)entriesLba * SectorSize; + this._stream.Position = (long)entriesLba * this._sectorSize; this._stream.ReadExactly(buf); var crc = Crc32.Compute(buf); if (crc != entriesCrc) @@ -1127,9 +1287,9 @@ private void VerifyGpt(List issues) { } // Backup header. - var totalSectors = this._stream.Length / SectorSize; - this._stream.Position = (totalSectors - 1) * SectorSize; - var backupSector = new byte[SectorSize]; + var totalSectors = this._stream.Length / this._sectorSize; + this._stream.Position = (totalSectors - 1) * this._sectorSize; + var backupSector = new byte[this._sectorSize]; this._stream.ReadExactly(backupSector); var backupOk = VerifyGptHeader(backupSector, isPrimary: false, expectedMyLba: (ulong)(totalSectors - 1), issues, out var backupEntriesLba, out var backupEntryCount, out var backupEntrySize, out var backupEntriesCrc); @@ -1138,7 +1298,7 @@ private void VerifyGpt(List issues) { var entryBytes = (long)backupEntryCount * backupEntrySize; if (entryBytes > 0 && entryBytes <= int.MaxValue) { var buf = new byte[entryBytes]; - this._stream.Position = (long)backupEntriesLba * SectorSize; + this._stream.Position = (long)backupEntriesLba * this._sectorSize; this._stream.ReadExactly(buf); var crc = Crc32.Compute(buf); if (crc != backupEntriesCrc) @@ -1166,7 +1326,7 @@ private static bool VerifyGptHeader(byte[] sector, bool isPrimary, ulong expecte return false; } var headerSize = BinaryPrimitives.ReadUInt32LittleEndian(sector.AsSpan(12)); - if (headerSize < GptHeaderSize || headerSize > SectorSize) { + if (headerSize < GptHeaderSize || headerSize > sector.Length) { issues.Add($"{label} GPT header size {headerSize} out of range."); return false; } diff --git a/Compression.Core/DiskImage/PartitionTableDetector.cs b/Compression.Core/DiskImage/PartitionTableDetector.cs index c87e1b495..93d234c13 100644 --- a/Compression.Core/DiskImage/PartitionTableDetector.cs +++ b/Compression.Core/DiskImage/PartitionTableDetector.cs @@ -4,89 +4,158 @@ namespace Compression.Core.DiskImage; /// /// Detects and parses partition tables from raw disk data. -/// Tries GPT first (since GPT disks also have a protective MBR), then falls back to MBR. /// +/// +/// Probe order, first hit wins: +/// +/// GPT, at 512- or 4096-byte logical blocks — ahead of the MBR, because a GPT +/// disk carries a protective (or hybrid) MBR too; +/// SGI volume header and Sun label — sector-0 labels with a magic and a +/// checksum, so they are certain when they match, and their last bytes could otherwise pass +/// for an MBR boot signature; +/// MBR, with its extended/logical (EBR) chain; +/// Apple Partition Map; +/// a stand-alone BSD disklabel (no MBR around it), both magics and its geometry verified; +/// an Amiga Rigid Disk Block in the first 16 blocks, checksum verified. +/// +/// public static class PartitionTableDetector { /// /// Result of partition table detection, including the scheme used and the discovered partitions. /// public sealed class DetectionResult { - /// The partition scheme detected: "GPT", "MBR", "APM", or "None". + /// The partition scheme detected: "GPT", "MBR", "APM", "BSD", "RDB", "Sun", "SGI", or "None". public required string Scheme { get; init; } /// Discovered partitions. Empty if no partition table was found. public required List Partitions { get; init; } + + /// Logical block size the table counts in (512 unless a 4Kn GPT was found). + public int SectorSize { get; init; } = 512; } /// Minimum data size needed to attempt detection (GPT header is at offset 512). private const int MinimumSize = 1024; + private static DetectionResult None => new() { Scheme = "None", Partitions = [] }; + /// /// Attempts to detect a partition table in the given stream. - /// Tries GPT first (higher precedence), then MBR. /// Returns an empty partition list if no valid partition table is found. /// /// A seekable stream containing raw disk image data. /// Detection result with scheme name and partitions. public static DetectionResult Detect(Stream diskData) { if (!diskData.CanSeek || diskData.Length < MinimumSize) - return new DetectionResult { Scheme = "None", Partitions = [] }; + return None; - // Read the first 1024 bytes for quick signature checks. - // Clamp against Length as long first to avoid overflow on multi-GiB disks. - var header = new byte[(int)Math.Min(4096L, diskData.Length)]; + // Enough to see LBA 1 at every supported block size, and an APM map at 2048-byte blocks. + var header = new byte[(int)Math.Min(GptParser.ProbeBytes, diskData.Length)]; diskData.Position = 0; - var bytesRead = diskData.Read(header, 0, header.Length); + var bytesRead = diskData.ReadAtLeast(header, header.Length, throwOnEndOfStream: false); diskData.Position = 0; if (bytesRead < MinimumSize) - return new DetectionResult { Scheme = "None", Partitions = [] }; + return None; + var head = header.AsSpan(0, bytesRead); - // Try GPT first (GPT disks have a protective MBR, so GPT takes priority). - if (GptParser.IsGpt(header.AsSpan(0, bytesRead))) { + // GPT first (GPT disks have a protective MBR, so GPT takes priority). + if (GptParser.IsGpt(head) || HasProtectiveEntry(head)) { try { - var gptPartitions = GptParser.Parse(diskData); + var gptPartitions = GptParser.Parse(diskData, out var info); if (gptPartitions.Count > 0) - return new DetectionResult { Scheme = "GPT", Partitions = gptPartitions }; - } catch { + return new DetectionResult { Scheme = "GPT", Partitions = gptPartitions, SectorSize = info.SectorSize }; + } catch (Exception ex) when (ex is IOException or InvalidDataException or ArgumentException or OverflowException) { // GPT parsing failed — fall through to MBR. } } + if (SgiDvhParser.IsDvh(head)) + return Bounded("SGI", SgiDvhParser.Parse(head), diskData.Length); + if (SunVtocParser.IsSunLabel(head)) + return Bounded("Sun", SunVtocParser.Parse(head), diskData.Length); + // Try MBR. - if (MbrParser.IsMbr(header.AsSpan(0, bytesRead))) { + if (MbrParser.IsMbr(head)) { try { - var mbrPartitions = MbrParser.Parse(diskData); - // Filter out partitions that reference data beyond the stream. - var validPartitions = mbrPartitions - .Where(p => p.StartOffset >= 0 && p.StartOffset < diskData.Length && p.Size > 0) - .ToList(); - if (validPartitions.Count > 0) - return new DetectionResult { Scheme = "MBR", Partitions = validPartitions }; - } catch { + // A protective entry describes the GPT, not data; with no readable GPT it holds nothing. + var mbrPartitions = MbrParser.Parse(diskData).Where(p => p.TypeCode != "0xEE").ToList(); + var result = Bounded("MBR", mbrPartitions, diskData.Length); + if (result.Partitions.Count > 0) + return result; + } catch (Exception ex) when (ex is IOException or InvalidDataException or ArgumentException) { // MBR parsing failed — fall through to APM. } } // Try APM (Apple Partition Map). APM disks carry neither an MBR boot // signature nor a GPT header, so this is reached only when both above fail. - if (ApmParser.IsApm(header.AsSpan(0, bytesRead))) { + if (ApmParser.IsApm(head)) { try { - var apmPartitions = ApmParser.Parse(diskData); - var validApm = apmPartitions - .Where(p => p.StartOffset >= 0 && p.StartOffset < diskData.Length && p.Size > 0) - .ToList(); - if (validApm.Count > 0) - return new DetectionResult { Scheme = "APM", Partitions = validApm }; - } catch { - // APM parsing failed — no partition table. + var apm = Bounded("APM", ApmParser.Parse(diskData), diskData.Length); + if (apm.Partitions.Count > 0) + return apm; + } catch (Exception ex) when (ex is IOException or InvalidDataException or ArgumentException) { + // APM parsing failed — try the remaining schemes. + } + } + + try { + if (BsdDisklabelParser.IsStandaloneDisklabel(diskData)) { + var bsd = Bounded("BSD", BsdDisklabelParser.Parse(diskData, 0), diskData.Length); + if (bsd.Partitions.Count > 0) + return bsd; } + if (AmigaRdbParser.IsRdb(diskData)) { + var rdb = Bounded("RDB", AmigaRdbParser.Parse(diskData), diskData.Length); + if (rdb.Partitions.Count > 0) + return rdb; + } + } catch (Exception ex) when (ex is IOException or InvalidDataException or ArgumentException) { + // An unreadable label holds nothing. + } finally { + diskData.Position = 0; } - return new DetectionResult { Scheme = "None", Partitions = [] }; + return None; } + /// + /// Whether the first sector holds a partition table of a scheme that has no MBR-style + /// boot signature to go on — SGI, Sun, a stand-alone BSD disklabel or an Amiga RDB — each + /// verified by its own checksum (BSD: by its two magics and geometry). Used by content detection, which has already ruled out + /// GPT, MBR and APM by then. + /// + public static bool HasSelfValidatingLabel(Stream disk) { + if (!disk.CanSeek || disk.Length < MinimumSize) return false; + var saved = disk.Position; + try { + var sector0 = new byte[512]; + disk.Position = 0; + if (disk.ReadAtLeast(sector0, 512, throwOnEndOfStream: false) < 512) return false; + return SgiDvhParser.IsDvh(sector0) || SunVtocParser.IsSunLabel(sector0) + || BsdDisklabelParser.IsStandaloneDisklabel(disk) || AmigaRdbParser.IsRdb(disk); + } catch (Exception ex) when (ex is IOException or InvalidDataException or ArgumentException) { + return false; + } finally { + disk.Position = saved; + } + } + + private static bool HasProtectiveEntry(ReadOnlySpan head) { + if (!MbrParser.IsMbr(head)) return false; + for (var i = 0; i < 4; ++i) + if (head[0x1BE + i * 16 + 4] == 0xEE) return true; + return false; + } + + /// Keeps only partitions that start inside the image and have a size. + private static DetectionResult Bounded(string scheme, List partitions, long length) => new() { + Scheme = scheme, + Partitions = partitions.Where(p => p.StartOffset >= 0 && p.StartOffset < length && p.Size > 0).ToList(), + }; + /// /// Attempts to detect a partition table in the given byte array. /// Convenience overload that wraps the data in a . diff --git a/Compression.Core/DiskImage/PartitionedDiskLister.cs b/Compression.Core/DiskImage/PartitionedDiskLister.cs index dfe67ccad..54316a58c 100644 --- a/Compression.Core/DiskImage/PartitionedDiskLister.cs +++ b/Compression.Core/DiskImage/PartitionedDiskLister.cs @@ -87,7 +87,8 @@ private static bool TryListBsdSlices(Stream disk, PartitionEntry part, string pr /// private static bool TryReadBsdSlices(Stream disk, PartitionEntry part, out List slices) { slices = []; - if (part.Size <= 0 || part.StartOffset < 0 || part.StartOffset >= disk.Length) + // A slice of a disklabel can start at the label itself; it does not nest a second time. + if (part.Source == "BSD" || part.Size <= 0 || part.StartOffset < 0 || part.StartOffset >= disk.Length) return false; try { using var window = new PartitionWindowStream(disk, part.StartOffset, part.Size); @@ -178,160 +179,309 @@ public static string MakePartitionPrefix(PartitionEntry part) { return $"Partition{part.Index + 1}_{safeName}"; } + // ── Partition addressing ────────────────────────────────────────────── + + /// + /// One listed partition: a top-level partition, or a slice of a BSD disklabel nested inside + /// one, with the prefix its entries are listed under. + /// + /// The table entry (for a BSD slice, the slice's own entry). + /// The listing prefix, e.g. Partition1_FAT32 or Partition1_FreeBSD/Partition1_4_2BSD. + /// Byte offset of the partition on the disk. + /// Bytes of the partition that lie inside the image. + /// Whether this is a slice inside a BSD disklabel rather than a table entry. + public sealed record PartitionTarget(PartitionEntry Partition, string Prefix, long Offset, long Size, bool IsBsdSlice); + + /// + /// Every partition the listing shows a prefix for, in listing order: top-level partitions, + /// with a BSD-disklabel container replaced by its slices. + /// + public static List EnumerateTargets(Stream disk, PartitionTableDetector.DetectionResult detection) { + var result = new List(); + foreach (var part in detection.Partitions) { + var prefix = MakePartitionPrefix(part); + if (TryReadBsdSlices(disk, part, out var slices)) { + foreach (var slice in slices) + result.Add(new PartitionTarget(slice, $"{prefix}/{MakePartitionPrefix(slice)}", slice.StartOffset, + Clamp(disk, slice), IsBsdSlice: true)); + continue; + } + result.Add(new PartitionTarget(part, prefix, part.StartOffset, Clamp(disk, part), IsBsdSlice: false)); + } + return result; + } + + private static long Clamp(Stream disk, PartitionEntry part) + => Math.Max(0, Math.Min(part.Size, disk.Length - part.StartOffset)); + + /// + /// Finds the partition an entry name addresses. is the rest of the + /// name below the partition's prefix — empty when the name is the partition itself + /// (PartitionN_Type or PartitionN_Type.raw). + /// + public static PartitionTarget? Resolve(IEnumerable targets, string entryName, out string innerPath) { + var name = entryName.Replace('\\', '/').Trim('/'); + innerPath = string.Empty; + PartitionTarget? best = null; + foreach (var t in targets) { + if (name.Equals(t.Prefix, StringComparison.Ordinal) || name.Equals(t.Prefix + ".raw", StringComparison.Ordinal)) { + innerPath = string.Empty; + return t; + } + if (name.StartsWith(t.Prefix + "/", StringComparison.Ordinal) && (best is null || t.Prefix.Length > best.Prefix.Length)) + best = t; + } + if (best is not null) innerPath = name[(best.Prefix.Length + 1)..]; + return best; + } + + /// + /// Whether edits through its own code for + /// (Add or Remove) rather than the interface's generic extract → re-create + /// default, which re-creates the volume and loses what its writer does not reproduce. + /// + public static bool HasNativeEditor(IArchiveModifiable modifier, string method) { + var map = modifier.GetType().GetInterfaceMap(typeof(IArchiveModifiable)); + for (var i = 0; i < map.InterfaceMethods.Length; ++i) { + var m = map.InterfaceMethods[i]; + if (m.Name != method || m.GetParameters().Length != 2) continue; + return map.TargetMethods[i].DeclaringType != typeof(IArchiveModifiable); + } + return false; + } + // ── Add / Remove (partition-aware) ──────────────────────────────────── /// - /// Partition-aware . Returns true - /// when a partition table was present and at least one input was dispatched - /// through the partition-aware path; false when no partition table - /// exists so the caller can fall through to the existing single-FS path. + /// Partition-aware . + /// Returns false when no partition table exists (or the stream is read-only) so the + /// caller can fall through to its single-filesystem path. /// /// /// Two input shapes are recognised: /// - /// Partition<N>_<Type>/<inner> — - /// add the file to partition N's inner filesystem (delegated to its - /// ). - /// No prefix — treat the input bytes as a raw filesystem - /// image, find free space past the last existing partition, write a new - /// partition entry (FS type detected via ), - /// and copy the bytes into the new partition window. + /// PartitionN_Type/inner/path — adds the file to that partition's + /// filesystem through the filesystem's own in-place editor, working on a staged window over + /// the partition: it can neither write outside the partition nor grow past it. + /// A root-level filesystem image (recognised by content, or named + /// *.raw/*.img/PartitionN_…) — becomes a new partition in the free + /// space after the last one, 1 MiB-aligned where room allows, inside the table's usable + /// range. GPT and MBR tables only. /// + /// + /// Every edit is planned and staged before anything is written; a refusal — a filesystem with + /// no lossless in-place editor, a full volume, no free space, a full table — throws + /// and leaves the disk byte-identical. + /// /// public static bool TryAdd(Stream disk, IReadOnlyList inputs) { if (!disk.CanWrite) return false; var detection = PartitionTableDetector.Detect(disk); if (detection.Partitions.Count == 0) return false; + var targets = EnumerateTargets(disk, detection); - PartitionEditor? editor = null; + var innerAdds = new Dictionary>(); + var newPartitions = new List(); foreach (var input in inputs) { - if (input.IsDirectory) continue; - var data = input.InMemoryContent ?? File.ReadAllBytes(input.FullPath); - var name = input.ArchiveName.Replace('\\', '/'); - - if (TryParsePartitionPrefix(name, out var partIdx, out var innerPath) - && partIdx < detection.Partitions.Count) { - var part = detection.Partitions[partIdx]; - using var window = new PartitionWindowStream(disk, part.StartOffset, part.Size); - var innerDesc = InnerFsDetector.Detect(window); - if (innerDesc is not IArchiveModifiable mod) - throw new InvalidOperationException( - $"Partition {partIdx + 1} ({part.TypeName}) has no writable filesystem reader; cannot Add '{innerPath}'."); - window.Position = 0; - mod.Add(window, new[] { ArchiveInputInfo.InMemory(innerPath, data) }); + if (input.IsDirectory || string.IsNullOrEmpty(input.ArchiveName)) continue; + var target = Resolve(targets, input.ArchiveName, out var innerPath); + if (target is not null && innerPath.Length > 0) { + if (!innerAdds.TryGetValue(target, out var list)) innerAdds[target] = list = []; + list.Add(input with { ArchiveName = innerPath }); continue; } - - // Root-level FS image drop → new partition past the last existing one. - editor ??= new PartitionEditor(disk); - var ptype = DetectInputPartitionType(data); - var lengthAligned = AlignUp(data.LongLength, SectorSize); - var startOffset = FindAppendOffset(editor.ListPartitions(), disk.Length, lengthAligned); - if (startOffset < 0) + if (target is not null) + throw new InvalidOperationException( + $"'{input.ArchiveName}' names an existing partition; remove it first to replace it."); + if (input.ArchiveName.Replace('\\', '/').Trim('/').Contains('/')) throw new InvalidOperationException( - $"Not enough free space at the end of the disk to land a {data.LongLength}-byte filesystem image."); - editor.AddPartition(startOffset, lengthAligned, ptype, label: null); - disk.Position = startOffset; - disk.Write(data, 0, data.Length); - // Pad the partition window with zeros to its sector-aligned length. - var pad = lengthAligned - data.Length; - if (pad > 0) { - var zeros = new byte[512]; - var remaining = pad; - while (remaining > 0) { - var chunk = (int)Math.Min(remaining, zeros.Length); - disk.Write(zeros, 0, chunk); - remaining -= chunk; - } + $"'{input.ArchiveName}' is not inside any partition of this disk."); + newPartitions.Add(input); + } + + // Stage every filesystem edit first: nothing reaches the disk until all of them succeed. + var staged = innerAdds.Select(kv => StageInnerEdit(disk, kv.Key, "Add", + (mod, window) => mod.Add(window, kv.Value))).ToList(); + var plan = newPartitions.Count > 0 ? PlanNewPartitions(disk, detection, newPartitions) : []; + + foreach (var window in staged) window.Commit(); + if (plan.Count > 0) { + var editor = new PartitionEditor(disk, detection.SectorSize); + foreach (var (input, start, length, type, label) in plan) { + WritePartitionData(disk, input, start, length); + editor.AddPartition(start, length, type, label); } } + disk.Flush(); return true; } /// - /// Partition-aware . Returns true - /// when a partition table was present and at least one entry was removed; - /// false if no partition table was detected. Entry name shapes: - /// Partition<N>_<Type>/<inner> deletes an inner-FS - /// file; Partition<N>_<Type> or - /// Partition<N>_<Type>.raw deletes the whole partition. + /// Partition-aware . Returns + /// false when no partition table exists (or the stream is read-only). Entry name shapes: + /// PartitionN_Type/inner removes a file — or a folder with everything beneath it — + /// through the partition filesystem's own in-place editor; PartitionN_Type or + /// PartitionN_Type.raw deletes the partition's table entry (GPT and MBR, including + /// logical partitions), leaving its sectors as they were. /// + /// + /// A name that addresses nothing, a filesystem without a lossless in-place editor, or a + /// whole-partition delete on a scheme the editor cannot rewrite is refused with the disk left + /// byte-identical. + /// public static bool TryRemove(Stream disk, string[] entryNames) { if (!disk.CanWrite) return false; var detection = PartitionTableDetector.Detect(disk); if (detection.Partitions.Count == 0) return false; - - PartitionEditor? editor = null; - foreach (var rawName in entryNames) { - var name = rawName.Replace('\\', '/'); - - if (TryParsePartitionPrefix(name, out var partIdx, out var innerPath) - && partIdx < detection.Partitions.Count) { - var part = detection.Partitions[partIdx]; - using var window = new PartitionWindowStream(disk, part.StartOffset, part.Size); - var innerDesc = InnerFsDetector.Detect(window); - if (innerDesc is IArchiveModifiable mod) { - window.Position = 0; - mod.Remove(window, new[] { innerPath }); - } + var targets = EnumerateTargets(disk, detection); + + var innerRemoves = new Dictionary>(); + var deletions = new List(); + foreach (var rawName in entryNames ?? []) { + var target = Resolve(targets, rawName, out var innerPath) + ?? throw new FileNotFoundException($"'{rawName}' is not on this disk.", rawName); + if (innerPath.Length > 0) { + if (!innerRemoves.TryGetValue(target, out var list)) innerRemoves[target] = list = []; + list.Add(innerPath); continue; } + if (detection.Scheme is not ("GPT" or "MBR") || target.IsBsdSlice) + throw new InvalidOperationException( + $"Deleting '{target.Prefix}' would need a {(target.IsBsdSlice ? "BSD disklabel" : detection.Scheme)} table writer; only GPT and MBR entries can be deleted."); + if (!deletions.Contains(target)) deletions.Add(target); + } - // Whole-partition delete: "Partition3_FAT12" or "Partition3_FAT12.raw" - var stripped = name.EndsWith(".raw", StringComparison.OrdinalIgnoreCase) ? name[..^4] : name; - if (TryParsePartitionIndex(stripped, out var idx) && idx < detection.Partitions.Count) { - editor ??= new PartitionEditor(disk); - editor.DeletePartition(idx); + var staged = innerRemoves.Select(kv => StageInnerEdit(disk, kv.Key, "Remove", + (mod, window) => mod.Remove(window, [.. kv.Value]))).ToList(); + + PartitionEditor? editor = null; + var editorIndices = new List(); + if (deletions.Count > 0) { + editor = new PartitionEditor(disk, detection.SectorSize); + foreach (var target in deletions) { + var entry = editor.ListPartitions().FirstOrDefault(e => + e.StartOffset == target.Partition.StartOffset && e.Size == target.Partition.Size + && e.Source != "MBR (Extended Container)") + ?? throw new InvalidOperationException($"'{target.Prefix}' has no entry the partition editor can delete."); + editorIndices.Add(entry.Index); } } + + foreach (var window in staged) window.Commit(); + if (editor is not null) + foreach (var index in editorIndices.OrderDescending()) + editor.DeletePartition(index); + disk.Flush(); return true; } - private static bool TryParsePartitionPrefix(string entryName, out int partitionIndex, out string innerPath) { - partitionIndex = -1; - innerPath = string.Empty; - var slash = entryName.IndexOf('/'); - if (slash <= 0) return false; - var head = entryName[..slash]; - if (!TryParsePartitionIndex(head, out partitionIndex)) return false; - innerPath = entryName[(slash + 1)..]; - return innerPath.Length > 0; + /// + /// Runs a filesystem edit against a staged window over . The edit + /// must go through the filesystem's own in-place editor; one that would only re-create the + /// volume is refused, as is one that throws. Nothing is written until the caller commits. + /// + private static StagedWindowStream StageInnerEdit(Stream disk, PartitionTarget target, string method, + Action edit) { + var window = new StagedWindowStream(disk, target.Offset, target.Size); + var descriptor = InnerFsDetector.Detect(window); + if (descriptor is not IArchiveModifiable modifier) + throw new InvalidOperationException( + $"{target.Prefix} holds {(descriptor is null ? "no filesystem CompressionWorkbench recognises" : $"a {descriptor.DisplayName} volume CompressionWorkbench cannot edit")}; nothing was changed."); + if (!HasNativeEditor(modifier, method)) + throw new InvalidOperationException( + $"{target.Prefix}: {descriptor.DisplayName} can only be edited by re-creating the volume, which would not preserve it; nothing was changed."); + try { + window.Position = 0; + edit(modifier, window); + } catch (Exception ex) when (ex is IOException or NotSupportedException or InvalidDataException + or ArgumentException or InvalidOperationException) { + throw new InvalidOperationException($"{target.Prefix} ({descriptor.DisplayName}) refused the edit: {ex.Message}", ex); + } + return window; } - private static bool TryParsePartitionIndex(string head, out int zeroBasedIndex) { - zeroBasedIndex = -1; - if (!head.StartsWith("Partition", StringComparison.Ordinal)) return false; - var us = head.IndexOf('_'); - if (us <= 9) return false; - if (!int.TryParse(head[9..us], out var oneBased) || oneBased <= 0) return false; - zeroBasedIndex = oneBased - 1; - return true; + private static List<(ArchiveInputInfo Input, long Start, long Length, PartitionType Type, string? Label)> PlanNewPartitions( + Stream disk, PartitionTableDetector.DetectionResult detection, List inputs) { + if (detection.Scheme is not ("GPT" or "MBR")) + throw new InvalidOperationException($"New partitions can only be added to a GPT or MBR table, not {detection.Scheme}."); + + var probe = new PartitionEditor(new ReadOnlyProbe(disk), detection.SectorSize); + var sectorSize = probe.LogicalSectorSize; + var (usableStart, usableEnd) = probe.UsableRange; + var existing = probe.ListPartitions(); + var primaries = existing.Count(e => e.Source != "EBR"); + var capacity = probe.Scheme == PartitionScheme.Gpt ? 128 : 4; + if ((probe.Scheme == PartitionScheme.Gpt ? existing.Count : primaries) + inputs.Count > capacity) + throw new InvalidOperationException( + $"The {detection.Scheme} table has room for {capacity} partitions; adding {inputs.Count} would exceed it."); + + var cursor = Math.Max(usableStart, existing.Count == 0 ? 0 : existing.Max(e => e.StartOffset + e.Size)); + var plan = new List<(ArchiveInputInfo, long, long, PartitionType, string?)>(); + foreach (var input in inputs) { + using var content = OpenInput(input); + var type = DetectInputPartitionType(content); + var nameLooksLikeImage = Path.GetFileName(input.ArchiveName) is var leaf + && (leaf.EndsWith(".raw", StringComparison.OrdinalIgnoreCase) || leaf.EndsWith(".img", StringComparison.OrdinalIgnoreCase) + || leaf.StartsWith("Partition", StringComparison.Ordinal)); + if (type == PartitionType.Unknown && !nameLooksLikeImage) + throw new InvalidOperationException( + $"'{input.ArchiveName}' is neither a filesystem image nor a raw partition image (*.raw/*.img); a disk holds partitions, not loose files."); + if (content.Length <= 0) + throw new InvalidOperationException($"'{input.ArchiveName}' is empty; a partition needs at least one sector."); + + var length = AlignUp(content.Length, sectorSize); + var aligned = AlignUp(cursor, MiB); + var start = aligned + length <= usableEnd ? aligned : AlignUp(cursor, sectorSize); + if (start + length > usableEnd) + throw new InvalidOperationException( + $"Not enough free space after the last partition for '{input.ArchiveName}' ({content.Length:N0} bytes); nothing was changed."); + plan.Add((input, start, length, type == PartitionType.Unknown ? PartitionType.Linux : type, + Path.GetFileNameWithoutExtension(input.ArchiveName))); + cursor = start + length; + } + return plan; } - private const int SectorSize = 512; + private const long MiB = 1024 * 1024; - private static long AlignUp(long value, int alignment) - => (value + alignment - 1) / alignment * alignment; + private static Stream OpenInput(ArchiveInputInfo input) + => input.InMemoryContent is { } bytes + ? new MemoryStream(bytes, writable: false) + : new FileStream(input.FullPath, FileMode.Open, FileAccess.Read, FileShare.Read); - private static long FindAppendOffset(IReadOnlyList existing, long diskLength, long lengthBytes) { - long highest = SectorSize; // skip MBR - foreach (var e in existing) { - var end = e.StartOffset + e.Size; - if (end > highest) highest = end; - } - highest = AlignUp(highest, SectorSize); - return highest + lengthBytes <= diskLength ? highest : -1; + private static void WritePartitionData(Stream disk, ArchiveInputInfo input, long start, long length) { + using var content = OpenInput(input); + disk.Position = start; + content.CopyTo(disk); + var pad = length - content.Length; + if (pad > 0) disk.Write(new byte[pad]); } + /// A view that satisfies 's writable-stream check but refuses writes — for planning only. + private sealed class ReadOnlyProbe(Stream inner) : Stream { + public override bool CanRead => true; + public override bool CanSeek => true; + public override bool CanWrite => true; + public override long Length => inner.Length; + public override long Position { get => inner.Position; set => inner.Position = value; } + public override void Flush() { } + public override int Read(byte[] buffer, int offset, int count) => inner.Read(buffer, offset, count); + public override long Seek(long offset, SeekOrigin origin) => inner.Seek(offset, origin); + public override void SetLength(long value) => throw new NotSupportedException(); + public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException("Planning view."); + } + + private static long AlignUp(long value, long alignment) + => (value + alignment - 1) / alignment * alignment; + /// /// Maps the input bytes to a logical by running - /// over a wrapping memory stream and translating - /// the resulting descriptor ID. + /// over the input and translating the resulting descriptor ID; + /// when no filesystem is recognised. /// - private static PartitionType DetectInputPartitionType(byte[] data) { + private static PartitionType DetectInputPartitionType(Stream data) { if (data.Length < 512) return PartitionType.Unknown; - using var ms = new MemoryStream(data, writable: false); - var desc = InnerFsDetector.Detect(ms); + var desc = InnerFsDetector.Detect(data); if (desc is null) return PartitionType.Unknown; return desc.Id switch { "Fat" => PartitionType.Fat32Lba, diff --git a/Compression.Core/DiskImage/SgiDvhParser.cs b/Compression.Core/DiskImage/SgiDvhParser.cs new file mode 100644 index 000000000..581104172 --- /dev/null +++ b/Compression.Core/DiskImage/SgiDvhParser.cs @@ -0,0 +1,86 @@ +using System.Buffers.Binary; + +namespace Compression.Core.DiskImage; + +/// +/// Parses the SGI disk volume header (DVH) in sector 0 — the IRIX partition scheme. Big-endian. +/// +/// +/// +/// The header opens with the magic 0x0BE5A941 and carries a checksum chosen so that the +/// 32-bit sum of all 128 big-endian longwords of the 512-byte sector is zero. Sixteen partition +/// records at byte 312 each hold a length and a first block (512-byte blocks) and a type. The +/// volume partition (type 6, conventionally slot 10) covers the whole disk and is not listed; +/// the volume-header partition (type 0) is, since it holds the boot files. +/// +/// References: +/// +/// Silicon Graphics, IRIX sys/dvh.h — struct volume_header, VHMAGIC, partition types, checksum rule. +/// +/// +public static class SgiDvhParser { + + private const int LabelSize = 512; + private const uint Magic = 0x0BE5A941; + private const int PartitionsOffset = 312; + private const int PartitionCount = 16; + private const int PartitionRecordSize = 12; + private const uint VolumeType = 6; + + /// Reports whether is an SGI volume header with a valid checksum. + public static bool IsDvh(ReadOnlySpan sector0) { + if (sector0.Length < LabelSize || BinaryPrimitives.ReadUInt32BigEndian(sector0) != Magic) return false; + uint sum = 0; + for (var i = 0; i < LabelSize; i += 4) + sum += BinaryPrimitives.ReadUInt32BigEndian(sector0[i..]); + return sum == 0; + } + + /// Parses the header's partitions, excluding the whole-volume partition and empty slots. + /// Sector 0 is not an SGI volume header. + public static List Parse(ReadOnlySpan sector0) { + if (!IsDvh(sector0)) throw new InvalidDataException("Sector 0 is not a checksummed SGI volume header."); + + var result = new List(); + for (var i = 0; i < PartitionCount; ++i) { + var record = sector0[(PartitionsOffset + i * PartitionRecordSize)..]; + var blocks = BinaryPrimitives.ReadUInt32BigEndian(record); + var first = BinaryPrimitives.ReadUInt32BigEndian(record[4..]); + var type = BinaryPrimitives.ReadUInt32BigEndian(record[8..]); + if (blocks == 0 || type == VolumeType) continue; + + result.Add(new PartitionEntry { + Index = result.Count, + StartOffset = (long)first * LabelSize, + Size = (long)blocks * LabelSize, + TypeName = TypeName(type), + TypeCode = $"0x{type:X2}", + Name = i.ToString(System.Globalization.CultureInfo.InvariantCulture), + Source = "SGI" + }); + } + return result; + } + + private static string TypeName(uint type) => type switch { + 0x00 => "SGI volhdr", + 0x01 => "SGI trkrepl", + 0x02 => "SGI secrepl", + 0x03 => "SGI raw", + 0x04 => "SGI bsd", + 0x05 => "SGI sysv", + 0x06 => "SGI volume", + 0x07 => "SGI efs", + 0x08 => "SGI lvol", + 0x09 => "SGI rlvol", + 0x0A => "SGI xfs", + 0x0B => "SGI xfslog", + 0x0C => "SGI xlv", + 0x0D => "SGI xvm", + 0x82 => "Linux swap", + 0x83 => "Linux native", + 0x8E => "Linux LVM", + 0xFD => "Linux RAID", + _ => $"SGI type 0x{type:X2}" + }; +} diff --git a/Compression.Core/DiskImage/StagedWindowStream.cs b/Compression.Core/DiskImage/StagedWindowStream.cs new file mode 100644 index 000000000..62b53b8c2 --- /dev/null +++ b/Compression.Core/DiskImage/StagedWindowStream.cs @@ -0,0 +1,153 @@ +namespace Compression.Core.DiskImage; + +/// +/// A writable, fixed-length window over a range of a disk whose writes are held back until +/// . Reads see the staged writes on top of the disk's bytes, so a +/// filesystem editor working through the window behaves exactly as it would on a standalone +/// image; but nothing reaches the disk unless the whole edit finishes and the caller commits. +/// An editor that throws half-way, or that tries to grow its volume past the partition, leaves +/// the disk byte-identical. +/// +/// +/// Writes are staged in 4 KiB pages, so the memory held is proportional to what the edit +/// touches, not to the partition. +/// +public sealed class StagedWindowStream : Stream { + private const int PageSize = 4096; + + private readonly Stream _disk; + private readonly long _offset; + private readonly long _length; + private readonly Dictionary _pages = []; + private long _position; + private bool _committed; + + /// Opens a staged window of bytes at on . + public StagedWindowStream(Stream disk, long offset, long length) { + ArgumentNullException.ThrowIfNull(disk); + if (!disk.CanSeek || !disk.CanRead) throw new ArgumentException("The disk must be readable and seekable.", nameof(disk)); + ArgumentOutOfRangeException.ThrowIfNegative(offset); + ArgumentOutOfRangeException.ThrowIfNegative(length); + this._disk = disk; + this._offset = offset; + this._length = length; + } + + /// Whether any write has been staged. + public bool HasChanges => this._pages.Count > 0; + + /// + public override bool CanRead => true; + /// + public override bool CanSeek => true; + /// + public override bool CanWrite => !this._committed; + /// + public override long Length => this._length; + + /// + public override long Position { + get => this._position; + set { + ArgumentOutOfRangeException.ThrowIfNegative(value); + this._position = value; + } + } + + /// + public override int Read(byte[] buffer, int offset, int count) => this.Read(buffer.AsSpan(offset, count)); + + /// + public override int Read(Span buffer) { + if (this._position >= this._length) return 0; + var total = (int)Math.Min(buffer.Length, this._length - this._position); + var done = 0; + while (done < total) { + var pos = this._position + done; + var pageIndex = pos / PageSize; + var inPage = (int)(pos % PageSize); + var chunk = Math.Min(total - done, PageSize - inPage); + if (this._pages.TryGetValue(pageIndex, out var page)) { + page.AsSpan(inPage, chunk).CopyTo(buffer.Slice(done, chunk)); + } else { + this._disk.Position = this._offset + pos; + var read = this._disk.ReadAtLeast(buffer.Slice(done, chunk), chunk, throwOnEndOfStream: false); + if (read < chunk) buffer.Slice(done + read, chunk - read).Clear(); + } + done += chunk; + } + this._position += total; + return total; + } + + /// + public override void Write(byte[] buffer, int offset, int count) => this.Write(buffer.AsSpan(offset, count)); + + /// + public override void Write(ReadOnlySpan buffer) { + if (this._committed) throw new InvalidOperationException("The staged window has already been committed."); + if (this._position + buffer.Length > this._length) + throw new IOException( + $"Write at {this._position} for {buffer.Length} bytes runs past the end of the {this._length}-byte partition."); + var done = 0; + while (done < buffer.Length) { + var pos = this._position + done; + var pageIndex = pos / PageSize; + var inPage = (int)(pos % PageSize); + var chunk = Math.Min(buffer.Length - done, PageSize - inPage); + buffer.Slice(done, chunk).CopyTo(this.PageFor(pageIndex).AsSpan(inPage, chunk)); + done += chunk; + } + this._position += buffer.Length; + } + + private byte[] PageFor(long pageIndex) { + if (this._pages.TryGetValue(pageIndex, out var page)) return page; + page = new byte[PageSize]; + var pageStart = pageIndex * PageSize; + var valid = (int)Math.Min(PageSize, this._length - pageStart); + this._disk.Position = this._offset + pageStart; + _ = this._disk.ReadAtLeast(page.AsSpan(0, valid), valid, throwOnEndOfStream: false); + this._pages[pageIndex] = page; + return page; + } + + /// + public override long Seek(long offset, SeekOrigin origin) { + this.Position = origin switch { + SeekOrigin.Begin => offset, + SeekOrigin.Current => this._position + offset, + SeekOrigin.End => this._length + offset, + _ => throw new ArgumentOutOfRangeException(nameof(origin)), + }; + return this._position; + } + + /// + /// A partition cannot grow from inside: asking for more than its length fails, and asking for + /// less leaves the trailing bytes where they are, as unused space inside the partition. + /// + public override void SetLength(long value) { + if (value > this._length) + throw new IOException($"The filesystem asked for {value} bytes, but its partition holds {this._length}."); + } + + /// Staged writes stay staged; only writes them. + public override void Flush() { } + + /// Writes every staged page to the disk, in disk order, then flushes it. + public void Commit() { + if (this._committed) return; + if (this._pages.Count > 0 && !this._disk.CanWrite) + throw new NotSupportedException("The disk is read-only."); + foreach (var (pageIndex, page) in this._pages.OrderBy(p => p.Key)) { + var pageStart = pageIndex * PageSize; + var valid = (int)Math.Min(PageSize, this._length - pageStart); + this._disk.Position = this._offset + pageStart; + this._disk.Write(page, 0, valid); + } + this._disk.Flush(); + this._committed = true; + this._pages.Clear(); + } +} diff --git a/Compression.Core/DiskImage/SunVtocParser.cs b/Compression.Core/DiskImage/SunVtocParser.cs new file mode 100644 index 000000000..4896b639e --- /dev/null +++ b/Compression.Core/DiskImage/SunVtocParser.cs @@ -0,0 +1,100 @@ +using System.Buffers.Binary; + +namespace Compression.Core.DiskImage; + +/// +/// Parses the Sun disk label (SunOS / Solaris SPARC "VTOC" label) in sector 0. Big-endian. +/// +/// +/// +/// The 512-byte label ends in the magic 0xDABE at byte 508 and a checksum at 510 chosen +/// so that the XOR of all 256 big-endian 16-bit words is zero. Eight partition records at byte +/// 444 hold a start cylinder and a length in sectors; a cylinder is ntrks × nsect +/// sectors, from the geometry fields at bytes 436 and 438. The VTOC at byte 128 tags each +/// slice (root, swap, usr, whole disk, Linux native, …) and is recognised by its sanity word +/// 0x600DDEEE; a label without one still describes its partitions. +/// +/// +/// The whole-disk slice (tag 5, conventionally slice 2) overlaps every other slice and is not +/// listed. +/// +/// References: +/// +/// Sun Microsystems, sys/dklabel.h — struct dk_label layout, DKL_MAGIC, checksum rule. +/// Sun Microsystems, sys/vtoc.h — slice tag values (V_BOOT, V_ROOT, V_BACKUP, …) and VTOC_SANE. +/// +/// +public static class SunVtocParser { + + private const int LabelSize = 512; + private const ushort Magic = 0xDABE; + private const int MagicOffset = 508; + private const uint VtocSane = 0x600DDEEE; + private const int VtocSanityOffset = 188; + private const int VtocInfosOffset = 142; + private const int NtrksOffset = 436; + private const int NsectOffset = 438; + private const int PartitionsOffset = 444; + private const int PartitionCount = 8; + private const ushort WholeDiskTag = 5; + + /// Reports whether is a Sun label with a valid checksum. + public static bool IsSunLabel(ReadOnlySpan sector0) { + if (sector0.Length < LabelSize) return false; + if (BinaryPrimitives.ReadUInt16BigEndian(sector0[MagicOffset..]) != Magic) return false; + ushort xor = 0; + for (var i = 0; i < LabelSize; i += 2) + xor ^= BinaryPrimitives.ReadUInt16BigEndian(sector0[i..]); + return xor == 0; + } + + /// Parses the label's slices, excluding the whole-disk slice and empty slots. + /// Sector 0 is not a Sun label. + public static List Parse(ReadOnlySpan sector0) { + if (!IsSunLabel(sector0)) throw new InvalidDataException("Sector 0 is not a checksummed Sun disk label."); + + var ntrks = BinaryPrimitives.ReadUInt16BigEndian(sector0[NtrksOffset..]); + var nsect = BinaryPrimitives.ReadUInt16BigEndian(sector0[NsectOffset..]); + var cylinderSectors = (long)ntrks * nsect; + var hasVtoc = BinaryPrimitives.ReadUInt32BigEndian(sector0[VtocSanityOffset..]) == VtocSane; + + var result = new List(); + for (var i = 0; i < PartitionCount; ++i) { + var record = sector0[(PartitionsOffset + i * 8)..]; + var startCylinder = BinaryPrimitives.ReadUInt32BigEndian(record); + var sectors = BinaryPrimitives.ReadUInt32BigEndian(record[4..]); + if (sectors == 0) continue; + + var tag = hasVtoc ? BinaryPrimitives.ReadUInt16BigEndian(sector0[(VtocInfosOffset + i * 4)..]) : (ushort)0; + if (tag == WholeDiskTag || (!hasVtoc && i == 2)) continue; + + result.Add(new PartitionEntry { + Index = result.Count, + StartOffset = startCylinder * cylinderSectors * LabelSize, + Size = (long)sectors * LabelSize, + TypeName = TagName(tag), + TypeCode = $"0x{tag:X2}", + Name = ((char)('a' + i)).ToString(), + Source = "Sun" + }); + } + return result; + } + + private static string TagName(ushort tag) => tag switch { + 0x00 => "Unassigned", + 0x01 => "Boot", + 0x02 => "SunOS root", + 0x03 => "SunOS swap", + 0x04 => "SunOS usr", + 0x05 => "Whole disk", + 0x06 => "SunOS stand", + 0x07 => "SunOS var", + 0x08 => "SunOS home", + 0x82 => "Linux swap", + 0x83 => "Linux native", + 0x8E => "Linux LVM", + 0xFD => "Linux raid autodetect", + _ => $"Sun tag 0x{tag:X2}" + }; +} diff --git a/Compression.Core/README.md b/Compression.Core/README.md index 483e87991..e918da817 100644 --- a/Compression.Core/README.md +++ b/Compression.Core/README.md @@ -205,7 +205,7 @@ Use the concrete version you intend to consume; this document does not predict a -Every public and protected member of all 728 types, generated from the built assembly and its XML documentation, is in [REFERENCE.md](https://github.com/Hawkynt/CompressionWorkbench/blob/main/Compression.Core/REFERENCE.md). +Every public and protected member of all 734 types, generated from the built assembly and its XML documentation, is in [REFERENCE.md](https://github.com/Hawkynt/CompressionWorkbench/blob/main/Compression.Core/REFERENCE.md). diff --git a/Compression.Core/REFERENCE.md b/Compression.Core/REFERENCE.md index ac9593fae..86510a075 100644 --- a/Compression.Core/REFERENCE.md +++ b/Compression.Core/REFERENCE.md @@ -3609,7 +3609,16 @@ Implements `IBuildingBlock`. ### Namespace `Compression.Core.DiskImage` -[`ApmParser`](#apmparser) · [`BsdDisklabelParser`](#bsddisklabelparser) · [`ClusterAllocator`](#clusterallocator) · [`DeferredPayloads`](#deferredpayloads) · [`ExtBlockGroupGeometry`](#extblockgroupgeometry) · [`ExtBlockGroupGeometry.ExtGeometry`](#extblockgroupgeometryextgeometry) · [`ExtentCopy`](#extentcopy) · [`FilePayload`](#filepayload) · [`GptParser`](#gptparser) · [`ImageAccessor`](#imageaccessor) · [`MbrParser`](#mbrparser) · [`MbrWrapper`](#mbrwrapper) · [`MbrWrapper.PartitionType`](#mbrwrapperpartitiontype) · [`PartitionEditor`](#partitioneditor) · [`PartitionEntry`](#partitionentry) · [`PartitionScheme`](#partitionscheme) · [`PartitionTableDetector`](#partitiontabledetector) · [`PartitionTableDetector.DetectionResult`](#partitiontabledetectordetectionresult) · [`PartitionTableVerification`](#partitiontableverification) · [`PartitionType`](#partitiontype) · [`PartitionTypeDatabase`](#partitiontypedatabase) · [`PartitionTypeMapping`](#partitiontypemapping) · [`PartitionWindowStream`](#partitionwindowstream) · [`PartitionedDiskLister`](#partitioneddisklister) · [`SparseBlockImage`](#sparseblockimage) +[`AmigaRdbParser`](#amigardbparser) · [`ApmParser`](#apmparser) · [`BsdDisklabelParser`](#bsddisklabelparser) · [`ClusterAllocator`](#clusterallocator) · [`DeferredPayloads`](#deferredpayloads) · [`ExtBlockGroupGeometry`](#extblockgroupgeometry) · [`ExtBlockGroupGeometry.ExtGeometry`](#extblockgroupgeometryextgeometry) · [`ExtentCopy`](#extentcopy) · [`FilePayload`](#filepayload) · [`GptParser`](#gptparser) · [`GptParser.HeaderInfo`](#gptparserheaderinfo) · [`ImageAccessor`](#imageaccessor) · [`MbrParser`](#mbrparser) · [`MbrWrapper`](#mbrwrapper) · [`MbrWrapper.PartitionType`](#mbrwrapperpartitiontype) · [`PartitionEditor`](#partitioneditor) · [`PartitionEntry`](#partitionentry) · [`PartitionScheme`](#partitionscheme) · [`PartitionTableDetector`](#partitiontabledetector) · [`PartitionTableDetector.DetectionResult`](#partitiontabledetectordetectionresult) · [`PartitionTableVerification`](#partitiontableverification) · [`PartitionType`](#partitiontype) · [`PartitionTypeDatabase`](#partitiontypedatabase) · [`PartitionTypeMapping`](#partitiontypemapping) · [`PartitionWindowStream`](#partitionwindowstream) · [`PartitionedDiskLister`](#partitioneddisklister) · [`PartitionedDiskLister.PartitionTarget`](#partitioneddisklisterpartitiontarget) · [`SgiDvhParser`](#sgidvhparser) · [`SparseBlockImage`](#sparseblockimage) · [`StagedWindowStream`](#stagedwindowstream) · [`SunVtocParser`](#sunvtocparser) + +#### `AmigaRdbParser` + +Parses the Amiga Rigid Disk Block (RDB) partition scheme. Big-endian throughout. + +| Member | Signature | Summary | +| --- | --- | --- | +| `IsRdb` | `static bool IsRdb(Stream disk)` | Reports whether a checksummed `RDSK` block sits in the first 16 blocks. | +| `Parse` | `static List Parse(Stream disk)` | Parses every partition the RDB's `PART` chain describes. | #### `ApmParser` @@ -3627,6 +3636,7 @@ Parses the 4.4BSD `disklabel` that FreeBSD/NetBSD/OpenBSD write into a BSD slice | Member | Signature | Summary | | --- | --- | --- | | `IsDisklabel` | `static bool IsDisklabel(Stream slice)` | Reports whether a BSD disklabel magic is present at either candidate offset within the given slice stream. | +| `IsStandaloneDisklabel` | `static bool IsStandaloneDisklabel(Stream disk)` | Reports whether `disk` carries a disklabel that stands on its own — no MBR slice around it, as a "dangerously dedicated" FreeBSD disk or a NetBSD/OpenBSD disk on a non-PC port writes it. With nothing else vouching for the bytes the label must prove itself structurally: `d_magic` and `d_magic2` both present 132 bytes apart, a sector size that is a power of two in [512, 8192], a partition count within `MAXPARTITIONS`, and every used partition starting inside the disk. | | `Parse` | `static List Parse(Stream slice, long sliceStartByteOffset = 0)` | Parses the BSD disklabel contained in `slice` and returns its filesystem partitions. | #### `ClusterAllocator` @@ -3716,13 +3726,39 @@ Implements `IEquatable`. #### `GptParser` -Parses GPT (GUID Partition Table) headers and partition entries. GPT header is at LBA 1 (offset 512 for 512-byte sectors), with partition entries starting at LBA 2+. +Parses GPT (GUID Partition Table) headers and partition entries. + +| Member | Signature | Summary | +| --- | --- | --- | +| `ProbeBytes` | `const int ProbeBytes` | Bytes of the image needed to see LBA 1 at every supported block size. | +| `SupportedSectorSizes` | `static IReadOnlyList SupportedSectorSizes { get; }` | Logical block sizes probed for the header at LBA 1, in probe order. | +| `DetectSectorSize` | `static int DetectSectorSize(ReadOnlySpan data)` | The logical block size for which LBA 1 holds a valid GPT header, or 0 when none does. | +| `HasValidHeader` | `static bool HasValidHeader(ReadOnlySpan data)` | Checks that LBA 1 holds a GPT header that is internally consistent, not merely the "EFI PART" signature, at any supported block size: see `HasValidHeader`. | +| `HasValidHeader` | `static bool HasValidHeader(ReadOnlySpan data, int sectorSize)` | Checks that LBA 1, counted in `sectorSize`-byte blocks, holds a GPT header that is internally consistent: the header size lies in [92, block size], the header's own CRC32 matches (computed with the CRC field zeroed, UEFI 2.10 §5.3.2), it names itself as LBA 1 and its partition entries are 128 x 2^n bytes each, as the specification requires. | +| `IsGpt` | `static bool IsGpt(ReadOnlySpan data)` | Checks whether LBA 1 carries the "EFI PART" signature at any supported block size. | +| `Parse` | `static List Parse(Stream diskData)` | Parses all partitions from a GPT disk image, at whichever block size its header validates. | +| `Parse` | `static List Parse(Stream diskData, out HeaderInfo info)` | Parses all partitions and reports the header that described them. | +| `ReadHeader` | `static HeaderInfo ReadHeader(Stream diskData)` | The partition table's layout facts, or `null` when the image carries no usable GPT. Uses the same header `Parse` would. | + +#### `GptParser.HeaderInfo` + +Layout facts read from the header that was used to parse the table. + +Implements `IEquatable`. | Member | Signature | Summary | | --- | --- | --- | -| `HasValidHeader` | `static bool HasValidHeader(ReadOnlySpan data)` | Checks that LBA 1 holds a GPT header that is internally consistent, not merely the "EFI PART" signature: the header size lies in [92, 512], the header's own CRC32 matches (computed with the CRC field zeroed, UEFI 2.10 §5.3.2), it names itself as LBA 1 and its partition entries are 128 x 2^n bytes each, as the specification requires. | -| `IsGpt` | `static bool IsGpt(ReadOnlySpan data)` | Checks whether the given data contains a valid GPT header at LBA 1. | -| `Parse` | `static List Parse(Stream diskData)` | Parses all partitions from a GPT disk image. | +| `HeaderInfo` | `HeaderInfo(int SectorSize, ulong MyLba, ulong AlternateLba, ulong FirstUsableLba, ulong LastUsableLba, Guid DiskGuid, ulong EntriesLba, uint EntryCount, uint EntrySize, bool FromBackup)` | Layout facts read from the header that was used to parse the table. | +| `AlternateLba` | `ulong AlternateLba { get; init; }` | The LBA of the other header. | +| `DiskGuid` | `Guid DiskGuid { get; init; }` | The disk GUID. | +| `EntriesLba` | `ulong EntriesLba { get; init; }` | Starting LBA of the entry array the used header points at. | +| `EntryCount` | `uint EntryCount { get; init; }` | Number of entries in the array. | +| `EntrySize` | `uint EntrySize { get; init; }` | Size of one entry in bytes. | +| `FirstUsableLba` | `ulong FirstUsableLba { get; init; }` | First LBA a partition may occupy. | +| `FromBackup` | `bool FromBackup { get; init; }` | `true` when the primary was unusable and the backup was read. | +| `LastUsableLba` | `ulong LastUsableLba { get; init; }` | Last LBA a partition may occupy. | +| `MyLba` | `ulong MyLba { get; init; }` | The LBA the used header sits at (1 for the primary). | +| `SectorSize` | `int SectorSize { get; init; }` | Logical block size every LBA is counted in. | #### `ImageAccessor` @@ -3749,14 +3785,14 @@ Implements `IDisposable`. #### `MbrParser` -Parses MBR (Master Boot Record) partition tables, including extended/logical partition chains. The MBR sits at LBA 0 (offset 0) of a disk image, with 4 primary partition entries at offset 0x1BE. +Parses MBR (Master Boot Record) partition tables, including extended/logical partition chains. The MBR sits at LBA 0 (offset 0) of a disk image, with 4 primary partition entries at offset 0x1BE. The entries count in logical blocks: 512 bytes on a classic or 512e disk, 4096 on a 4Kn disk. | Member | Signature | Summary | | --- | --- | --- | -| `HasValidPartitionTable` | `static bool HasValidPartitionTable(ReadOnlySpan sector0, long diskLength)` | Checks whether sector 0 holds a partition table that is actually usable, rather than merely ending in the 0x55AA boot signature — which every FAT, NTFS and exFAT volume boot record does as well. | +| `HasValidPartitionTable` | `static bool HasValidPartitionTable(ReadOnlySpan sector0, long diskLength, int sectorSize = 512)` | Checks whether sector 0 holds a partition table that is actually usable, rather than merely ending in the 0x55AA boot signature — which every FAT, NTFS and exFAT volume boot record does as well. | | `IsMbr` | `static bool IsMbr(ReadOnlySpan data)` | Checks whether the given data starts with a valid MBR. | -| `ParsePrimary` | `static List ParsePrimary(ReadOnlySpan mbr)` | Parses the 4 primary partition entries from an MBR without following extended chains. Operates on a 512-byte buffer (no stream needed). | -| `Parse` | `static List Parse(Stream diskData)` | Parses all partitions from an MBR, including extended/logical partitions. | +| `ParsePrimary` | `static List ParsePrimary(ReadOnlySpan mbr, int sectorSize = 512)` | Parses the 4 primary partition entries from an MBR without following extended chains. Operates on a 512-byte buffer (no stream needed). | +| `Parse` | `static List Parse(Stream diskData, int sectorSize = 512)` | Parses all partitions from an MBR, including extended/logical partitions. | #### `MbrWrapper` @@ -3788,15 +3824,20 @@ Read/write partition-table editor for raw disk images. Handles MBR (4 primary en | Member | Signature | Summary | | --- | --- | --- | | `PartitionEditor` | `PartitionEditor(Stream stream)` | Opens an editor over the given disk-image stream. The stream must be readable, writable, and seekable. The scheme (MBR/GPT/None) is auto-detected from the on-disk signature. | -| `SectorSize` | `const int SectorSize` | Standard sector size used by the editor. | +| `PartitionEditor` | `PartitionEditor(Stream stream, int sectorSize)` | Opens an editor over the given disk-image stream, counting LBAs in `sectorSize`-byte blocks unless a GPT on the disk validates at another supported block size — the table on the disk decides, since rewriting it at a different size would move every partition. | +| `SectorSize` | `const int SectorSize` | The classic logical block size, and the one a disk without a GPT is assumed to use unless the caller says otherwise. A 4Kn GPT disk is edited in 4096-byte blocks: see `LogicalSectorSize`. | | `InPlaceFilesystemConverter` | `static Func InPlaceFilesystemConverter { get; set; }` | Pluggable in-place filesystem-variant converter. Set by the higher-level `Compression.Lib` layer at startup (it can reference the per-family FileSystem.* projects, which Core cannot). When non-null, `ConvertFilesystem` attempts a metadata-only conversion via this delegate before falling back to `false`. Contract: takes (partitionStream, sourceFsId, targetFsId), returns `true` if the conversion succeeded entirely via metadata edits, `false` if no in-place path exists for the pair (the caller should then do a full migration: extract files → FormatPartition → re-import). | | `InPlaceFilesystemResizer` | `static Func InPlaceFilesystemResizer { get; set; }` | Pluggable in-place filesystem-resizer. Set by the higher-level `Compression.Lib` layer at startup (Core cannot reference FileSystem.* projects, so the actual resizer lives there). The contract is: (partitionStream, fsId, newSizeBytes, isShrink) → bool Returns `true` when the resize succeeded entirely via in-place edits, `false` when no in-place path exists for the fsId (the caller should fall back to a full extract → reformat → re-import). | +| `LogicalSectorSize` | `int LogicalSectorSize { get; }` | The logical block size every LBA on this disk is counted in: the size at which the GPT header validated, or — for an MBR or blank disk — the size the editor was opened with. | | `Scheme` | `PartitionScheme Scheme { get; }` | Partition scheme currently in use on the disk image. | +| `SupportedSectorSizes` | `static IReadOnlyList SupportedSectorSizes { get; }` | Logical block sizes the editor supports. | +| `UsableRange` | `ValueTuple UsableRange { get; }` | The byte range partitions may occupy under the current scheme: everything after LBA 0 for MBR; between the primary and backup GPT structures for GPT. | | `AddLogicalPartition` | `void AddLogicalPartition(long startByteOffset, long lengthBytes, PartitionType type, string label)` | Adds a new logical partition inside the existing MBR extended container. The byte range must lie entirely within the extended container and not overlap any existing logical (each logical occupies its EBR sector plus the data range). | | `AddPartition` | `void AddPartition(long startByteOffset, long lengthBytes, PartitionType type, string label)` | Adds a new primary partition entry covering [`startByteOffset`, `startByteOffset` + `lengthBytes`). | | `ConvertFilesystem` | `bool ConvertFilesystem(int partitionIndex, string newFsId)` | Attempts an in-place filesystem-variant conversion of the partition at `partitionIndex`. Reads the partition's current contents as a substream, dispatches to `InPlaceFilesystemConverter` (set by the Compression.Lib bootstrap), and returns the converter's answer. Returns `false` when:No converter has been registered (Core-only build).The source/target pair has no metadata-only conversion path (e.g. FAT → ext, NTFS → exFAT — any cross-family conversion).The image geometry rules out the requested target (e.g. a tiny FAT12 floppy can't fit FAT32's 32-reserved-sector overhead). In any of these cases the caller is expected to fall back to a full extract → reformat → re-import migration. | | `ConvertGptToMbr` | `void ConvertGptToMbr()` | Reads the current GPT partition table, takes the first 4 entries (MBR hard limit), zero-fills both GPT header areas, and writes a fresh MBR with the translated entries. | | `ConvertMbrToGpt` | `void ConvertMbrToGpt()` | Reads the current MBR partition table, drops it, then rewrites the equivalent GPT layout: protective MBR at LBA 0, primary GPT header at LBA 1, entries starting at LBA 2, backup GPT at the end of the disk. Translates MBR type bytes to GPT GUIDs via `PartitionTypeMapping`. The extended container (if present) is dropped — its logical children are promoted to top-level GPT entries. | +| `CreateEmpty` | `static PartitionEditor CreateEmpty(Stream stream, PartitionScheme scheme, int sectorSize = 512)` | Writes an empty partition table of `scheme` over the start (and, for GPT, the end) of `stream` and returns an editor over it. A GPT gets the standard 128 entries of 128 bytes, its backup at the last LBA and a protective MBR. | | `DeletePartition` | `void DeletePartition(int index)` | Deletes the partition at the given index. The partition byte range is not modified — see `PurgePartition` to zero it. | | `FormatPartition` | `void FormatPartition(int index, string formatId, FormatCreateOptions options)` | Writes a fresh filesystem image of the given `formatId` into the partition's byte range. The format must be registered with `FormatRegistry` and support `IArchiveCreatable`. The resulting bytes must fit within the partition (an exception is thrown otherwise). | | `ListPartitions` | `IReadOnlyList ListPartitions()` | Returns a snapshot of the current partition table. | @@ -3834,14 +3875,15 @@ Partition scheme detected on a disk image. #### `PartitionTableDetector` -Detects and parses partition tables from raw disk data. Tries GPT first (since GPT disks also have a protective MBR), then falls back to MBR. +Detects and parses partition tables from raw disk data. | Member | Signature | Summary | | --- | --- | --- | -| `Detect` | `static DetectionResult Detect(Stream diskData)` | Attempts to detect a partition table in the given stream. Tries GPT first (higher precedence), then MBR. Returns an empty partition list if no valid partition table is found. | +| `Detect` | `static DetectionResult Detect(Stream diskData)` | Attempts to detect a partition table in the given stream. Returns an empty partition list if no valid partition table is found. | | `Detect` | `static DetectionResult Detect(byte[] diskData)` | Attempts to detect a partition table in the given byte array. Convenience overload that wraps the data in a `MemoryStream`. | | `ExtractPartitionData` | `static byte[] ExtractPartitionData(Stream diskData, PartitionEntry partition)` | Extracts partition data from a disk image stream. Returns the raw bytes of the partition at the given offset and size. | | `ExtractPartitionData` | `static byte[] ExtractPartitionData(byte[] diskData, PartitionEntry partition)` | Extracts partition data from a byte array. | +| `HasSelfValidatingLabel` | `static bool HasSelfValidatingLabel(Stream disk)` | Whether the first sector holds a partition table of a scheme that has no MBR-style boot signature to go on — SGI, Sun, a stand-alone BSD disklabel or an Amiga RDB — each verified by its own checksum (BSD: by its two magics and geometry). Used by content detection, which has already ruled out GPT, MBR and APM by then. | #### `PartitionTableDetector.DetectionResult` @@ -3851,7 +3893,8 @@ Result of partition table detection, including the scheme used and the discovere | --- | --- | --- | | `DetectionResult` | `DetectionResult()` | | | `Partitions` | `List Partitions { get; init; }` | Discovered partitions. Empty if no partition table was found. | -| `Scheme` | `string Scheme { get; init; }` | The partition scheme detected: "GPT", "MBR", "APM", or "None". | +| `Scheme` | `string Scheme { get; init; }` | The partition scheme detected: "GPT", "MBR", "APM", "BSD", "RDB", "Sun", "SGI", or "None". | +| `SectorSize` | `int SectorSize { get; init; }` | Logical block size the table counts in (512 unless a 4Kn GPT was found). | #### `PartitionTableVerification` @@ -3941,11 +3984,38 @@ Partition-aware archive surface for raw disk image streams (VHD/VHDX/VMDK/Qcow2/ | Member | Signature | Summary | | --- | --- | --- | +| `EnumerateTargets` | `static List EnumerateTargets(Stream disk, DetectionResult detection)` | Every partition the listing shows a prefix for, in listing order: top-level partitions, with a BSD-disklabel container replaced by its slices. | | `Extract` | `static bool Extract(Stream disk, string outputDir, string password, string[] files)` | Extracts entries across every partition into per-partition subdirectories of `outputDir`. Returns `true` if a partition table was detected and handled; `false` if no partition table was found (caller falls through to the unpartitioned path). | +| `HasNativeEditor` | `static bool HasNativeEditor(IArchiveModifiable modifier, string method)` | Whether `modifier` edits through its own code for `method` (`Add` or `Remove`) rather than the interface's generic extract → re-create default, which re-creates the volume and loses what its writer does not reproduce. | | `List` | `static List List(Stream disk, string password)` | Lists entries across every partition. Returns `null` if no partition table is present so the caller can fall through to the unpartitioned path. | | `MakePartitionPrefix` | `static string MakePartitionPrefix(PartitionEntry part)` | The directory name a partition is listed under: `Partition{N}_{TypeName}`, with N one-based and the type name reduced to letters, digits, '-' and '_'. | -| `TryAdd` | `static bool TryAdd(Stream disk, IReadOnlyList inputs)` | Partition-aware `Add`. Returns `true` when a partition table was present and at least one input was dispatched through the partition-aware path; `false` when no partition table exists so the caller can fall through to the existing single-FS path. | -| `TryRemove` | `static bool TryRemove(Stream disk, string[] entryNames)` | Partition-aware `Remove`. Returns `true` when a partition table was present and at least one entry was removed; `false` if no partition table was detected. Entry name shapes: `Partition_/` deletes an inner-FS file; `Partition_` or `Partition_.raw` deletes the whole partition. | +| `Resolve` | `static PartitionTarget Resolve(IEnumerable targets, string entryName, out string innerPath)` | Finds the partition an entry name addresses. `innerPath` is the rest of the name below the partition's prefix — empty when the name is the partition itself (`PartitionN_Type` or `PartitionN_Type.raw`). | +| `TryAdd` | `static bool TryAdd(Stream disk, IReadOnlyList inputs)` | Partition-aware `Add`. Returns `false` when no partition table exists (or the stream is read-only) so the caller can fall through to its single-filesystem path. | +| `TryRemove` | `static bool TryRemove(Stream disk, string[] entryNames)` | Partition-aware `Remove`. Returns `false` when no partition table exists (or the stream is read-only). Entry name shapes: `PartitionN_Type/inner` removes a file — or a folder with everything beneath it — through the partition filesystem's own in-place editor; `PartitionN_Type` or `PartitionN_Type.raw` deletes the partition's table entry (GPT and MBR, including logical partitions), leaving its sectors as they were. | + +#### `PartitionedDiskLister.PartitionTarget` + +One listed partition: a top-level partition, or a slice of a BSD disklabel nested inside one, with the prefix its entries are listed under. + +Implements `IEquatable`. + +| Member | Signature | Summary | +| --- | --- | --- | +| `PartitionTarget` | `PartitionTarget(PartitionEntry Partition, string Prefix, long Offset, long Size, bool IsBsdSlice)` | One listed partition: a top-level partition, or a slice of a BSD disklabel nested inside one, with the prefix its entries are listed under. | +| `IsBsdSlice` | `bool IsBsdSlice { get; init; }` | Whether this is a slice inside a BSD disklabel rather than a table entry. | +| `Offset` | `long Offset { get; init; }` | Byte offset of the partition on the disk. | +| `Partition` | `PartitionEntry Partition { get; init; }` | The table entry (for a BSD slice, the slice's own entry). | +| `Prefix` | `string Prefix { get; init; }` | The listing prefix, e.g. `Partition1_FAT32` or `Partition1_FreeBSD/Partition1_4_2BSD`. | +| `Size` | `long Size { get; init; }` | Bytes of the partition that lie inside the image. | + +#### `SgiDvhParser` + +Parses the SGI disk volume header (DVH) in sector 0 — the IRIX partition scheme. Big-endian. + +| Member | Signature | Summary | +| --- | --- | --- | +| `IsDvh` | `static bool IsDvh(ReadOnlySpan sector0)` | Reports whether `sector0` is an SGI volume header with a valid checksum. | +| `Parse` | `static List Parse(ReadOnlySpan sector0)` | Parses the header's partitions, excluding the whole-volume partition and empty slots. | #### `SparseBlockImage` @@ -3965,6 +4035,39 @@ Sparse, block-addressed image buffer. Only the blocks a writer actually touches | `WriteTo` | `void WriteTo(Stream output)` | Writes the volume from the current position, extending the stream to its full size. | | `Write` | `void Write(long offset, ReadOnlySpan data)` | Copies `data` to `offset`, spanning blocks as needed. | +#### `StagedWindowStream` + +A writable, fixed-length window over a range of a disk whose writes are held back until `Commit`. Reads see the staged writes on top of the disk's bytes, so a filesystem editor working through the window behaves exactly as it would on a standalone image; but nothing reaches the disk unless the whole edit finishes and the caller commits. An editor that throws half-way, or that tries to grow its volume past the partition, leaves the disk byte-identical. + +Inherits `Stream`. Implements `IAsyncDisposable`, `IDisposable`. + +| Member | Signature | Summary | +| --- | --- | --- | +| `StagedWindowStream` | `StagedWindowStream(Stream disk, long offset, long length)` | Opens a staged window of `length` bytes at `offset` on `disk`. | +| `CanRead` | `override bool CanRead { get; }` | | +| `CanSeek` | `override bool CanSeek { get; }` | | +| `CanWrite` | `override bool CanWrite { get; }` | | +| `HasChanges` | `bool HasChanges { get; }` | Whether any write has been staged. | +| `Length` | `override long Length { get; }` | | +| `Position` | `override long Position { get; set; }` | | +| `Commit` | `void Commit()` | Writes every staged page to the disk, in disk order, then flushes it. | +| `Flush` | `override void Flush()` | Staged writes stay staged; only `Commit` writes them. | +| `Read` | `override int Read(Span buffer)` | | +| `Read` | `override int Read(byte[] buffer, int offset, int count)` | | +| `Seek` | `override long Seek(long offset, SeekOrigin origin)` | | +| `SetLength` | `override void SetLength(long value)` | A partition cannot grow from inside: asking for more than its length fails, and asking for less leaves the trailing bytes where they are, as unused space inside the partition. | +| `Write` | `override void Write(ReadOnlySpan buffer)` | | +| `Write` | `override void Write(byte[] buffer, int offset, int count)` | | + +#### `SunVtocParser` + +Parses the Sun disk label (SunOS / Solaris SPARC "VTOC" label) in sector 0. Big-endian. + +| Member | Signature | Summary | +| --- | --- | --- | +| `IsSunLabel` | `static bool IsSunLabel(ReadOnlySpan sector0)` | Reports whether `sector0` is a Sun label with a valid checksum. | +| `Parse` | `static List Parse(ReadOnlySpan sector0)` | Parses the label's slices, excluding the whole-disk slice and empty slots. | + ### Namespace `Compression.Core.DiskImage.Raid` [`ImsmContainer`](#imsmcontainer) · [`ImsmDisk`](#imsmdisk) · [`ImsmMetadataParser`](#imsmmetadataparser) · [`ImsmVolume`](#imsmvolume) · [`Md09SuperblockParser`](#md09superblockparser) · [`Md1SuperblockParser`](#md1superblockparser) · [`RaidArray`](#raidarray) · [`RaidAssembledStream`](#raidassembledstream) · [`RaidAssembler`](#raidassembler) · [`RaidLevel`](#raidlevel) · [`RaidMember`](#raidmember) · [`RaidMemberMetadata`](#raidmembermetadata) · [`RaidMetadataFormat`](#raidmetadataformat) diff --git a/Compression.Lib/FileFormats/FileFormat.PartitionedDisk/PartitionedDiskFormatDescriptor.cs b/Compression.Lib/FileFormats/FileFormat.PartitionedDisk/PartitionedDiskFormatDescriptor.cs index d0d66502b..03d905c3a 100644 --- a/Compression.Lib/FileFormats/FileFormat.PartitionedDisk/PartitionedDiskFormatDescriptor.cs +++ b/Compression.Lib/FileFormats/FileFormat.PartitionedDisk/PartitionedDiskFormatDescriptor.cs @@ -1,4 +1,5 @@ #pragma warning disable CS1591 +using System.Text.RegularExpressions; using Compression.Core.DiskImage; using Compression.Registry; @@ -15,15 +16,25 @@ namespace FileFormat.PartitionedDisk; /// /// There is no signature to register. The format is claimed by /// FormatDetector's disk-image probe only after the table has been validated: a GPT header -/// whose CRC32 matches, a legacy MBR whose four entries all carry a legal boot indicator and -/// describe non-overlapping ranges that start inside the image, or an Apple Partition Map. No -/// extension is claimed either: .img, .raw, .dd and .bin are just -/// as often a bare filesystem or something else entirely, and it is the content that decides. +/// whose CRC32 matches (at 512- or 4096-byte logical blocks), a legacy MBR whose four entries all +/// carry a legal boot indicator and describe non-overlapping ranges that start inside the image, +/// an Apple Partition Map, or a checksummed SGI volume header, Sun label, stand-alone BSD +/// disklabel or Amiga Rigid Disk Block. No extension is claimed either: .img, +/// .raw, .dd and .bin are just as often a bare filesystem or something else +/// entirely, and it is the content that decides. /// /// /// Partitions are read through a window over the image, never copied out, so browsing a /// multi-gigabyte disk touches only the sectors each filesystem reader asks for. /// +/// +/// Editing goes through two paths. A file inside a partition is added or removed by that +/// partition's own filesystem editor, working in place on a staged window over the partition — +/// refused when the filesystem has no lossless in-place editor, and committed only when the whole +/// edit succeeds. Whole partitions are added (a root-level filesystem or *.raw image) and +/// deleted (PartitionN_Type) on GPT and MBR tables. lays partition +/// images out on a fresh GPT or MBR disk. +/// /// References: /// /// UEFI Specification 2.10, §5.2 (legacy MBR) and §5.3 (GUID Partition Table) @@ -31,7 +42,8 @@ namespace FileFormat.PartitionedDisk; /// https://en.wikipedia.org/wiki/Disk_image — overview /// /// -public sealed class PartitionedDiskFormatDescriptor : IFormatDescriptor, IArchiveFormatOperations { +public sealed class PartitionedDiskFormatDescriptor : IFormatDescriptor, IArchiveFormatOperations, IArchiveModifiable, IArchiveCreatable, + IArchiveWriteConstraints { /// Gets the id. public string Id => "PartitionedDisk"; /// Gets the display name. @@ -41,7 +53,8 @@ public sealed class PartitionedDiskFormatDescriptor : IFormatDescriptor, IArchiv /// Gets the capabilities. public FormatCapabilities Capabilities => FormatCapabilities.CanList | FormatCapabilities.CanExtract | FormatCapabilities.CanTest | - FormatCapabilities.SupportsMultipleEntries; + FormatCapabilities.CanCreate | FormatCapabilities.CanModify | + FormatCapabilities.SupportsMultipleEntries | FormatCapabilities.SupportsDirectories; /// Gets the default extension. public string DefaultExtension => ".img"; /// Gets the extensions — none: generic disk-image extensions are settled by content. @@ -58,7 +71,7 @@ public sealed class PartitionedDiskFormatDescriptor : IFormatDescriptor, IArchiv public AlgorithmFamily Family => AlgorithmFamily.Archive; /// Gets the description. public string Description => - "Raw whole-disk image (MBR, GPT or APM); each partition browses as the filesystem inside it."; + "Raw whole-disk image (GPT, MBR, APM, BSD, Amiga RDB, Sun, SGI); each partition browses as the filesystem inside it."; private const string WholeDiskEntry = "disk.raw"; @@ -94,36 +107,208 @@ public void Extract(Stream stream, string outputDir, string? password, string[]? /// /// Opens an entry without extracting the partition around it: a PartitionN_Type.raw /// entry is a read-only window over the partition's sectors, and a file inside a recognised - /// filesystem is opened by that filesystem's reader over the same window. + /// filesystem is opened by that filesystem's reader over the same window — for a BSD slice, + /// the window over the slice. /// public Stream OpenEntry(Stream archive, string entryName, string? password) { ArgumentNullException.ThrowIfNull(archive); ArgumentException.ThrowIfNullOrWhiteSpace(entryName); - var name = entryName.Replace('\\', '/'); - var table = PartitionTableDetector.Detect(archive); - foreach (var part in table.Partitions) { - var prefix = PartitionedDiskLister.MakePartitionPrefix(part); - - if (name.Equals(prefix + ".raw", StringComparison.Ordinal)) - return new ReadOnlyWindow(new PartitionWindowStream(archive, part.StartOffset, ClampedSize(archive, part))); - - if (!name.StartsWith(prefix + "/", StringComparison.Ordinal)) - continue; - - var window = new PartitionWindowStream(archive, part.StartOffset, ClampedSize(archive, part)); - if (BsdDisklabelParser.IsDisklabel(window)) - break; // slices nest one level deeper; the lister's own layout handles them + var targets = PartitionedDiskLister.EnumerateTargets(archive, PartitionTableDetector.Detect(archive)); + if (PartitionedDiskLister.Resolve(targets, entryName, out var innerPath) is { } target) { + var window = new PartitionWindowStream(archive, target.Offset, target.Size); + if (innerPath.Length == 0) + return new ReadOnlyWindow(window); if (InnerFsDetector.Detect(window) is IArchiveFormatOperations inner) { window.Position = 0; - return inner.OpenEntry(window, name[(prefix.Length + 1)..], password); + return inner.OpenEntry(window, innerPath, password); } } - // BSD slices, and anything else the lister nests deeper, take the extracting default. return ((IArchiveFormatOperations)new ExtractingView(this)).OpenEntry(archive, entryName, password); } + // ── IArchiveModifiable ───────────────────────────────────────────── + + /// + /// Adds files inside partitions (PartitionN_Type/path) through each partition + /// filesystem's own in-place editor, and root-level filesystem or *.raw images as new + /// partitions. Refusals leave the disk byte-identical: see + /// . + /// + /// The edit was refused; nothing was written. + public void Add(Stream archive, IReadOnlyList inputs) { + ArgumentNullException.ThrowIfNull(archive); + ArgumentNullException.ThrowIfNull(inputs); + if (!PartitionedDiskLister.TryAdd(archive, inputs)) + throw new InvalidOperationException("The image carries no partition table that can be edited."); + } + + /// + /// Removes files inside partitions through each partition filesystem's own in-place editor, + /// and deletes whole partitions (PartitionN_Type) from GPT and MBR tables. Refusals + /// leave the disk byte-identical: see . + /// + /// The edit was refused; nothing was written. + /// A name addresses nothing on the disk; nothing was written. + public void Remove(Stream archive, string[] entryNames) { + ArgumentNullException.ThrowIfNull(archive); + if (!PartitionedDiskLister.TryRemove(archive, entryNames ?? [])) + throw new InvalidOperationException("The image carries no partition table that can be edited."); + } + + /// + /// Not offered: emptying a whole disk would mean deleting every partition, which is a + /// partition-table decision, not a maintenance verb — each volume is maintained on its own. + /// + public bool CanPurgeToEmpty => false; + + // ── IArchiveWriteConstraints ─────────────────────────────────────── + + /// + /// Inside a partition (PartitionN_Type/…) any file goes to that partition's filesystem, + /// which has the last word. At the disk root only partition images belong: a filesystem image, + /// or a raw image named *.raw/*.img/PartitionN_…. + /// + public bool CanAccept(ArchiveInputInfo input, out string? reason) { + ArgumentNullException.ThrowIfNull(input); + reason = null; + var name = input.ArchiveName.Replace('\\', '/').Trim('/'); + if (input.IsDirectory || name.Contains('/') || IsNamedPartitionImage(name)) return true; + try { + using var content = OpenInput(input); + if (DetectPartitionType(content) is not null) return true; + } catch (Exception ex) when (ex is IOException or UnauthorizedAccessException) { + // unreadable: refuse below + } + reason = $"'{name}' is not a partition image; open a partition's folder to add files to its filesystem."; + return false; + } + + /// No ceiling of its own: each partition's filesystem bounds what it holds. + public long? MaxTotalArchiveSize => null; + + /// Summary for drop tooltips. + public string AcceptedInputsDescription => + "inside a partition folder: files for its filesystem; at the disk root: filesystem or *.raw/*.img partition images"; + + private static bool IsNamedPartitionImage(string name) { + var leaf = Path.GetFileName(name); + return leaf.EndsWith(".raw", StringComparison.OrdinalIgnoreCase) + || leaf.EndsWith(".img", StringComparison.OrdinalIgnoreCase) + || PartitionName.IsMatch(leaf); + } + + // ── IArchiveCreatable ────────────────────────────────────────────── + + private static readonly Regex PartitionName = new(@"^Partition(\d+)_", RegexOptions.CultureInvariant); + + /// + /// Writes a fresh disk holding one partition per input image. An input must be a filesystem + /// image (recognised by content) or be named as a raw partition image (*.raw, + /// *.img, or PartitionN_… as this format lists them); a loose file is refused. + /// + /// + /// + /// Layout: the first partition starts at 1 MiB and each further one at the next 1 MiB + /// boundary — the alignment sgdisk, parted and Windows use — sized to its image + /// rounded up to whole sectors. Inputs named PartitionN_… are placed in N order, + /// others in the order given. The partition type follows the image's filesystem. + /// + /// + /// Options (): scheme = gpt + /// (default, up to 128 partitions, named after the input) or mbr (up to 4 primaries, + /// below 2^32 sectors); sectorSize = 512 (default) or 4096 for a 4Kn disk. + /// + /// + /// No inputs, a loose file, or an unknown option value. + public void Create(Stream output, IReadOnlyList inputs, FormatCreateOptions options) { + ArgumentNullException.ThrowIfNull(output); + ArgumentNullException.ThrowIfNull(inputs); + options ??= new FormatCreateOptions(); + + var scheme = (options.GetString("scheme") ?? "gpt").ToLowerInvariant() switch { + "gpt" => PartitionScheme.Gpt, + "mbr" or "dos" or "msdos" => PartitionScheme.Mbr, + var other => throw new ArgumentException($"Unknown partition scheme '{other}'; use gpt or mbr.", nameof(options)), + }; + var sectorSize = int.TryParse(options.GetString("sectorSize") ?? "512", out var ss) ? ss : 0; + if (!PartitionEditor.SupportedSectorSizes.Contains(sectorSize)) + throw new ArgumentException("sectorSize must be 512 or 4096.", nameof(options)); + + var images = inputs.Where(i => !i.IsDirectory && !string.IsNullOrEmpty(i.ArchiveName)).ToList(); + if (images.Count == 0) + throw new ArgumentException("A partitioned disk needs at least one partition image.", nameof(inputs)); + if (images.All(i => PartitionName.IsMatch(Path.GetFileName(i.ArchiveName)))) + images = [.. images.OrderBy(i => int.Parse(PartitionName.Match(Path.GetFileName(i.ArchiveName)).Groups[1].Value, + System.Globalization.CultureInfo.InvariantCulture))]; + + var capacity = scheme == PartitionScheme.Gpt ? 128 : 4; + if (images.Count > capacity) + throw new ArgumentException($"A {scheme} table holds at most {capacity} partitions; {images.Count} images were given.", nameof(inputs)); + + // Plan the layout before writing anything. + const long alignment = 1024 * 1024; + var layout = new List<(ArchiveInputInfo Input, long Start, long Length, PartitionType Type)>(); + var cursor = alignment; + foreach (var image in images) { + using var content = OpenInput(image); + var type = DetectPartitionType(content); + if (type is null && !IsNamedPartitionImage(image.ArchiveName)) + throw new ArgumentException( + $"'{image.ArchiveName}' is neither a filesystem image nor a raw partition image (*.raw/*.img); a disk holds partitions, not loose files.", + nameof(inputs)); + if (content.Length == 0) + throw new ArgumentException($"'{image.ArchiveName}' is empty; a partition needs at least one sector.", nameof(inputs)); + var length = (content.Length + sectorSize - 1) / sectorSize * sectorSize; + layout.Add((image, cursor, length, type ?? PartitionType.Linux)); + cursor = (cursor + length + alignment - 1) / alignment * alignment; + } + + var lastEnd = layout[^1].Start + layout[^1].Length; + var trailer = scheme == PartitionScheme.Gpt ? (16384L + sectorSize - 1) / sectorSize * sectorSize + sectorSize : 0; + var diskLength = (lastEnd + trailer + sectorSize - 1) / sectorSize * sectorSize; + if (scheme == PartitionScheme.Mbr && diskLength / sectorSize > uint.MaxValue) + throw new ArgumentException("The images need more than 2^32 sectors, which an MBR cannot address; use scheme=gpt.", nameof(options)); + + output.SetLength(0); + output.SetLength(diskLength); + var editor = PartitionEditor.CreateEmpty(output, scheme, sectorSize); + foreach (var (input, start, length, type) in layout) { + using (var content = OpenInput(input)) { + output.Position = start; + content.CopyTo(output); + } + editor.AddPartition(start, length, type, scheme == PartitionScheme.Gpt ? GptLabel(input.ArchiveName) : null); + } + output.Flush(); + } + + /// A GPT name from the input's file name: without extension and PartitionN_ prefix, at most 36 UTF-16 units. + private static string GptLabel(string archiveName) { + var stem = Path.GetFileNameWithoutExtension(archiveName.Replace('\\', '/')); + stem = PartitionName.Replace(stem, ""); + return stem.Length > 36 ? stem[..36] : stem; + } + + private static Stream OpenInput(ArchiveInputInfo input) + => input.InMemoryContent is { } bytes + ? new MemoryStream(bytes, writable: false) + : new FileStream(input.FullPath, FileMode.Open, FileAccess.Read, FileShare.Read); + + /// The partition type an image's filesystem calls for, or null when no filesystem is recognised. + private static PartitionType? DetectPartitionType(Stream image) { + if (image.Length < 512) return null; + return InnerFsDetector.Detect(image)?.Id switch { + null => null, + "Fat" => PartitionType.Fat32Lba, + "ExFat" or "Ntfs" or "Hpfs" => PartitionType.NtfsExfat, + "HfsPlus" or "Hfs" => PartitionType.AppleHfsPlus, + "Apfs" => PartitionType.AppleApfs, + _ => PartitionType.Linux, + }; + } + /// Exposes List/Extract without the OpenEntry override, to reach the interface default. private sealed class ExtractingView(PartitionedDiskFormatDescriptor owner) : IArchiveFormatOperations { public List List(Stream stream, string? password) => owner.List(stream, password); @@ -131,9 +316,6 @@ public void Extract(Stream stream, string outputDir, string? password, string[]? => owner.Extract(stream, outputDir, password, files); } - private static long ClampedSize(Stream disk, PartitionEntry part) - => Math.Max(0, Math.Min(part.Size, disk.Length - part.StartOffset)); - /// Read-only view of a partition window: browsing must never write to the disk. private sealed class ReadOnlyWindow(Stream inner) : Stream { public override bool CanRead => inner.CanRead; diff --git a/Compression.Lib/FormatDetector.DiskImage.cs b/Compression.Lib/FormatDetector.DiskImage.cs index ab71f72b6..c5774e52f 100644 --- a/Compression.Lib/FormatDetector.DiskImage.cs +++ b/Compression.Lib/FormatDetector.DiskImage.cs @@ -64,12 +64,15 @@ private static IFormatDescriptor[] FilesystemDescriptors { /// virtual-disk containers, compressed streams, filesystems that start at byte 0; /// ISO 9660 / UDF volume descriptors at 32 KiB — ahead of any partition /// table, so a hybrid ISO whose system area carries an MBR or GPT browses as the ISO; - /// a GPT header whose CRC32 matches → ; + /// a GPT header whose CRC32 matches, at 512- or 4096-byte logical blocks → + /// ; /// sector 0 ending in 0x55AA: a validated MBR table versus a volume boot record /// (FAT BPB, NTFS/exFAT/… OEM signature). A valid table with no boot record is the disk; a boot /// record with no valid table is the filesystem; when both parse, the disk wins only if one of /// its partitions holds a recognisable filesystem; /// an Apple Partition Map → ; + /// a checksummed SGI volume header or Sun label, a stand-alone BSD disklabel (both magics) or a checksummed + /// Amiga Rigid Disk Block → ; /// when is set, a filesystem superblock /// signature (≥ 0.75 confidence) anywhere up to the deepest registered one (ZFS at 128 KiB). /// @@ -105,6 +108,8 @@ internal static Format DetectDiskImageByContent(Stream stream, bool includeSuper if (ApmParser.IsApm(head)) return Format.PartitionedDisk; + if (PartitionTableDetector.HasSelfValidatingLabel(stream)) return Format.PartitionedDisk; + if (!includeSuperblocks) return Format.Unknown; var deep = (int)Math.Min(FormatDetector._maxFilesystemMagicExtent, length); diff --git a/Compression.Lib/FormatDetector.cs b/Compression.Lib/FormatDetector.cs index 25df6b418..849934333 100644 --- a/Compression.Lib/FormatDetector.cs +++ b/Compression.Lib/FormatDetector.cs @@ -152,7 +152,7 @@ private static Format DetectByMagicFromFile(string path) { } /// Get the descriptor for a format, or null if no descriptor exists (Sfx, Iso, Udf). - private static IFormatDescriptor? GetDesc(Format f) { + internal static IFormatDescriptor? GetDesc(Format f) { EnsureRegistryMapped(); return FormatDetector._formatToDesc!.GetValueOrDefault(f); } diff --git a/Compression.Lib/NestedStreamResolver.cs b/Compression.Lib/NestedStreamResolver.cs index bafaf9f4f..bf6183c91 100644 --- a/Compression.Lib/NestedStreamResolver.cs +++ b/Compression.Lib/NestedStreamResolver.cs @@ -115,6 +115,10 @@ public void DisposeAll() { var desc = DetectFormat(current); if (desc == null) return null; + // A raw whole-disk image: the filesystems are inside its partitions, not at byte 0. + if (desc.Id == PartitionedDiskId) + return TryDescendPartitions(current, ownership, pathSegments, depth + 1); + // If it's a disk image container, try to open the virtual disk stream if (DiskImageFormatIds.Contains(desc.Id)) { var virtualStream = TryOpenVirtualDisk(current, desc.Id); @@ -155,13 +159,14 @@ public void DisposeAll() { if (detection.Scheme == "None" || detection.Partitions.Count == 0) return null; - // Try each partition, preferring the first that yields a writable FS - foreach (var part in detection.Partitions) { - if (part.Size <= 0 || part.StartOffset < 0 || part.StartOffset >= diskStream.Length) + // Try each partition — a BSD slice container contributes its slices — preferring the first + // that yields a writable FS. + foreach (var target in PartitionedDiskLister.EnumerateTargets(diskStream, detection)) { + var part = target.Partition; + if (target.Size <= 0 || target.Offset < 0 || target.Offset >= diskStream.Length) continue; - var actualSize = Math.Min(part.Size, diskStream.Length - part.StartOffset); - var partStream = new WritableSubStream(diskStream, part.StartOffset, actualSize); + var partStream = new WritableSubStream(diskStream, target.Offset, target.Size); // Don't add to ownership — it doesn't own the underlying stream var partSegments = new List(pathSegments) { @@ -175,48 +180,41 @@ public void DisposeAll() { return null; } + private const string PartitionedDiskId = "PartitionedDisk"; + + /// + /// Identifies the stream: a fixed VHD by its footer first (its first bytes are the guest + /// disk's own), then by content — the disk-image probe that recognises a raw partitioned + /// disk, and signatures as deep as the deepest registered one (superblocks at 32/64/128 KiB + /// included) rather than a fixed 4 KiB — then the FAT boot-sector heuristic. + /// private static IFormatDescriptor? DetectFormat(Stream stream) { if (stream.Length < 4) return null; var savedPos = stream.Position; try { - stream.Position = 0; - var headerSize = (int)Math.Min(4096, stream.Length); - var header = new byte[headerSize]; - var bytesRead = 0; - while (bytesRead < headerSize) { - var n = stream.Read(header, bytesRead, headerSize - bytesRead); - if (n == 0) break; - bytesRead += n; - } - - var headerSpan = header.AsSpan(0, bytesRead); - IFormatDescriptor? best = null; - var bestConfidence = 0.0; - - foreach (var desc in FormatRegistry.All) { - foreach (var sig in desc.MagicSignatures) { - if (MatchesMagic(headerSpan, sig) && sig.Confidence > bestConfidence) { - bestConfidence = sig.Confidence; - best = desc; - } - } - } - - if (best != null) return best; - // Heuristic: VHD fixed-disk detection — the "conectix" footer is at the // END of the file (offset = length - 512), not at offset 0. The registry // magic signature only checks offset 0 (which works for dynamic VHD). if (stream.Length >= 512 && stream.CanSeek) { stream.Position = stream.Length - 512; Span footer = stackalloc byte[8]; - if (stream.Read(footer) == 8 && "conectix"u8.SequenceEqual(footer)) { + if (stream.ReadAtLeast(footer, 8, throwOnEndOfStream: false) == 8 && "conectix"u8.SequenceEqual(footer)) { var vhdDesc = FormatRegistry.GetById("Vhd"); if (vhdDesc != null) return vhdDesc; } } + var format = FormatDetector.DetectByContent(stream); + if (format != FormatDetector.Format.Unknown && FormatDetector.GetDesc(format) is { } byContent) + return byContent; + + stream.Position = 0; + var headerSize = (int)Math.Min(4096, stream.Length); + var header = new byte[headerSize]; + var bytesRead = stream.ReadAtLeast(header, headerSize, throwOnEndOfStream: false); + var headerSpan = header.AsSpan(0, bytesRead); + // Heuristic: FAT detection (no magic, BPB-based) if (bytesRead >= 64 && headerSpan[0] is 0xEB or 0xE9) { var bytesPerSector = (int)(headerSpan[11] | (headerSpan[12] << 8)); @@ -234,18 +232,6 @@ public void DisposeAll() { } } - private static bool MatchesMagic(ReadOnlySpan header, MagicSignature sig) { - if (header.Length < sig.Offset + sig.Bytes.Length) return false; - var slice = header.Slice(sig.Offset, sig.Bytes.Length); - if (sig.Mask != null) { - for (var i = 0; i < sig.Bytes.Length; i++) - if ((slice[i] & sig.Mask[i]) != (sig.Bytes[i] & sig.Mask[i])) - return false; - return true; - } - return slice.SequenceEqual(sig.Bytes); - } - /// /// Opens the raw virtual disk content as a seekable read/write stream. /// Uses format-specific stream wrappers (VhdStream, VhdxStream, etc.) that diff --git a/Compression.NativeUI/ViewModels/MainViewModel.cs b/Compression.NativeUI/ViewModels/MainViewModel.cs index 1ded0cc36..af0a2ada9 100644 --- a/Compression.NativeUI/ViewModels/MainViewModel.cs +++ b/Compression.NativeUI/ViewModels/MainViewModel.cs @@ -2074,8 +2074,10 @@ internal void HandleFileDrop(string[] files) { // Per-file + cumulative-size check via optional constraints. if (ops is Compression.Registry.IArchiveWriteConstraints constraints) { long cumulative = 0; + // The entry lands in the folder being browsed, as AddFilesToArchiveImpl names it. + var folderPrefix = _currentFolder.Length > 0 ? _currentFolder.TrimEnd('/') + "/" : ""; foreach (var path in files) { - var entryName = System.IO.Path.GetFileName(path); + var entryName = folderPrefix + System.IO.Path.GetFileName(path); var size = new FileInfo(path).Exists ? new FileInfo(path).Length : 0; cumulative += size; var input = new Compression.Registry.ArchiveInputInfo(path, entryName, IsDirectory: false); diff --git a/Compression.NativeUI/Views/PartitionsWindow.cs b/Compression.NativeUI/Views/PartitionsWindow.cs index 2a1f78f7c..5005e789f 100644 --- a/Compression.NativeUI/Views/PartitionsWindow.cs +++ b/Compression.NativeUI/Views/PartitionsWindow.cs @@ -268,8 +268,8 @@ private void RefreshGrid() { var partitions = editor.ListPartitions(); var rows = partitions.Select(p => { - var startLba = p.StartOffset / PartitionEditor.SectorSize; - var endLba = (p.StartOffset + p.Size) / PartitionEditor.SectorSize - 1; + var startLba = p.StartOffset / editor.LogicalSectorSize; + var endLba = (p.StartOffset + p.Size) / editor.LogicalSectorSize - 1; return new PartitionRow { Index = p.Index, IndexDisplay = p.Index.ToString(), diff --git a/Compression.Tests/DiskImage/PartitionSchemeDetectionTests.cs b/Compression.Tests/DiskImage/PartitionSchemeDetectionTests.cs new file mode 100644 index 000000000..f0c73bd31 --- /dev/null +++ b/Compression.Tests/DiskImage/PartitionSchemeDetectionTests.cs @@ -0,0 +1,317 @@ +#pragma warning disable CS1591 +using System.Buffers.Binary; +using Compression.Core.DiskImage; + +namespace Compression.Tests.DiskImage; + +/// +/// Partition schemes beyond the classic 512-byte MBR/GPT: 4Kn GPT, GPT recovery from the backup +/// header, protective/hybrid MBRs, EBR chains, and the checksummed labels of SGI, Sun, BSD and +/// Amiga. The labels here are laid out field by field from the scheme's specification; the +/// external fixture checks the same readers against labels parted and sfdisk write. +/// +[TestFixture] +public class PartitionSchemeDetectionTests { + + [SetUp] + public void EnsureRegistered() => Compression.Lib.FormatRegistration.EnsureInitialized(); + + /// A zeroed, writable stream whose buffer the test may patch through GetBuffer. + private static MemoryStream Buffered(long length) { + var bytes = new byte[length]; + return new MemoryStream(bytes, 0, bytes.Length, writable: true, publiclyVisible: true); + } + + private static MemoryStream GptDisk(int sectorSize, long sectors, params (long Start, long Length)[] parts) { + var ms = Buffered(sectors * sectorSize); + var editor = PartitionEditor.CreateEmpty(ms, PartitionScheme.Gpt, sectorSize); + foreach (var (start, length) in parts) + editor.AddPartition(start * sectorSize, length * sectorSize, PartitionType.Linux, "p"); + return ms; + } + + // ── GPT ──────────────────────────────────────────────────────────── + + [TestCase(512)] + [TestCase(4096)] + [Category("HappyPath")] + public void Given_GptAtBlockSize_When_Detected_Then_SectorSizeAndOffsetsFollowIt(int sectorSize) { + using var disk = GptDisk(sectorSize, 4096, (256, 512)); + var head = disk.ToArray().AsSpan(0, GptParser.ProbeBytes); + + Assert.That(GptParser.DetectSectorSize(head), Is.EqualTo(sectorSize)); + var detection = PartitionTableDetector.Detect(disk); + Assert.That(detection.Scheme, Is.EqualTo("GPT")); + Assert.That(detection.SectorSize, Is.EqualTo(sectorSize)); + Assert.That(detection.Partitions.Single().StartOffset, Is.EqualTo(256L * sectorSize)); + Assert.That(detection.Partitions.Single().Size, Is.EqualTo(512L * sectorSize)); + } + + [Test, Category("HappyPath")] + public void Given_4KnGpt_When_Written_Then_EntryArrayTakesFourBlocksAndUsableRangeFollows() { + using var disk = GptDisk(4096, 1024); + var info = GptParser.ReadHeader(disk)!; + Assert.That(info.SectorSize, Is.EqualTo(4096)); + Assert.That(info.FirstUsableLba, Is.EqualTo(6UL), "LBA0 MBR + LBA1 header + 4 blocks of 128x128-byte entries"); + Assert.That(info.LastUsableLba, Is.EqualTo(1024UL - 6), "backup array (4) + backup header (1) at the end"); + Assert.That(info.AlternateLba, Is.EqualTo(1023UL)); + } + + [Test, Category("Boundary")] + public void Given_GptPartitionEndingAtLastUsableLba_When_Added_Then_AcceptedButOneMoreSectorRefused() { + using var disk = GptDisk(512, 2048); + var editor = new PartitionEditor(disk); + var lastUsable = 2048 - 34; // 32 backup-array sectors + backup header, then the LBA before + Assert.That(() => editor.AddPartition(1024L * 512, (lastUsable - 1024 + 2) * 512L, PartitionType.Linux, null), + Throws.InvalidOperationException, "would run into the backup entry array"); + editor.AddPartition(1024L * 512, (lastUsable - 1024 + 1) * 512L, PartitionType.Linux, null); + Assert.That(editor.Verify().IsValid, Is.True, string.Join("; ", editor.Verify().Issues)); + Assert.That(() => editor.AddPartition(33L * 512, 8 * 512, PartitionType.Linux, null), + Throws.InvalidOperationException, "LBA 33 is the primary entry array"); + } + + [Test, Category("Boundary")] + public void Given_GptWith128Entries_When_129thAdded_Then_Refused() { + using var disk = GptDisk(512, 34 + 128 + 34 + 8); + var editor = new PartitionEditor(disk); + for (var i = 0; i < 128; ++i) + editor.AddPartition((34L + i) * 512, 512, PartitionType.Linux, null); + Assert.That(PartitionTableDetector.Detect(disk).Partitions, Has.Count.EqualTo(128)); + Assert.That(() => editor.AddPartition((34L + 128) * 512, 512, PartitionType.Linux, null), Throws.InvalidOperationException); + } + + [Test, Category("ErrorHandling")] + public void Given_PrimaryHeaderCrcBroken_When_Parsed_Then_TheBackupHeaderIsUsed() { + using var disk = GptDisk(512, 4096, (2048, 1024)); + disk.Position = 512 + 40; // FirstUsableLBA inside the primary header: breaks its CRC + disk.WriteByte(0x77); + + var detection = PartitionTableDetector.Detect(disk); + Assert.That(detection.Scheme, Is.EqualTo("GPT")); + Assert.That(detection.Partitions.Single().StartOffset, Is.EqualTo(2048L * 512)); + Assert.That(GptParser.ReadHeader(disk)!.FromBackup, Is.True); + } + + [Test, Category("ErrorHandling")] + public void Given_PrimaryEntryArrayCorrupted_When_Parsed_Then_TheBackupArrayIsUsed() { + using var disk = GptDisk(512, 4096, (2048, 1024)); + disk.Position = 2 * 512 + 32; // first entry's FirstLBA + disk.Write(new byte[8]); + + var part = PartitionTableDetector.Detect(disk).Partitions.Single(); + Assert.That(part.StartOffset, Is.EqualTo(2048L * 512)); + } + + [Test, Category("ErrorHandling")] + public void Given_BothGptHeadersBroken_When_Detected_Then_NoTableAndNoProtectiveEntryListed() { + using var disk = GptDisk(512, 4096, (2048, 1024)); + disk.Position = 512 + 40; disk.WriteByte(0x77); + disk.Position = 4095L * 512 + 40; disk.WriteByte(0x77); + disk.Position = 512; disk.Write(new byte[8]); // and no signature for the lenient reading either + + var detection = PartitionTableDetector.Detect(disk); + Assert.That(detection.Partitions, Is.Empty, "a protective 0xEE entry describes the GPT, not data"); + } + + [Test, Category("EdgeCase")] + public void Given_HybridMbr_When_Detected_Then_GptWins() { + using var disk = GptDisk(512, 4096, (2048, 1024)); + // Hybrid: a real MBR entry mirroring the GPT partition beside the 0xEE entry. + var entry = disk.GetBuffer().AsSpan(0x1BE + 16, 16); + entry[4] = 0x83; + BinaryPrimitives.WriteUInt32LittleEndian(entry[8..], 2048); + BinaryPrimitives.WriteUInt32LittleEndian(entry[12..], 1024); + + var detection = PartitionTableDetector.Detect(disk); + Assert.That(detection.Scheme, Is.EqualTo("GPT")); + Assert.That(detection.Partitions.Single().Source, Is.EqualTo("GPT")); + } + + [Test, Category("EdgeCase")] + public void Given_GptEntryWithInvertedRange_When_Parsed_Then_Skipped() { + using var disk = GptDisk(512, 4096, (2048, 1024), (3072, 16)); + // Invert the second entry and recompute both CRCs through a re-read of the editor's own layout. + var bytes = disk.GetBuffer(); + var slot = bytes.AsSpan(2 * 512 + 128, 128); + BinaryPrimitives.WriteUInt64LittleEndian(slot[40..], 100); // LastLBA < FirstLBA + FixGptCrcs(bytes, 512, 4096); + + Assert.That(PartitionTableDetector.Detect(disk).Partitions, Has.Count.EqualTo(1)); + } + + private static void FixGptCrcs(byte[] disk, int sectorSize, long sectors) { + foreach (var headerLba in new[] { 1L, sectors - 1 }) { + var header = disk.AsSpan((int)(headerLba * sectorSize), 92); + var entriesLba = BinaryPrimitives.ReadUInt64LittleEndian(header[72..]); + var count = BinaryPrimitives.ReadUInt32LittleEndian(header[80..]); + var size = BinaryPrimitives.ReadUInt32LittleEndian(header[84..]); + if (headerLba != 1) // keep the backup array identical to the primary + disk.AsSpan(2 * sectorSize, (int)(count * size)).CopyTo(disk.AsSpan((int)((long)entriesLba * sectorSize))); + var crc = Compression.Core.Checksums.Crc32.Compute(disk.AsSpan((int)((long)entriesLba * sectorSize), (int)(count * size))); + BinaryPrimitives.WriteUInt32LittleEndian(header[88..], crc); + header.Slice(16, 4).Clear(); + BinaryPrimitives.WriteUInt32LittleEndian(header[16..], Compression.Core.Checksums.Crc32.Compute(header)); + } + } + + // ── MBR / EBR ────────────────────────────────────────────────────── + + [Test, Category("HappyPath")] + public void Given_ExtendedPartitionWithTwoLogicals_When_Detected_Then_BothLogicalsListedAtTheirAbsoluteOffsets() { + using var disk = Buffered(8192 * 512); + var editor = PartitionEditor.CreateEmpty(disk, PartitionScheme.Mbr); + editor.AddPartition(2048L * 512, 6000L * 512, PartitionType.ExtendedLba, null); + editor.AddLogicalPartition(2049L * 512, 1000L * 512, PartitionType.Linux, null); + editor.AddLogicalPartition(4000L * 512, 1000L * 512, PartitionType.Fat32Lba, null); + + var parts = PartitionTableDetector.Detect(disk).Partitions; + Assert.That(parts.Select(p => (p.Source, p.StartOffset)), + Is.EqualTo(new[] { ("EBR", 2049L * 512), ("EBR", 4000L * 512) })); + } + + [Test, Category("ErrorHandling")] + public void Given_EbrChainThatLoops_When_Detected_Then_TheWalkStops() { + using var disk = Buffered(8192 * 512); + var editor = PartitionEditor.CreateEmpty(disk, PartitionScheme.Mbr); + editor.AddPartition(2048L * 512, 6000L * 512, PartitionType.ExtendedLba, null); + editor.AddLogicalPartition(2049L * 512, 1000L * 512, PartitionType.Linux, null); + // Point the first EBR's link back at itself. + var link = disk.GetBuffer().AsSpan(2048 * 512 + 0x1BE + 16, 16); + link[4] = 0x05; + BinaryPrimitives.WriteUInt32LittleEndian(link[8..], 0); + BinaryPrimitives.WriteUInt32LittleEndian(link[12..], 1); + + Assert.That(PartitionTableDetector.Detect(disk).Partitions, Has.Count.EqualTo(1)); + } + + // ── SGI / Sun / BSD / Amiga ──────────────────────────────────────── + + [Test, Category("HappyPath")] + public void Given_SgiVolumeHeader_When_Detected_Then_PartitionsListedWithoutTheWholeVolume() { + var disk = new byte[8 * 1024 * 1024]; + var s = disk.AsSpan(0, 512); + BinaryPrimitives.WriteUInt32BigEndian(s, 0x0BE5A941); + void Part(int slot, uint blocks, uint first, uint type) { + var r = disk.AsSpan(312 + slot * 12); + BinaryPrimitives.WriteUInt32BigEndian(r, blocks); + BinaryPrimitives.WriteUInt32BigEndian(r[4..], first); + BinaryPrimitives.WriteUInt32BigEndian(r[8..], type); + } + Part(0, 4096, 2048, 0x83); + Part(8, 2048, 0, 0); // volume header + Part(10, 16384, 0, 6); // whole volume + uint sum = 0; + for (var i = 0; i < 512; i += 4) sum += BinaryPrimitives.ReadUInt32BigEndian(s[i..]); + BinaryPrimitives.WriteUInt32BigEndian(s[504..], unchecked(0u - sum)); + + var detection = PartitionTableDetector.Detect(disk); + Assert.That(detection.Scheme, Is.EqualTo("SGI")); + Assert.That(detection.Partitions.Select(p => (p.StartOffset, p.Size)), + Is.EqualTo(new[] { (2048L * 512, 4096L * 512), (0L, 2048L * 512) })); + using var ms = new MemoryStream(disk); + Assert.That(PartitionTableDetector.HasSelfValidatingLabel(ms), Is.True); + + disk[400] ^= 1; // checksum now wrong + Assert.That(PartitionTableDetector.Detect(disk).Scheme, Is.EqualTo("None")); + } + + [Test, Category("HappyPath")] + public void Given_SunLabel_When_Detected_Then_SlicesPlacedByCylinderGeometry() { + var disk = new byte[16 * 1024 * 1024]; + var s = disk.AsSpan(0, 512); + BinaryPrimitives.WriteUInt32BigEndian(s[128..], 1); // vtoc version + BinaryPrimitives.WriteUInt16BigEndian(s[140..], 8); // nparts + BinaryPrimitives.WriteUInt16BigEndian(s[142..], 0x83); // slice a: Linux native + BinaryPrimitives.WriteUInt16BigEndian(s[150..], 5); // slice c: whole disk + BinaryPrimitives.WriteUInt32BigEndian(s[188..], 0x600DDEEE); // vtoc sanity + BinaryPrimitives.WriteUInt16BigEndian(s[436..], 16); // ntrks + BinaryPrimitives.WriteUInt16BigEndian(s[438..], 63); // nsect + BinaryPrimitives.WriteUInt32BigEndian(s[444..], 2); // a: start cylinder 2 + BinaryPrimitives.WriteUInt32BigEndian(s[448..], 1008 * 5); + BinaryPrimitives.WriteUInt32BigEndian(s[464..], 1008 * 32); + BinaryPrimitives.WriteUInt16BigEndian(s[508..], 0xDABE); + ushort xor = 0; + for (var i = 0; i < 510; i += 2) xor ^= BinaryPrimitives.ReadUInt16BigEndian(s[i..]); + BinaryPrimitives.WriteUInt16BigEndian(s[510..], xor); + + var detection = PartitionTableDetector.Detect(disk); + Assert.That(detection.Scheme, Is.EqualTo("Sun")); + var part = detection.Partitions.Single(); + Assert.That(part.StartOffset, Is.EqualTo(2L * 16 * 63 * 512)); + Assert.That(part.Size, Is.EqualTo(1008L * 5 * 512)); + } + + [Test, Category("HappyPath")] + public void Given_StandaloneBsdDisklabel_When_Detected_Then_SlicesListedFromTheWholeDisk() { + var disk = new byte[8 * 1024 * 1024]; + var l = disk.AsSpan(512, 148 + 8 * 16); + BinaryPrimitives.WriteUInt32LittleEndian(l, 0x82564557); + BinaryPrimitives.WriteUInt32LittleEndian(l[40..], 512); + BinaryPrimitives.WriteUInt32LittleEndian(l[132..], 0x82564557); + BinaryPrimitives.WriteUInt16LittleEndian(l[138..], 8); + void Part(int slot, uint size, uint offset, byte fstype) { + var r = disk.AsSpan(512 + 148 + slot * 16); + BinaryPrimitives.WriteUInt32LittleEndian(r, size); + BinaryPrimitives.WriteUInt32LittleEndian(r[4..], offset); + r[12] = fstype; + } + Part(0, 4096, 64, 7); // a: 4.2BSD + Part(1, 2048, 4160, 1); // b: swap + Part(2, 16384, 0, 0); // c: whole disk + ushort xor = 0; + for (var i = 0; i < l.Length; i += 2) xor ^= BinaryPrimitives.ReadUInt16LittleEndian(l[i..]); + BinaryPrimitives.WriteUInt16LittleEndian(l[136..], xor); + + var detection = PartitionTableDetector.Detect(disk); + Assert.That(detection.Scheme, Is.EqualTo("BSD")); + Assert.That(detection.Partitions.Select(p => (p.Name, p.StartOffset)), + Is.EqualTo(new[] { ("a", 64L * 512), ("b", 4160L * 512) })); + + disk[512 + 132] ^= 1; + Assert.That(PartitionTableDetector.Detect(disk).Scheme, Is.EqualTo("None"), "without d_magic2 the label proves nothing"); + } + + [Test, Category("HappyPath")] + public void Given_AmigaRigidDiskBlock_When_Detected_Then_PartitionsPlacedByDosEnvecGeometry() { + var disk = new byte[16 * 1024 * 1024]; + void Checksum(Span block) { + var longs = BinaryPrimitives.ReadUInt32BigEndian(block[4..]); + block.Slice(8, 4).Clear(); + uint sum = 0; + for (var i = 0; i < longs; ++i) sum += BinaryPrimitives.ReadUInt32BigEndian(block[(i * 4)..]); + BinaryPrimitives.WriteUInt32BigEndian(block[8..], unchecked(0u - sum)); + } + var rdb = disk.AsSpan(1 * 512, 512); // RDSK in block 1 + BinaryPrimitives.WriteUInt32BigEndian(rdb, 0x5244534B); + BinaryPrimitives.WriteUInt32BigEndian(rdb[4..], 64); + BinaryPrimitives.WriteUInt32BigEndian(rdb[16..], 512); + BinaryPrimitives.WriteUInt32BigEndian(rdb[24..], 0xFFFFFFFF); + BinaryPrimitives.WriteUInt32BigEndian(rdb[28..], 3); // partition list → block 3 + BinaryPrimitives.WriteUInt32BigEndian(rdb[32..], 0xFFFFFFFF); + Checksum(rdb); + + var part = disk.AsSpan(3 * 512, 512); + BinaryPrimitives.WriteUInt32BigEndian(part, 0x50415254); + BinaryPrimitives.WriteUInt32BigEndian(part[4..], 64); + BinaryPrimitives.WriteUInt32BigEndian(part[16..], 0xFFFFFFFF); // no next + part[36] = 3; "DH0"u8.CopyTo(part[37..]); + void Env(int i, uint v) => BinaryPrimitives.WriteUInt32BigEndian(disk.AsSpan(3 * 512 + 128 + i * 4), v); + Env(0, 16); Env(1, 128); Env(3, 2); Env(5, 32); Env(9, 4); Env(10, 99); Env(16, 0x444F5301); + Checksum(part); + + var detection = PartitionTableDetector.Detect(disk); + Assert.That(detection.Scheme, Is.EqualTo("RDB")); + var p = detection.Partitions.Single(); + Assert.That(p.Name, Is.EqualTo("DH0")); + Assert.That(p.TypeName, Is.EqualTo("DOS1")); + Assert.That(p.StartOffset, Is.EqualTo(4L * 2 * 32 * 512)); + Assert.That(p.Size, Is.EqualTo(96L * 2 * 32 * 512)); + } + + [Test, Category("EdgeCase")] + public void Given_ZeroedDisk_When_Detected_Then_NoSchemeClaimsIt() { + Assert.That(PartitionTableDetector.Detect(new byte[1024 * 1024]).Scheme, Is.EqualTo("None")); + using var ms = new MemoryStream(new byte[1024 * 1024]); + Assert.That(PartitionTableDetector.HasSelfValidatingLabel(ms), Is.False); + } +} diff --git a/Compression.Tests/DiskImage/PartitionedDiskExternalTests.cs b/Compression.Tests/DiskImage/PartitionedDiskExternalTests.cs new file mode 100644 index 000000000..538c46dc2 --- /dev/null +++ b/Compression.Tests/DiskImage/PartitionedDiskExternalTests.cs @@ -0,0 +1,258 @@ +#pragma warning disable CS1591 +using System.Globalization; +using Compression.Core.DiskImage; +using Compression.Registry; +using FileFormat.PartitionedDisk; + +namespace Compression.Tests.DiskImage; + +/// +/// The partitioned-disk reader and writer against the reference Linux tools: tables written here +/// must pass sgdisk -v / sfdisk --verify and read back in parted; filesystems +/// edited in place inside a partition must pass fsck.vfat / e2fsck; and the +/// tables sfdisk, fdisk -b 4096 and parted (bsd, amiga, sun, dvh) write must +/// be read at the offsets those tools report. Skipped when WSL (or, off Windows, a tool) is missing. +/// +[TestFixture] +[Category("ExternalFsInterop")] +public class PartitionedDiskExternalTests { + + private static readonly PartitionedDiskFormatDescriptor Desc = new(); + private string _dir = null!; + + [SetUp] + public void SetUp() { + Compression.Lib.FormatRegistration.EnsureInitialized(); + this._dir = Path.Combine(Path.GetTempPath(), "cwb_pdisk_ext_" + Guid.NewGuid().ToString("N")[..10]); + Directory.CreateDirectory(this._dir); + } + + [TearDown] + public void TearDown() { + try { Directory.Delete(this._dir, recursive: true); } catch { /* best effort */ } + } + + private static void RequireTools(params string[] tools) { + if (!FsInteropToolbox.WslAvailable) Assert.Ignore("WSL is not available."); + foreach (var tool in tools) + if (!FsInteropToolbox.WslHasTool(tool)) + Assert.Ignore($"'{tool}' is not installed in the WSL distro."); + } + + /// Runs a bash script in the fixture directory; returns stdout+stderr, asserting success unless told otherwise. + private string Bash(string script, bool mustSucceed = true) { + var dir = FsInteropToolbox.WinToWsl(this._dir); + var result = FsInteropToolbox.RunWsl($"set -e; export LC_ALL=C; cd {dir}; {script}"); + var output = result.StdOut + result.StdErr; + if (mustSucceed) + Assert.That(result.ExitCode, Is.EqualTo(0), $"script failed:\n{script}\n{output}"); + return output; + } + + private string PathOf(string name) => Path.Combine(this._dir, name); + + private static FormatCreateOptions Options(string scheme, int sectorSize = 512) => new() { + FormatSpecific = new(StringComparer.OrdinalIgnoreCase) { + ["scheme"] = scheme, ["sectorSize"] = sectorSize.ToString(CultureInfo.InvariantCulture), + }, + }; + + private void CreateDisk(string disk, string scheme, int sectorSize, params string[] images) { + using var fs = File.Create(this.PathOf(disk)); + Desc.Create(fs, [.. images.Select(i => new ArchiveInputInfo(this.PathOf(i), i, false))], Options(scheme, sectorSize)); + } + + /// Partition (start, size) in bytes as parted -m unit B print reports them. + private List<(long Start, long Size)> PartedLayout(string disk) { + var output = this.Bash($"parted -s -m {disk} unit B print"); + var result = new List<(long, long)>(); + foreach (var line in output.Split('\n')) { + var f = line.Trim().Split(':'); + if (f.Length < 4 || !int.TryParse(f[0], out _)) continue; + result.Add((long.Parse(f[1].TrimEnd('B'), CultureInfo.InvariantCulture), + long.Parse(f[3].TrimEnd('B'), CultureInfo.InvariantCulture))); + } + return result; + } + + // ── Create, verified by the table tools and fsck ─────────────────── + + [Test] + public void Given_VfatAndExt4Images_When_CreatedAsGpt_Then_SgdiskSfdiskPartedAndFsckAgree() { + RequireTools("sgdisk", "sfdisk", "parted", "mkfs.vfat", "fsck.vfat", "mkfs.ext4", "e2fsck", "mcopy"); + this.Bash( + "echo dos > dos.txt; mkdir -p root && echo linux > root/linux.txt; " + + "mkfs.vfat -C a.img 8192 >/dev/null; mcopy -i a.img dos.txt ::/DOS.TXT; " + + "truncate -s 12M b.img; mkfs.ext4 -q -F -d root b.img"); + + this.CreateDisk("disk.img", "gpt", 512, "a.img", "b.img"); + + var sgdisk = this.Bash("sgdisk -v disk.img"); + Assert.That(sgdisk, Does.Contain("No problems found"), sgdisk); + var sfdisk = this.Bash("sfdisk --verify disk.img"); + Assert.That(sfdisk, Does.Contain("No errors detected"), sfdisk); + + var layout = this.PartedLayout("disk.img"); + var ours = PartitionTableDetector.Detect(File.ReadAllBytes(this.PathOf("disk.img"))).Partitions; + Assert.That(layout, Is.EqualTo(ours.Select(p => (p.StartOffset, p.Size)).ToList())); + Assert.That(layout[0].Start, Is.EqualTo(1024 * 1024)); + + var p1 = ours[0]; var p2 = ours[1]; + this.Bash( + $"dd if=disk.img of=p1.img bs=512 skip={p1.StartOffset / 512} count={p1.Size / 512} status=none; " + + $"dd if=disk.img of=p2.img bs=512 skip={p2.StartOffset / 512} count={p2.Size / 512} status=none; " + + "fsck.vfat -n p1.img; e2fsck -fn p2.img; test \"$(mcopy -i disk.img@@1M ::/DOS.TXT -)\" = dos"); + } + + [Test] + public void Given_VfatImage_When_CreatedAsMbr_Then_SfdiskVerifiesAndTheTypeIsFat32() { + RequireTools("sfdisk", "mkfs.vfat", "fsck.vfat", "mcopy"); + this.Bash("echo dos > dos.txt; mkfs.vfat -C a.img 8192 >/dev/null; mcopy -i a.img dos.txt ::/DOS.TXT"); + + this.CreateDisk("disk.img", "mbr", 512, "a.img"); + + var verify = this.Bash("sfdisk --verify disk.img"); + Assert.That(verify, Does.Contain("No errors detected"), verify); + var dump = this.Bash("sfdisk --dump disk.img"); + Assert.That(dump, Does.Contain("label: dos")); + Assert.That(dump, Does.Match(@"start=\s*2048, size=\s*16384, type=c"), dump); + Assert.That(this.Bash("mcopy -i disk.img@@1M ::/DOS.TXT -").Trim(), Is.EqualTo("dos")); + } + + [Test] + public void Given_VfatImage_When_CreatedAs4KnGpt_Then_FdiskWith4096ByteSectorsReadsTheSameLayout() { + RequireTools("fdisk", "mkfs.vfat", "mcopy"); + this.Bash("echo dos > dos.txt; mkfs.vfat -S 4096 -C a.img 8192 >/dev/null; mcopy -i a.img dos.txt ::/DOS.TXT"); + + this.CreateDisk("disk.img", "gpt", 4096, "a.img"); + + var listing = this.Bash("fdisk -b 4096 -l disk.img"); + Assert.That(listing, Does.Contain("Disklabel type: gpt"), listing); + Assert.That(listing, Does.Match(@"disk\.img1\s+256\s+2303\s+2048"), listing); // 1 MiB / 4096 = LBA 256, 8 MiB = 2048 blocks + Assert.That(listing, Does.Not.Contain("corrupt").And.Not.Contain("mismatch"), listing); + Assert.That(this.Bash("mcopy -i disk.img@@1M ::/DOS.TXT -").Trim(), Is.EqualTo("dos")); + } + + // ── Read what the tools write ────────────────────────────────────── + + [Test] + public void Given_FdiskGptAt4096ByteSectors_When_Listed_Then_TheVfatPartitionBrowses() { + RequireTools("fdisk", "mkfs.vfat", "mcopy"); + this.Bash( + "echo dos > dos.txt; truncate -s 24M disk.img; " + + "printf 'g\\nn\\n1\\n256\\n+8M\\nw\\n' | fdisk -b 4096 disk.img >/dev/null; " + + "mkfs.vfat -S 4096 -C p1.img 8192 >/dev/null; mcopy -i p1.img dos.txt ::/DOS.TXT; " + + "dd if=p1.img of=disk.img bs=4096 seek=256 conv=notrunc status=none"); + + var detection = PartitionTableDetector.Detect(File.ReadAllBytes(this.PathOf("disk.img"))); + Assert.That(detection.Scheme, Is.EqualTo("GPT")); + Assert.That(detection.SectorSize, Is.EqualTo(4096)); + Assert.That(detection.Partitions.Single().StartOffset, Is.EqualTo(256L * 4096)); + Assert.That(Compression.Lib.FormatDetector.Detect(this.PathOf("disk.img")).ToString(), Is.EqualTo("PartitionedDisk")); + Assert.That(Compression.Lib.ArchiveOperations.List(this.PathOf("disk.img"), null).Select(e => e.Name), + Has.Some.Match("^Partition1_.*/DOS\\.TXT$")); + } + + [Test] + public void Given_SfdiskExtendedWithLogicals_When_Listed_Then_LogicalsAppearAtSfdiskOffsets() { + RequireTools("sfdisk", "mkfs.vfat", "mcopy"); + this.Bash( + "echo dos > dos.txt; truncate -s 32M disk.img; " + + "printf 'label: dos\\nstart=2048, size=8192, type=83\\nstart=10240, type=5\\nstart=12288, size=8192, type=c\\nstart=22528, size=8192, type=83\\n' | sfdisk -q disk.img; " + + "mkfs.vfat --offset=12288 disk.img 4096 >/dev/null; mcopy -i disk.img@@6M dos.txt ::/DOS.TXT"); + + var dump = this.Bash("sfdisk --dump disk.img"); + var sfdiskStarts = System.Text.RegularExpressions.Regex.Matches(dump, @"start=\s*(\d+), size=\s*(\d+), type=(\w+)") + .Where(m => m.Groups[3].Value != "5") + .Select(m => (long.Parse(m.Groups[1].Value, CultureInfo.InvariantCulture) * 512, long.Parse(m.Groups[2].Value, CultureInfo.InvariantCulture) * 512)) + .ToList(); + var ours = PartitionTableDetector.Detect(File.ReadAllBytes(this.PathOf("disk.img"))).Partitions; + Assert.That(ours.Select(p => (p.StartOffset, p.Size)), Is.EquivalentTo(sfdiskStarts), dump); + Assert.That(ours.Count(p => p.Source == "EBR"), Is.EqualTo(2)); + Assert.That(Compression.Lib.ArchiveOperations.List(this.PathOf("disk.img"), null).Select(e => e.Name), + Has.Some.Match("/DOS\\.TXT$")); + } + + [TestCase("bsd")] + [TestCase("amiga")] + [TestCase("sun")] + [TestCase("dvh")] + public void Given_PartedLabel_When_Detected_Then_PartitionsMatchPartedsOffsets(string label) { + RequireTools("parted"); + // mkpart's leading arguments differ per label: amiga takes a name, dvh a primary/logical + // kind plus a filesystem type, bsd and sun only a filesystem type. + var kind = label switch { "amiga" => "primary", "dvh" => "primary ext2", _ => "ext2" }; + var made = this.Bash( + $"truncate -s 64M disk.img; parted -s disk.img mklabel {label}; " + + $"parted -s disk.img mkpart {kind} 4MiB 24MiB; parted -s disk.img mkpart {kind} 24MiB 44MiB; " + + "xxd -l 1536 disk.img | grep -v ' 0000 0000 0000 0000 0000 0000 0000 0000 '", mustSucceed: false); + var parted = this.PartedLayout("disk.img"); + if (parted.Count == 0) Assert.Ignore($"parted cannot write a {label} label here: {made}"); + + var bytes = File.ReadAllBytes(this.PathOf("disk.img")); + var detection = PartitionTableDetector.Detect(bytes); + var expectedScheme = label switch { "bsd" => "BSD", "amiga" => "RDB", "sun" => "Sun", _ => "SGI" }; + Assert.That(detection.Scheme, Is.EqualTo(expectedScheme), made); + // parted lists the label's bookkeeping slices (SGI volume header, Sun whole-disk) as partitions + // too; every data partition we report must be one parted reports, and each of parted's two + // mkpart partitions must be among ours. + var ours = detection.Partitions.Select(p => (p.StartOffset, p.Size)).ToList(); + Assert.That(parted, Is.SupersetOf(ours.Where(o => parted.Contains(o)))); + var dataPartitions = parted.Where(p => p.Start >= 1024 * 1024 - 64 * 1024 && p.Size >= 16L * 1024 * 1024).ToList(); + Assert.That(dataPartitions, Has.Count.EqualTo(2), $"{string.Join(", ", parted)}\n{made}"); + Assert.That(ours, Is.SupersetOf(dataPartitions), $"ours: {string.Join(", ", ours)}; parted: {string.Join(", ", parted)}"); + + using var ms = new MemoryStream(bytes); + Assert.That(PartitionTableDetector.HasSelfValidatingLabel(ms), Is.True); + Assert.That(Compression.Lib.FormatDetector.Detect(this.PathOf("disk.img")).ToString(), Is.EqualTo("PartitionedDisk")); + } + + // ── Edit in place, verified by fsck ──────────────────────────────── + + [Test] + public void Given_SfdiskGptWithVfatAndExt4_When_FilesEditedInPlace_Then_FsckIsCleanAndTheToolsReadTheEdits() { + RequireTools("sfdisk", "sgdisk", "mkfs.vfat", "fsck.vfat", "mkfs.ext4", "e2fsck", "debugfs", "mcopy"); + this.Bash( + "mkdir -p rootfs && echo linux > rootfs/linux.txt && echo gone > rootfs/gone.txt && echo dos > dos.txt; " + + "truncate -s 24M disk.img; " + + "printf 'label: gpt\\nstart=2048, size=16384, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7\\nstart=18432, size=28672, type=0FC63DAF-8483-4772-8E79-3D69D8477DE4\\n' | sfdisk -q disk.img; " + + "mkfs.vfat --offset=2048 disk.img 8192 >/dev/null; mcopy -i disk.img@@1M dos.txt ::/DOS.TXT; " + + "mkfs.ext4 -q -F -d rootfs -E offset=9437184 disk.img 14M"); + + var path = this.PathOf("disk.img"); + var before = File.ReadAllBytes(path); + var parts = PartitionTableDetector.Detect(before).Partitions; + string fat, ext; + using (var fs = new FileStream(path, FileMode.Open, FileAccess.ReadWrite)) { + fat = PartitionedDiskLister.MakePartitionPrefix(parts[0]); + ext = PartitionedDiskLister.MakePartitionPrefix(parts[1]); + Desc.Add(fs, [ + ArchiveInputInfo.InMemory($"{fat}/NEW.TXT", "added to fat\n"u8.ToArray()), + ArchiveInputInfo.InMemory($"{ext}/added.txt", "added to ext4\n"u8.ToArray()), + ]); + Desc.Remove(fs, [$"{ext}/gone.txt"]); + } + + var after = File.ReadAllBytes(path); + Assert.That(after.Length, Is.EqualTo(before.Length)); + // Nothing outside the two partitions moved: table, gaps and backup GPT are byte-identical. + var inside = new bool[after.Length]; + foreach (var p in parts) Array.Fill(inside, true, (int)p.StartOffset, (int)p.Size); + for (var i = 0; i < after.Length; ++i) + if (!inside[i] && after[i] != before[i]) + Assert.Fail($"byte {i} outside the partitions changed"); + + var output = this.Bash( + "sgdisk -v disk.img; " + + $"dd if=disk.img of=p1.img bs=512 skip={parts[0].StartOffset / 512} count={parts[0].Size / 512} status=none; " + + $"dd if=disk.img of=p2.img bs=512 skip={parts[1].StartOffset / 512} count={parts[1].Size / 512} status=none; " + + "fsck.vfat -n p1.img; e2fsck -fn p2.img; " + + "echo FAT=$(mcopy -i disk.img@@1M ::/NEW.TXT -); " + + "echo EXT=$(debugfs -R 'cat /added.txt' p2.img 2>/dev/null); " + + "echo GONE=$(debugfs -R 'ls -l /' p2.img 2>/dev/null | grep -c gone.txt || true)"); + Assert.That(output, Does.Contain("No problems found"), output); + Assert.That(output, Does.Contain("FAT=added to fat"), output); + Assert.That(output, Does.Contain("EXT=added to ext4"), output); + Assert.That(output, Does.Contain("GONE=0"), output); + } +} diff --git a/Compression.Tests/DiskImage/PartitionedDiskWriteTests.cs b/Compression.Tests/DiskImage/PartitionedDiskWriteTests.cs new file mode 100644 index 000000000..f41973b5e --- /dev/null +++ b/Compression.Tests/DiskImage/PartitionedDiskWriteTests.cs @@ -0,0 +1,279 @@ +#pragma warning disable CS1591 +using System.Buffers.Binary; +using Compression.Core.DiskImage; +using Compression.Registry; +using FileFormat.PartitionedDisk; + +namespace Compression.Tests.DiskImage; + +/// +/// Write support of the raw partitioned-disk format: files edited inside a partition's +/// filesystem in place, partitions added and deleted, fresh disks created from partition images — +/// and every refusal leaving the image byte-identical. +/// +[TestFixture] +public class PartitionedDiskWriteTests { + + [SetUp] + public void EnsureRegistered() => Compression.Lib.FormatRegistration.EnsureInitialized(); + + private static readonly PartitionedDiskFormatDescriptor Desc = new(); + + private static byte[] FatImage(string name, byte[] content, int sectors = 2880) { + var writer = new FileSystem.Fat.FatWriter(); + writer.AddFile(name, content); + return writer.Build(totalSectors: sectors); + } + + private static MemoryStream CreateDisk(string scheme, int sectorSize, params (string Name, byte[] Data)[] images) { + var ms = new MemoryStream(); + var options = new FormatCreateOptions { FormatSpecific = new(StringComparer.OrdinalIgnoreCase) { + ["scheme"] = scheme, ["sectorSize"] = sectorSize.ToString(System.Globalization.CultureInfo.InvariantCulture) } }; + Desc.Create(ms, [.. images.Select(i => ArchiveInputInfo.InMemory(i.Name, i.Data))], options); + return ms; + } + + private static string Prefix(Stream disk, int index) { + var detection = PartitionTableDetector.Detect(disk); + return PartitionedDiskLister.MakePartitionPrefix(detection.Partitions[index]); + } + + private static byte[] ReadEntry(Stream disk, string name) { + using var s = Desc.OpenEntry(disk, name, null); + using var ms = new MemoryStream(); + s.CopyTo(ms); + return ms.ToArray(); + } + + // ── Create ───────────────────────────────────────────────────────── + + [TestCase("gpt", 512)] + [TestCase("gpt", 4096)] + [TestCase("mbr", 512)] + [Category("HappyPath")] + public void Given_TwoFatImages_When_Created_Then_EachPartitionListsItsFileAtMiBAlignment(string scheme, int sectorSize) { + using var disk = CreateDisk(scheme, sectorSize, + ("boot.img", FatImage("A.TXT", "first"u8.ToArray())), + ("data.img", FatImage("B.TXT", "second"u8.ToArray()))); + + var detection = PartitionTableDetector.Detect(disk); + Assert.That(detection.Scheme, Is.EqualTo(scheme.ToUpperInvariant())); + Assert.That(detection.SectorSize, Is.EqualTo(sectorSize)); + Assert.That(detection.Partitions, Has.Count.EqualTo(2)); + Assert.That(detection.Partitions[0].StartOffset, Is.EqualTo(1024 * 1024)); + Assert.That(detection.Partitions[1].StartOffset % (1024 * 1024), Is.Zero); + + var names = Desc.List(disk, null).Select(e => e.Name).ToList(); + Assert.That(names, Has.Some.Match("^Partition1_[^/]+/A\\.TXT$"), string.Join(", ", names)); + Assert.That(names, Has.Some.Match("^Partition2_[^/]+/B\\.TXT$"), string.Join(", ", names)); + + var verify = new PartitionEditor(disk).Verify(); + Assert.That(verify.IsValid, Is.True, string.Join("; ", verify.Issues)); + } + + [Test, Category("HappyPath")] + public void Given_GptCreate_When_InputNamed_Then_PartitionNameIsTheStem() { + using var disk = CreateDisk("gpt", 512, ("Partition1_EFI.raw", new byte[4096])); + var part = PartitionTableDetector.Detect(disk).Partitions.Single(); + Assert.That(part.Name, Is.EqualTo("EFI.raw".Replace(".raw", ""))); + Assert.That(part.Size, Is.EqualTo(4096)); + } + + [Test, Category("ErrorHandling")] + public void Given_LooseTextFile_When_Created_Then_Refused() { + Assert.That(() => CreateDisk("gpt", 512, ("PROBE.TXT", "hello"u8.ToArray())), Throws.ArgumentException); + } + + [Test, Category("ErrorHandling")] + public void Given_NoInputs_When_Created_Then_Refused() { + Assert.That(() => CreateDisk("gpt", 512), Throws.ArgumentException); + } + + [Test, Category("Boundary")] + public void Given_FiveImagesOnMbr_When_Created_Then_RefusedBecauseOnlyFourPrimariesExist() { + var images = Enumerable.Range(1, 5).Select(i => ($"p{i}.raw", new byte[512])).ToArray(); + Assert.That(() => CreateDisk("mbr", 512, images), Throws.ArgumentException); + Assert.That(() => CreateDisk("mbr", 512, images[..4]), Throws.Nothing); + } + + [TestCase("scheme", "apm")] + [TestCase("sectorSize", "1024")] + [Category("ErrorHandling")] + public void Given_UnknownOption_When_Created_Then_Refused(string key, string value) { + var options = new FormatCreateOptions { FormatSpecific = new(StringComparer.OrdinalIgnoreCase) { [key] = value } }; + Assert.That(() => Desc.Create(new MemoryStream(), [ArchiveInputInfo.InMemory("a.raw", new byte[512])], options), + Throws.ArgumentException); + } + + // ── Edit inside a partition ──────────────────────────────────────── + + [TestCase("gpt", 512)] + [TestCase("gpt", 4096)] + [TestCase("mbr", 512)] + [Category("HappyPath")] + public void Given_FatPartition_When_FileAdded_Then_ItReadsBackAndBytesOutsideThePartitionAreUntouched(string scheme, int sectorSize) { + using var disk = CreateDisk(scheme, sectorSize, + ("a.img", FatImage("A.TXT", "first"u8.ToArray())), + ("b.img", FatImage("B.TXT", "second"u8.ToArray()))); + var before = disk.ToArray(); + var p1 = PartitionTableDetector.Detect(disk).Partitions[0]; + var prefix = Prefix(disk, 0); + + Desc.Add(disk, [ArchiveInputInfo.InMemory($"{prefix}/NEW.TXT", "added in place"u8.ToArray())]); + + Assert.That(ReadEntry(disk, $"{prefix}/NEW.TXT"), Is.EqualTo("added in place"u8.ToArray())); + Assert.That(ReadEntry(disk, $"{prefix}/A.TXT"), Is.EqualTo("first"u8.ToArray())); + var after = disk.ToArray(); + Assert.That(after.Length, Is.EqualTo(before.Length)); + Assert.That(after.AsSpan(0, (int)p1.StartOffset).SequenceEqual(before.AsSpan(0, (int)p1.StartOffset)), Is.True, + "bytes before the partition changed"); + var end = (int)(p1.StartOffset + p1.Size); + Assert.That(after.AsSpan(end).SequenceEqual(before.AsSpan(end)), Is.True, "bytes after the partition changed"); + } + + [Test, Category("HappyPath")] + public void Given_FatPartition_When_FileRemoved_Then_ItIsGoneAndTheOtherStays() { + using var disk = CreateDisk("gpt", 512, ("a.img", FatImage("A.TXT", "first"u8.ToArray()))); + var prefix = Prefix(disk, 0); + Desc.Add(disk, [ArchiveInputInfo.InMemory($"{prefix}/KEEP.TXT", "keep"u8.ToArray())]); + + Desc.Remove(disk, [$"{prefix}/A.TXT"]); + + var names = Desc.List(disk, null).Select(e => e.Name).ToList(); + Assert.That(names, Has.None.EqualTo($"{prefix}/A.TXT")); + Assert.That(names, Has.Some.EqualTo($"{prefix}/KEEP.TXT")); + } + + [Test, Category("ErrorHandling")] + public void Given_RawPartitionWithoutFilesystem_When_FileAdded_Then_RefusedAndByteIdentical() { + using var disk = CreateDisk("gpt", 512, ("blob.raw", new byte[64 * 1024])); + var before = disk.ToArray(); + var prefix = Prefix(disk, 0); + + Assert.That(() => Desc.Add(disk, [ArchiveInputInfo.InMemory($"{prefix}/X.TXT", "x"u8.ToArray())]), + Throws.InvalidOperationException); + Assert.That(disk.ToArray(), Is.EqualTo(before)); + } + + [Test, Category("Boundary")] + public void Given_FullFatVolume_When_OversizedFileAdded_Then_RefusedAndByteIdentical() { + using var disk = CreateDisk("gpt", 512, ("a.img", FatImage("A.TXT", "first"u8.ToArray()))); + var before = disk.ToArray(); + var prefix = Prefix(disk, 0); + + Assert.That(() => Desc.Add(disk, [ArchiveInputInfo.InMemory($"{prefix}/BIG.BIN", new byte[4 * 1024 * 1024])]), + Throws.InvalidOperationException); + Assert.That(disk.ToArray(), Is.EqualTo(before)); + } + + [Test, Category("ErrorHandling")] + public void Given_TwoEditsWhereTheSecondFails_When_Added_Then_NeitherIsWritten() { + using var disk = CreateDisk("gpt", 512, + ("a.img", FatImage("A.TXT", "first"u8.ToArray())), + ("blob.raw", new byte[64 * 1024])); + var before = disk.ToArray(); + + Assert.That(() => Desc.Add(disk, [ + ArchiveInputInfo.InMemory($"{Prefix(disk, 0)}/OK.TXT", "fine"u8.ToArray()), + ArchiveInputInfo.InMemory($"{Prefix(disk, 1)}/NO.TXT", "refused"u8.ToArray()), + ]), Throws.InvalidOperationException); + Assert.That(disk.ToArray(), Is.EqualTo(before)); + } + + [Test, Category("ErrorHandling")] + public void Given_NameOutsideAnyPartition_When_Removed_Then_FileNotFoundAndByteIdentical() { + using var disk = CreateDisk("gpt", 512, ("a.img", FatImage("A.TXT", "first"u8.ToArray()))); + var before = disk.ToArray(); + Assert.That(() => Desc.Remove(disk, ["Partition9_Nope/A.TXT"]), Throws.InstanceOf()); + Assert.That(disk.ToArray(), Is.EqualTo(before)); + } + + // ── Whole partitions ─────────────────────────────────────────────── + + [TestCase("gpt")] + [TestCase("mbr")] + [Category("HappyPath")] + public void Given_TwoPartitions_When_FirstRemovedWhole_Then_TableShrinksAndItsSectorsStay(string scheme) { + using var disk = CreateDisk(scheme, 512, + ("a.img", FatImage("A.TXT", "first"u8.ToArray())), + ("b.img", FatImage("B.TXT", "second"u8.ToArray()))); + var first = PartitionTableDetector.Detect(disk).Partitions[0]; + var payload = disk.ToArray().AsSpan((int)first.StartOffset, (int)first.Size).ToArray(); + + Desc.Remove(disk, [Prefix(disk, 0)]); + + var left = PartitionTableDetector.Detect(disk).Partitions; + Assert.That(left, Has.Count.EqualTo(1)); + Assert.That(Desc.List(disk, null).Select(e => e.Name), Has.Some.EndsWith("/B.TXT")); + Assert.That(disk.ToArray().AsSpan((int)first.StartOffset, (int)first.Size).ToArray(), Is.EqualTo(payload)); + Assert.That(new PartitionEditor(disk).Verify().IsValid, Is.True); + } + + [Test, Category("HappyPath")] + public void Given_GptDisk_When_FilesystemImageAddedAtRoot_Then_ItBecomesAMiBAlignedPartition() { + using var disk = CreateDisk("gpt", 512, ("a.img", FatImage("A.TXT", "first"u8.ToArray()))); + disk.SetLength(disk.Length + 4 * 1024 * 1024); + new PartitionEditor(disk).Reload(); + // Re-home the backup GPT at the new end: a grown disk is what partition tools hand over. + var grow = new PartitionEditor(disk); + grow.ConvertGptToMbr(); + grow.ConvertMbrToGpt(); + + Desc.Add(disk, [ArchiveInputInfo.InMemory("second.img", FatImage("N.TXT", "new"u8.ToArray()))]); + + var parts = PartitionTableDetector.Detect(disk).Partitions; + Assert.That(parts, Has.Count.EqualTo(2)); + Assert.That(parts[1].StartOffset % (1024 * 1024), Is.Zero); + Assert.That(Desc.List(disk, null).Select(e => e.Name), Has.Some.EndsWith("/N.TXT")); + Assert.That(new PartitionEditor(disk).Verify().IsValid, Is.True); + } + + [Test, Category("Boundary")] + public void Given_NoFreeSpace_When_ImageAddedAtRoot_Then_RefusedAndByteIdentical() { + using var disk = CreateDisk("gpt", 512, ("a.img", FatImage("A.TXT", "first"u8.ToArray()))); + var before = disk.ToArray(); + Assert.That(() => Desc.Add(disk, [ArchiveInputInfo.InMemory("second.img", FatImage("N.TXT", "new"u8.ToArray()))]), + Throws.InvalidOperationException); + Assert.That(disk.ToArray(), Is.EqualTo(before)); + } + + [Test, Category("ErrorHandling")] + public void Given_LooseFileAtRoot_When_Added_Then_RefusedAndByteIdentical() { + using var disk = CreateDisk("gpt", 512, ("a.img", FatImage("A.TXT", "first"u8.ToArray()))); + var before = disk.ToArray(); + Assert.That(() => Desc.Add(disk, [ArchiveInputInfo.InMemory("notes.txt", "text"u8.ToArray())]), + Throws.InvalidOperationException); + Assert.That(disk.ToArray(), Is.EqualTo(before)); + } + + // ── Staged window ────────────────────────────────────────────────── + + [Test, Category("Boundary")] + public void Given_StagedWindow_When_WritePastItsEnd_Then_ThrowsAndDiskUntouched() { + var bytes = Enumerable.Range(0, 8192).Select(i => (byte)i).ToArray(); + using var disk = new MemoryStream(bytes.ToArray()); + var window = new StagedWindowStream(disk, 1024, 4096); + window.Position = 4090; + Assert.That(() => window.Write(new byte[7]), Throws.InstanceOf()); + Assert.That(() => window.SetLength(4097), Throws.InstanceOf()); + window.SetLength(100); // shrinking is a no-op + Assert.That(disk.ToArray(), Is.EqualTo(bytes)); + } + + [Test, Category("HappyPath")] + public void Given_StagedWrites_When_ReadBackBeforeCommit_Then_SeenOnlyThroughTheWindowUntilCommitted() { + var bytes = new byte[16384]; + using var disk = new MemoryStream(bytes.ToArray()); + var window = new StagedWindowStream(disk, 4096, 8192); + window.Position = 4094; // straddles a page boundary + window.Write([1, 2, 3, 4]); + window.Position = 4094; + var back = new byte[4]; + window.ReadExactly(back); + Assert.That(back, Is.EqualTo(new byte[] { 1, 2, 3, 4 })); + Assert.That(disk.ToArray(), Is.EqualTo(bytes), "nothing reaches the disk before Commit"); + + window.Commit(); + Assert.That(disk.ToArray().AsSpan(4096 + 4094, 4).ToArray(), Is.EqualTo(new byte[] { 1, 2, 3, 4 })); + } +} diff --git a/Compression.Tests/NativeUI/ShellContentDetectionTests.cs b/Compression.Tests/NativeUI/ShellContentDetectionTests.cs index 5590561ef..1cfafbcbd 100644 --- a/Compression.Tests/NativeUI/ShellContentDetectionTests.cs +++ b/Compression.Tests/NativeUI/ShellContentDetectionTests.cs @@ -59,16 +59,30 @@ private string Write(string name, byte[] data) { } [Test] - public void GivenAnImgHoldingAPartitionTable_WhenOpened_ThenItIsAPartitionedDiskAndNothingIsOfferedToBeAddedToIt() { + public void GivenAnImgHoldingAPartitionTable_WhenOpened_ThenItIsAPartitionedDiskWhoseAddsGoThroughThePartitions() { this._model.Open(this.Write("disk.img", PartitionedDisk())); Assert.Multiple(() => { Assert.That(this._model.Format, Is.EqualTo("PartitionedDisk"), "the extension says FAT; the bytes say whole disk"); - Assert.That(this._model.AddFilesCommand.CanExecute(null), Is.False, - "adding through the FAT writer would write a FAT structure over the partition table"); + // Adding is offered, but through the partition-aware writer — never the FAT writer, which + // would write a FAT structure over the partition table. + Assert.That(this._model.AddFilesCommand.CanExecute(null), Is.True); + Assert.That(FormatRegistry.GetArchiveOps("PartitionedDisk"), Is.InstanceOf()); }); } + [Test] + public void GivenAPartitionedDisk_WhenAFileIsDroppedIntoAPartitionFolder_ThenTheDropIsAccepted() { + var disk = PartitionedDisk(); + this._model.Open(this.Write("disk.img", disk)); + this._model.CurrentFolder = PartitionedDiskLister.MakePartitionPrefix(PartitionTableDetector.Detect(disk).Partitions[0]); + var dropped = this.Write("any.txt", "dropped"u8.ToArray()); + + var (allowed, _) = this._model.EvaluateDropAgainstCurrentArchive([dropped]); + + Assert.That(allowed, Is.True); + } + [Test] public void GivenAnImgHoldingABareFatVolume_WhenOpened_ThenFilesCanBeAddedToIt() { this._model.Open(this.Write("floppy.img", Fat())); diff --git a/Compression.Tests/Nested/NestedStreamResolverTests.cs b/Compression.Tests/Nested/NestedStreamResolverTests.cs index 94ac99129..732174fc3 100644 --- a/Compression.Tests/Nested/NestedStreamResolverTests.cs +++ b/Compression.Tests/Nested/NestedStreamResolverTests.cs @@ -111,4 +111,28 @@ public void ResolveDeepest_AddFileThroughResolved_AppearsOnReRead() { Assert.Ignore("FAT descriptor does not implement IArchiveModifiable (unexpected)"); } } + + [TestCase("gpt", 512)] + [TestCase("gpt", 4096)] + [TestCase("mbr", 512)] + [Category("HappyPath")] + public void Given_RawPartitionedDisk_When_Resolved_Then_TheFilesystemInsideThePartitionIsReturned(string scheme, int sectorSize) { + var fatWriter = new FileSystem.Fat.FatWriter(); + fatWriter.AddFile("inner.txt", "inside a partition"u8.ToArray()); + var fat = fatWriter.Build(totalSectors: 2880); + using var disk = new MemoryStream(); + new FileFormat.PartitionedDisk.PartitionedDiskFormatDescriptor().Create(disk, [ArchiveInputInfo.InMemory("fat.img", fat)], + new FormatCreateOptions { FormatSpecific = new(StringComparer.OrdinalIgnoreCase) { + ["scheme"] = scheme, ["sectorSize"] = sectorSize.ToString(System.Globalization.CultureInfo.InvariantCulture) } }); + + var resolution = NestedStreamResolver.ResolveDeepest(disk); + + Assert.That(resolution, Is.Not.Null, "a raw disk's partitions must be descended into"); + Assert.That(resolution!.InnerDescriptor.Id, Is.EqualTo("Fat")); + Assert.That(resolution.NestingPath, Does.Contain($"{scheme.ToUpperInvariant()} partition 0")); + Assert.That(resolution.InnerStream.Length, Is.EqualTo(fat.Length)); + resolution.InnerStream.Position = 0; + Assert.That(FormatRegistry.GetArchiveOps("Fat")!.List(resolution.InnerStream, null).Select(e => e.Name), + Has.Some.EqualTo("inner.txt")); + } } diff --git a/README.md b/README.md index bc4c13f2b..094cc1dff 100644 --- a/README.md +++ b/README.md @@ -457,6 +457,10 @@ checks. **Partition-table support.** `MbrParser`, `GptParser` and `PartitionTypeDatabase` feed recursive descent so `--recursive` can follow disk image → partition table → filesystem → nested format. +Raw disk images (`PartitionedDisk`) read GPT (512- and 4096-byte LBAs), MBR with logical +partitions, APM, BSD disklabels, Amiga RDB, Sun and SGI labels; files inside a partition are edited +in place by that partition's filesystem editor, GPT/MBR partitions are added and removed, and new +disks are created from partition images — see [ARCHITECTURE.md](ARCHITECTURE.md). Analysis does not change format ownership: discovering JPEG data inside damaged storage does not make JPEG an archive format, and finding an ext superblock does not make `Compression.Analysis` the ext diff --git a/docs/catalogue/GAPS.md b/docs/catalogue/GAPS.md index e0506f117..171540636 100644 --- a/docs/catalogue/GAPS.md +++ b/docs/catalogue/GAPS.md @@ -23,10 +23,10 @@ exists. It warns when a capability letter has drifted from the live descriptor. | Gap type | Rows | Meaning | | --- | ---: | --- | -| `MISSING` | 240 | a source lists it; no descriptor or building block covers it | -| `READ-ONLY-BUT-SOURCES-WRITE` | 24 | we only read it; a source or the format's own tool writes it | +| `MISSING` | 238 | a source lists it; no descriptor or building block covers it | +| `READ-ONLY-BUT-SOURCES-WRITE` | 20 | we only read it; a source or the format's own tool writes it | | `PARTIAL` | 37 | covered, but the reader handles only part of the format (see the table below) | -| `OK` | 224 | covered with no known gap against what the sources list | +| `OK` | 230 | covered with no known gap against what the sources list | | `OURS-ONLY` | 458 | in our registry, listed by none of the seven sources | ### Per source @@ -36,13 +36,13 @@ read-only rows. | Source | Rows it lists | Mapped | % | OK | | --- | ---: | ---: | ---: | ---: | -| 7-Zip (formats + codecs) | 96 | 89 | 92.7 | 72 | +| 7-Zip (formats + codecs) | 96 | 89 | 92.7 | 74 | | libarchive (formats + filters) | 28 | 27 | 96.4 | 23 | | XADMaster / The Unarchiver | 126 | 95 | 75.4 | 78 | -| binwalk v3 | 79 | 48 | 60.8 | 36 | +| binwalk v3 | 79 | 48 | 60.8 | 38 | | Universal Extractor 2 | 124 | 71 | 57.3 | 48 | | PRONOM | 87 | 63 | 72.4 | 43 | -| Wikipedia | 266 | 156 | 58.6 | 117 | +| Wikipedia | 266 | 156 | 58.6 | 118 | The libarchive and 7-Zip lists are almost fully covered. The rows still open are libarchive's grzip, plus seven 7-Zip rows: LVM, Android `super.img` (LP), AVB, TE, the Swap2/Swap4 filter, PKWARE strong diff --git a/docs/catalogue/our-registry.csv b/docs/catalogue/our-registry.csv index 9512a342d..bd9da9071 100644 --- a/docs/catalogue/our-registry.csv +++ b/docs/catalogue/our-registry.csv @@ -397,7 +397,7 @@ format,Paragon,Paragon Backup,Archive,,.pbf,Y,Y,Y,Y format,ParallelsHdd,Parallels Disk (HDS),Archive,,.hds,Y,Y,, format,Parquet,Apache Parquet,Archive,,.parquet,Y,Y,, format,Partclone,partclone (Clonezilla),Archive,,.aa;.img,Y,Y,Y, -format,PartitionedDisk,Partitioned disk image,Archive,,,Y,Y,, +format,PartitionedDisk,Partitioned disk image,Archive,,,Y,Y,Y,Y format,Pbp,PSP PBP Archive,Archive,,.pbp,Y,Y,Y, format,Pc98,NEC PC-98 DOS,Archive,Y,.hdm;.fdi;.d88,Y,Y,Y,Y format,Pcap,PCAP (libpcap capture),Archive,,.pcap;.cap,Y,Y,, diff --git a/docs/catalogue/reference-formats.csv b/docs/catalogue/reference-formats.csv index c842b846e..d168a2e19 100644 --- a/docs/catalogue/reference-formats.csv +++ b/docs/catalogue/reference-formats.csv @@ -243,10 +243,10 @@ ISO 9660 (Joliet / Rock Ridge / El Torito),iso;cdr;bin;cd-rom;cdi;dmg;img;joliet High Sierra format,iso;hsf,filesystem,,,,,,,Y,1,,,,,,MISSING,, UDF,udf;cdr;dmg;iso;toast,filesystem,R,,,,,fmt/1738;fmt/1739;fmt/1757,Y,3,https://www.osta.org/specs/,Udf,Y,Y,Y,PARTIAL,,type-1 partition maps only: no VAT/sparable/metadata partitions (UDF 1.50+ media) (UdfReader.cs) Roxio Toast image,toast,disk-image,,,,,,,Y,1,,Iso;Hfs,Y,Y,Y,OK,, -Apple Partition Map,apm;bin;cdr;dmg;img;iso;toast,partition-table,R,,,,,fmt/1740;fmt/1741;fmt/1757,,2,https://en.wikipedia.org/wiki/Apple_Partition_Map,PartitionedDisk,Y,,,READ-ONLY-BUT-SOURCES-WRITE,pdisk / hdiutil, -MBR partition table,mbr,partition-table,R,,,R,,,,2,https://en.wikipedia.org/wiki/Master_boot_record,PartitionedDisk,Y,,,READ-ONLY-BUT-SOURCES-WRITE,fdisk / sfdisk, -GUID Partition Table (GPT),gpt,partition-table,R,,,D,,,,2,https://uefi.org/specifications,PartitionedDisk,Y,,,READ-ONLY-BUT-SOURCES-WRITE,sgdisk / gdisk, -Raw disk image,img;raw;dd;bin,disk-image,,,,,,,Y,1,,PartitionedDisk,Y,,,READ-ONLY-BUT-SOURCES-WRITE,dd, +Apple Partition Map,apm;bin;cdr;dmg;img;iso;toast,partition-table,R,,,,,fmt/1740;fmt/1741;fmt/1757,,2,https://en.wikipedia.org/wiki/Apple_Partition_Map,PartitionedDisk,Y,Y,Y,PARTIAL,pdisk / hdiutil,APM tables are read and their partitions' filesystems edited in place; APM entries cannot be created; added or removed +MBR partition table,mbr,partition-table,R,,,R,,,,2,https://en.wikipedia.org/wiki/Master_boot_record,PartitionedDisk,Y,Y,Y,OK,fdisk / sfdisk,incl. EBR logical partitions; create/modify checked with sfdisk --verify and fsck +GUID Partition Table (GPT),gpt,partition-table,R,,,D,,,,2,https://uefi.org/specifications,PartitionedDisk,Y,Y,Y,OK,sgdisk / gdisk,512- and 4096-byte LBAs; create/modify checked with sgdisk -v; sfdisk --verify and fdisk -b 4096 +Raw disk image,img;raw;dd;bin,disk-image,,,,,,,Y,1,,PartitionedDisk,Y,Y,Y,OK,dd,files inside partitions are edited in place by the partition filesystem's own editor Linux LVM2,lvm,partition-table,R,,,,,,,1,,,,,,MISSING,,https://man7.org/linux/man-pages/man8/lvm.8.html Android logical partitions (super.img),img;lp,partition-table,R,,,,,,,1,,,,,,MISSING,,https://source.android.com/docs/core/ota/dynamic_partitions Apple Disk Image (UDIF),dmg;smi;img,disk-image,R,,,R,R,fmt/1071,Y,5,https://en.wikipedia.org/wiki/Apple_Disk_Image,Dmg,Y,Y,Y,OK,,