[MetehanYurtseven]

Summary

FileArchiver::backup_reader computes SHA-256 for every chunk, including chunks that contain only zeros. For sparse files and disk images, hashing the zeros dominates the backup time.

All-zero chunks have a known length: the Rabin hash of a 64-byte window of zeros is 0 for every polynomial, so runs of zeros are cut at chunk_min_size (see also #609). With the fixed-size chunker they have chunk_size. This PR computes the id of such a chunk once per backup and uses it for every chunk of that length that is all zeros. Other chunks are hashed as before.

restic does the same since 0.15.0 (restic/restic@c147422, restic/restic#3854, discussion in restic/restic#3903).

• Blob ids and the repository format are unchanged, so existing repositories deduplicate as before.
• The zero check only runs on chunks of exactly the zero chunk length. It processes data in 4 KiB blocks with a fold that the compiler vectorizes: about 64 GiB/s here, compared to 2.4 GiB/s for iter().all().

Measurements

4 GiB stream backed up through Repository::archive into a local repository, rustic_core 0.13.0 with this patch applied, Xeon Gold 6254 (no SHA-NI):

input	before	after
zeros	219 MiB/s	7050–8800 MiB/s
random data	140 MiB/s	140 MiB/s

SHA-256 alone runs at about 227 MiB/s on this CPU, so the zeros were limited by hashing.

Tests

• archiver::file_archiver::tests::is_zero_finds_any_nonzero_byte
• integration::chunker::test_backup_zeros for both chunkers: backs up a file with runs of zeros between data, checks with read_data, which compares every blob with its id, and dumps the file again. With a deliberately wrong zero id, both cases fail.
• cargo fmt --all -- --check, cargo clippy -p rustic_core --all-targets --all-features without warnings.

Five snapshot tests (test_backup_*, test_chunker_params, test_rewrite) fail on main as well, because the stored snapshots still contain error_count: 0, which is no longer serialized since #561. CI on main (647d00f) is red for the same reason. This PR doesn't touch them.

🤖 Generated with Claude Code

[MetehanYurtseven]

An end-to-end measurement with a real workload: a raw image backup of a whole Windows disk (100 GiB, about 56 GiB of it free NTFS space, read from a VSS snapshot with free clusters as zeros). The program is a downstream tool built on rustic_core 0.13.0, run with and without this patch cherry-picked onto 0.13.0. Each run went into a fresh local repository. The VM had 4 vCPUs on a Xeon Gold 6254 (no SHA-NI).

run	time	throughput	added to repository
0.13.0	17.0 min	106 MiB/s	16.7 GiB
0.13.0 + this patch	12.4 min	148 MiB/s	16.7 GiB
0.13.0 (control)	17.0 min	106 MiB/s	16.7 GiB

The saving (about 4.9 s per GiB of zeros) matches the SHA-256 throughput of that CPU. The repository size is identical, as expected with unchanged blob ids. The gain grows with the share of zeros: for a mostly empty 2 TB disk it would cut a run from hours to minutes.

🤖 Generated with Claude Code