[Dev-iL]

• closes: Move to stable ABI. #95

Regular CPython releases currently require new version-specific wheels. This change builds cp311-abi3 wheels while retaining version-specific full-API wheels for free-threaded CPython and PyPy. It addresses #95, parallels pycompression/python-isal#240, and follows the Python 3.11 floor/native-buffer guidance in pycompression/xopen#182 (comment).

Growing codec output uses native allocation/reallocation followed by a final bytes copy. CPython types use heap specifications and public accessors. The two factory types are owned by module state with traverse/clear hooks. Allocation, reference replacement and failure cleanup preserve the original ownership rules. Metadata now matches the already documented 3.11 support floor.

Benchmarks

Nine randomized paired rounds, 128 cases, original and final installed wheels, baseline-again controls, CPython 3.11.15, GCC 15.2 and identical pinned zlib-ng 2.3.3. Linux x86_64/i7-3770 without AVX2, CPU 1 pinned. Four corpora and four sizes include irregular output lengths. Ratios are candidate CPU time / original CPU time; above 1 is slower. Control absolute p95 is 2.88%, maximum 5.31%.

Operation, 16 MiB + 137 bytes	FASTQ	Python source	Random	Zeros
compress1	1.015x	1.011x	1.016x	1.010x
decompress	1.048x	1.060x	1.131x	1.184x
raw-decompress	1.052x	1.066x	1.142x	1.214x
stream-decompress	1.046x	1.068x	1.128x	1.204x
bounded-decompress	0.997x	1.003x	1.010x	0.982x
gzip-read128k	1.007x	0.992x	1.001x	1.015x
gzip-readinto128k	1.003x	0.999x	0.998x	1.004x
crc32	0.986x	0.998x	0.985x	1.006x

The worst tested cases are zeros/1048713/raw-decompress 2.810x; zeros/1048713/decompress 2.296x; zeros/1048713/stream-decompress 2.265x. The copy/allocation cost remains material for unbounded decoding. Compression, checksums and ordinary gzip chunk reads generally stay near baseline; small differences within the control range are inconclusive. These results do not promise universal parity.

Full methodology, case results, raw runs and reproduction are committed. The JSON/CSV preserve CPU and wall timings, iteration counts, input hashes and interpreter/extension paths. Build metadata records source, wheel and native-archive hashes. Results are from one Linux host, with other platforms covered by CI configuration rather than local measurements.

Relationship to the maintainer's design

• Preserved: Python 3.11 ABI floor, native buffers for growing output, final PyBytes_FromStringAndSize, and explicit measurement of the cost.
• Measured refinement: filled finite bytes allocations return directly. Short results still copy. Existing initial capacities are preserved. Full gzip chunks also avoid the extra copy, and readinto writes to the caller's buffer.
• Compatibility exception: free-threaded CPython and PyPy retain the full API and original bytes-resize path. Stable ABI is unavailable for free-threaded CPython. Existing stream locks, thread-state detach/attach regions and no-GIL declarations remain, including the explicit Windows Py_GIL_DISABLED macro. Further free-threaded performance work belongs in a follow-up.
• PyPy exception: retain original static types so they remain immutable and non-subclassable; its headers lack CPython's immutable-type flag. Factory-only types still reject direct calls.
• Native allocator adjustment: zlib-ng callbacks run with the thread state detached. They use paired libc allocation/free because PyMem_RawMalloc/RawFree only entered the Stable ABI in 3.13; PyMem_Malloc is inappropriate in those callbacks. Attached output-buffer paths still use PyMem allocation/reallocation. Native callback memory may be absent from tracemalloc.
• Allocation/ownership detail: retain FASTCALL and legal Limited-API PyObject_New with paired PyObject_Free, avoiding unnecessary zeroing of large structures. Factory type references use module state; existing process-global exception behavior remains. This does not claim new per-interpreter module isolation.
• Additional optimization: stop buffer growth when the native codec has already reported EOF. This benefits full-API builds too. Exploratory strategy comparisons apply the same correction to their reference baseline.

Earlier attempts and considered approaches

Alternative buffer strategies: benchmarks and reasons for ruling them out

These are separate prototype experiments, rather than the final installed-wheel comparison above. Nine randomized paired rounds compare each strategy with a full-API baseline containing the same EOF-stop correction. The table summarizes 24 large irregular-output decode cases: four corpora, two sizes (1 MiB + 137 and 16 MiB + 137), and three decoder APIs. Ratios are paired CPU time divided by baseline time; above 1 means slower. Ranges are across case medians, not confidence intervals. Across the full 72-case strategy study, baseline-again control variation has an inclusive p95 of 8.20% and a maximum of 8.74%.

Output strategy	Median time ratio	Range of case medians
Native growing buffer + final copy (selected)	1.181x	0.888–1.911x
1 MiB chunks, joined at the end	0.998x	0.392–1.651x
Eager BytesIO backing storage	1.373x	1.049–1.880x
BytesIO created on the first growth	1.368x	1.139–1.921x

• Joined chunks: the favorable 0.998x median conceals opposing corpus effects. At 16 MiB + 137, the earlier Python-source case takes 1.057x baseline with native buffers versus 1.467x with segments. Zeros at that size favor segments: 1.159x versus 0.393x. Segments were ruled out as a general replacement because their substantial wins also come with material regressions and additional chunk/join machinery.
• Eager BytesIO: a 1.373x median does not recover the native-buffer cost. Python calls, writable buffer exports and factory ownership add work and state.
• Delayed BytesIO: promotion on first growth reduces some eager-creation overhead, but the 1.368x median remains worse than native buffers and individual cases reach 1.921x. It was not adopted as a general replacement.
• Using early favorable cases as a blanket performance claim: the strategy study's Python corpus repeats 30,032 bytes of package source, shorter than a deflate window. The final-wheel benchmark instead uses a broader stdlib-source corpus and irregular output lengths. The prototype results support the design tradeoff; they do not establish final-wheel performance on the broader corpus.

The selected implementation follows the native-buffer/final-copy design for growing output and retains a direct bytes path for filled finite allocations. Free-threaded CPython/PyPy keep their full-API bytes-resize path. Per-case strategy summary and the full benchmark report retain the evidence. These prototype ratios should not be compared directly with the final-wheel table above.

Validation

The actual final wheels pass 14,588 tests plus 18 subtests on system CPython 3.14 (3 skips), 14,450 tests on free-threaded 3.13 with the GIL disabled (1 skip; two stdlib compliance files unavailable), and 14,576 tests plus 18 subtests on PyPy 3.11 (15 skips). The identical ABI3 wheel passes the new ownership/boundary tests on CPython 3.11–3.15. Strict abi3audit passes. New tests preserve type lifetimes, copying, immutable/factory-only types, output boundaries, failed initialization and default-GIL import behavior. Full flake8/mypy, strict Sphinx, YAML/embedded Python/link checks, wheel/sdist builds and twine checks pass; the committed report records the core build and validation evidence.

CI builds/audits one 3.11 ABI3 wheel per Linux/macOS Intel/macOS ARM/Windows platform and tests that same wheel on 3.11–3.15. Windows 3.14t exercises its build macro. Existing sanitizer jobs remain. Missing optional stdlib compliance modules are detected explicitly. Remote CI, non-Linux execution and ASan/TSan remain to be verified; local timings do not cover those platforms.

Checklist

• Pull request details were added to CHANGELOG.rst
• Documentation was updated (if needed)