[GuilhermeBn198]

Summary

Running the Test-Comp archive as its README says, on a clean ubuntu:24.04 with only python3 (what map2check.yml asked for), showed that AFL++ never ran:

• the bundled clang could not link the program's AFL++ binary: no ld, crt1.o/crti.o, libc/libm or libgcc on the host;
• Map2Check does not stop on it: it skips the fuzzer, gives the whole budget to KLEE and still writes a suite. So the smoke test passed (feat(testcomp): smoke test, README and licenses in the Test-Comp archive #85), and so would the FM-Tools CI, while every competition run would have been KLEE-only.

A second defect hid behind it: examples/cover_error.c declared reach_error only as extern. KLEE accepts that; the native AFL++ link does not. sv-benchmarks always defines it.

Changes:

• README: the host needs python3 and gcc-multilib (linker, C runtime and libgcc for 64 and 32 bits). It now also explains how the engines alternate and what each property's suite holds (up to 50 cases for Cover-Branches, half at most from the fuzzer's corpus).
• smoketest.sh: fails unless the log shows that AFL++ ran (both examples) and KLEE ran (Cover-Branches). The Cover-Branches budget goes from 30 to 60 s, since with 30 KLEE got no window; the run still ends in seconds.
• examples/cover_error.c: defines reach_error, as sv-benchmarks does.
• Wrapper docstring: named the 8.x suite sources (klee_log.csv, klee-last/*.ktest).

The FM-Tools entry gets gcc-multilib in required_ubuntu_packages (submission side, not in this repo). It is the smallest set that works:

host packages (--no-install-recommends)	AFL++ build
python3	fails: no ld
python3 binutils	fails: no -lc, -lgcc
python3 gcc	fails: no Scrt1.o, crti.o
python3 gcc-multilib	works, LP64 and ILP32

Earlier official validations (V99, V99c) ran in images with gcc, so their numbers reflect AFL++ active.

Test plan

The archive was rebuilt from this branch with install_pr73, whose code is develop's. Result: 693 files; license coverage passes; md5 a01c28c57f6f184c4a55809d4271c09c.

• smoketest.sh on ubuntu:24.04 with only python3: fails with "AFL++ did not run" (negative check)
• smoketest.sh on ubuntu:24.04 with python3 gcc-multilib: passes in 8 s
• smoketest.sh on ubuntu:26.04 (the FM-Tools CI image) with python3 gcc-multilib: passes in 6 s
• TestCov 3.11, the official Test-Comp archive (Zenodo 17698774), on the examples' suites:

example	LP64	ILP32
Cover-Error	TRUE, 1 test	TRUE, 1 test
Cover-Branches	100 % branch coverage, 14 tests	100 % branch coverage, 12 tests

• Logs show the full hybrid on Cover-Branches: AFL++ fuzzes and seeds KLEE with its corpus; KLEE explores from those seeds and seeds the fuzzer back.

🤖 Generated with Claude Code

[Copilot]

🟡 Changes recommended

The smoke checks accept pre-launch banners, allowing an engine startup failure to pass when the other engine produces the suite.

1 open issue

• Require post-launch evidence that each engine actually ran · New

What changed in this PR

This PR corrects the Test-Comp archive’s host requirements and strengthens smoke testing to catch silent fallback to KLEE-only execution.

Changes:

• Documents python3 and gcc-multilib requirements and hybrid test-suite generation.
• Adds engine checks to the smoke test and increases the Cover-Branches budget.
• Defines reach_error so the Cover-Error example links natively for AFL++.

File	Description
utils/​testcomp-archive/​smoketest.sh	Adds engine checks and a longer Cover-Branches budget.
utils/​testcomp-archive/​README.txt	Explains host dependencies and hybrid execution.
utils/​testcomp-archive/​make-archive.sh	Updates comments describing host requirements.
utils/​testcomp-archive/​examples/​cover_error.c	Defines reach_error for native linking.
utils/​map2check-testcomp-wrapper.py	Updates documentation of test-suite sources.

_{Review effort: Balanced}