Skip to content

build: release 5.0.2 - #429

Merged
danielplohmann merged 1 commit into
masterfrom
release/5.0.2
Oct 29, 2026
Merged

danielplohmann merged 1 commit into
masterfrom
release/5.0.2

Conversation

@danielplohmann

Copy link
Copy Markdown
Owner

Sets __version__ to 5.0.2 and cuts ## [Unreleased] into ## [v5.0.2] - 2026-10-29 (empty Deprecated/Removed/Security dropped, fresh Unreleased opened, compare links updated), per RELEASING.md.

5.0.2 is the version ESCAPER_DOWNWARD_COMPATIBILITY already names (#416), so ARM reports from 5.0.1 and earlier compare as stale once this ships.

Contents: #414, #415, #416, #417, #418-#423, #424, #427, #428, plus #425/#426 dependency bumps.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

📊 SMDA Performance Evaluation Benchmark Results

Generated on: 2026-10-29 21:12:35

Summary

Metric Result
Correctness ✅ PASS — 0 / 155 common file(s) differ
Determinism ✅ PASS — base 3 run(s), PR 3 run(s)
Verdict inconclusive — within run-to-run noise (±2.6%)
Median paired speedup -0.40% (95% CI [-0.84%, -0.09%])
Timing noise band ±2.6%

Per-side Timing Context (best of 3/3 runs per file)

Side Files Functions Median best time/file (s) Sum of best times (s) Throughput estimate (func/s)
base 155 173540 0.2844 95.15 ~1824
pr 155 173540 0.2808 95.57 ~1816

These context rows sum each file's best observed time across repeated runs; they are normalized comparison estimates, not single-run CI wall-clock times.

Paired per-file timing (positive speedup = PR faster):

Statistic Value
Files compared 155
Median paired speedup -0.40% (95% CI [-0.84%, -0.09%])
Mean speedup -0.84% (95% CI [-1.75%, +0.06%])
Std dev / IQR 5.62% / 5.66%
Wilcoxon signed-rank p 0.1225 (n=155)

ℹ️ base and PR are timed in the same job on the same runner, with their passes interleaved base/PR/PR/base/base/PR so that drift over the job cancels rather than landing on whichever side ran last. The noise band above is therefore run-to-run variation on one machine, which is what it is able to measure; a difference inside it is reported as inconclusive. Correctness and determinism are not timing-based and are unaffected either way.

Heaviest 10 samples by base time (median paired speedup -0.01%; a change whose cost grows with input size shows here first):

File Functions Base (s) PR (s) Speedup
elf.akira_1d3b5c650533d13c81e325972a912e3ff8776e36e18bca966dae50735f8ab296_unpacked.smda 13292 7.600 7.591 +0.11%
win.merlin_v0.6.4_59f3732d9629c37c11ca9e3cebdcd1f1db5b6223329670b10adb59b0980a880f_dump7_0x0000000000400000.smda 7332 7.530 7.491 +0.51%
win.merlin_v0.6.4_59f3732d9629c37c11ca9e3cebdcd1f1db5b6223329670b10adb59b0980a880f_unpacked.smda 7324 7.425 7.537 -1.51%
win.danabot_dc817fc49c1fa7615ebb5515e3f569a72db62ba4651a1d854ec206c96d7dd4ee_unpacked.smda 9056 4.964 4.971 -0.13%
elf.akira_vt-2024-03-16-rust_3298d203c2acb68c474e5fdad8379181890b4403d6491c523c13730129be3f75_unpacked.smda 2833 2.906 2.921 -0.51%
win.danabot_2021-10-21-loader_2a3acdcd76575762b18c18c644a745125f55ce121f742d2aad962521bc7f25fd_dump7_0x00ae0000.smda 7048 2.693 2.552 +5.22%
win.danabot_vt-2021-08-09-v4_c0419b07dd58f6b7e192bbf1072096d5e3ba10845d6f57d21a0627cbef92a5a7_unpacked.smda 7673 2.510 2.776 -10.62%
win.danabot_2020-03-30_b6cb8daec9e3e27dbb39e33d3dc55e7100673869754e8c8a158a0e655e1cbcce_dump7_0x00a40000.smda 6853 2.132 2.138 -0.30%
win.vidar_v39.4_f1e6ef95e8fb839899a496e6ce304bca0be925f2473d0f9ecf250ee6ba330233_unpacked.smda 2734 1.820 1.745 +4.13%
win.vidar_2018-12-05-v4.0_65287763245fdd8b56bb72298c78fea62405bd35794a06afbbe23cc5d38be90a_dump7_0x00400000.smda 3242 1.631 1.572 +3.66%

Determinism (self-check across repeated runs)

Side Runs Files Deterministic Median timing CV
base 3 155 ✅ 2.6%
pr 3 155 ✅ 2.4%
Pairwise run matrix (individual run medians, diagnostic)

Diagnostic only: each row compares one raw PR run against one raw base run. The headline verdict above uses paired per-file best-of-runs timings.

Comparison Run PR Files Base Files Common Function Set Matches Med PR (s) Med Base (s) Med Diff (s) Speedup %
pr_0 vs base_0 155 155 155 155/155 0.2864s 0.2844s +0.0020s -0.70%
pr_0 vs base_1 155 155 155 155/155 0.2864s 0.2871s -0.0007s +0.26%
pr_0 vs base_2 155 155 155 155/155 0.2864s 0.2917s -0.0053s +1.83%
pr_1 vs base_0 155 155 155 155/155 0.2887s 0.2844s +0.0043s -1.52%
pr_1 vs base_1 155 155 155 155/155 0.2887s 0.2871s +0.0016s -0.56%
pr_1 vs base_2 155 155 155 155/155 0.2887s 0.2917s -0.0030s +1.03%
pr_2 vs base_0 155 155 155 155/155 0.2930s 0.2844s +0.0086s -3.02%
pr_2 vs base_1 155 155 155 155/155 0.2930s 0.2871s +0.0059s -2.04%
pr_2 vs base_2 155 155 155 155/155 0.2930s 0.2917s +0.0013s -0.43%

@danielplohmann
danielplohmann merged commit 2a943f9 into master Oct 29, 2026
29 checks passed
@danielplohmann
danielplohmann deleted the release/5.0.2 branch October 29, 2026 21:15
@danielplohmann danielplohmann added this to the v5.0.2 milestone Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant