Skip to content

fix: sign the 32-bit codecs and cover the wire format against the reference - #27

Merged
derek-miller merged 13 commits into
mainfrom
agent/FL-16-wire-vectors
Sep 11, 2026
Merged

derek-miller merged 13 commits into
mainfrom
agent/FL-16-wire-vectors

Conversation

@svc-finitelabs

@svc-finitelabs svc-finitelabs Bot commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

FL-16 Part 2, FL-18 and FL-19 as one change, so every wire vector agrees with the reference implementation and there is no list of expected failures to maintain.

Wire-format coverage

  • test/test_messages_proto3.proto, the upstream conformance message vendored from v29.3 and trimmed of its well-known-type fields (FL-17).
  • test/generated/test_messages_proto3_schema.lua, wired into make check-schema. Generating it is itself the DRV-104 / DRV-105 regression test: 24 messages, 28 subschema references, all resolving.
  • test/generated/wire_vectors.lua, 37 vectors of reference bytes paired with the table they decode to, produced by tools/gen_wire_vectors. A golden is the reference implementation's own serialization, or an assembled payload the reference is asserted to parse to the declared message; the expected table is derived from that message, never written by hand.
  • test/wire_vectors_test.lua asserts both directions. Reference → Lua is strict. Lua → reference is semantic: this encoder never packs, so its bytes are re-decoded rather than compared bytewise, and make check-wire-vectors closes the loop by parsing Lua's bytes with the reference implementation. That target runs in CI as its own step.

Test architecture

Modules under test/*_test.lua are discovered by run_tests.sh, not registered. Each declares @test-name and @test-modes in its header, uses test/testlib.lua to report, and exits 0 or 1. The embedded selftest() stays in src/ so it ships and can run on a controller; test/protobuf_test.lua wraps it.

Codec fixes

  • FL-19. int32, enum, uint32 and sint32 read the full varint pair and use the low word, so a ten-byte -1 decodes as -1 on every interpreter instead of 1.8446744073709552e19 on 5.1, 5.2 and LuaJIT.
  • FL-18. sfixed32 decodes signed, and encode_fixed32 accepts the int32 range instead of raising on a negative value.

Both go through bit32.to_signed, new in lua-bitn v0.6.3, which is re-vendored here from the release asset.

bitn

Byte packing, the 64-bit splits and the float/double word assembly use bitn's u32_to_le_bytes, le_bytes_to_u32, u64_to_le_bytes, le_bytes_to_u64 and from_number instead of inline arithmetic. Output is byte-identical to main: a 390,483-case old-vs-new differential across every codec found exactly the two intended changes, negative sfixed32 encoding instead of raising.

On LuaJIT every codec is within a few percent of main except encode_double at about +25%; on Lua 5.4 the fixed-width codecs are 10–48% faster and decode_double about 55% slower; on Lua 5.1 float encode and decode are 33% and 58% faster. Measured with results consumed, one process per version.

Verified

  • make check and make check-wire-vectors clean; the amalgamated portable build passes its own selftest.
  • 252/252 selftest and 74/74 wire vectors on Lua 5.1, 5.2, 5.3, 5.4 and LuaJIT 2.1.
  • Removing a vector's fix fails the run: the suite has no known-gap list, so a red vector is a red run.

…ementation

FL-16 Part 2. Six protobuf defects have been found in the field or in review and
none by the test suite, and five of the six were wire-format or schema bugs
rather than bit twiddling. Everything built so far tested the scalar codec, the
one layer that produced the fewest of them.

Vendors the upstream conformance message and drives encode and decode against
the reference protobuf implementation as an oracle, replacing hand-written
literals. Both the golden bytes and the expected tables are derived by walking
the same reference message, which matters because three hand-written
expectations in the Part 1 work were themselves wrong.

The two directions are asserted differently on purpose. Reference to Lua is
strict, since the input is byte exact. Lua to reference is semantic and never
bytewise: this encoder never packs, so its output cannot match protoc's for any
repeated scalar even when it is correct.

The goldens are checked in, so make test still runs on a bare clone with no
Python. gen-wire-vectors and check-wire-vectors stay out of make check for the
reason check-types already documents.

Found three defects, each filed rather than fixed here so this stays a coverage
change:

  FL-17  the schema generator drops the google.protobuf imports, so any
         well-known-type subschema reference dangles. The vendored proto is
         trimmed for that reason.
  FL-18  sfixed32 is unsigned in both directions and encoding a negative one
         raises.
  FL-19  a negative int32 or enum decodes to 1.8e19 on 5.1, 5.2 and LuaJIT, and
         is correct on 5.3+ only because signed integer arithmetic wraps.

FL-19 is why known_gaps has two categories. A strict entry demands a failure,
which would only move which half of the CI matrix is red, so version-dependent
cases are reported rather than asserted.

Refs FL-16
A negative int32 or enum decoded to 1.8446744073709552e19 on Lua 5.1, 5.2 and
LuaJIT, and to the right answer on 5.3 and 5.4 only because the unsigned
`value[1] * 0x100000000 + value[2]` in bit64.to_number overflows signed 64-bit
integers and wraps. Nothing implemented sign extension; the value was aliased.

The fix is not sign extension either. Measured against the reference, the
operation is truncation to the field's declared width: the five-byte payload
0xFFFFFFFF is -1 as an int32 despite no sign bit being set at 64 bits, and
0xFFFFFFFF00000001 is 1 rather than a large negative. So 5.3 and 5.4 were not
merely right for the wrong reason, they were wrong on those inputs too, as were
all six matrix entries. int32 and uint32 are wire compatible by design, so a
peer writing uint32 = 4294967295 into a field a driver reads as int32 produces
exactly that five-byte form.

The same 64-bit payload has to decode to -1 as an int32 and 4294967295 as a
uint32, so no definition of bit64.to_number could serve both. The operation is
type directed and belongs here rather than in vendor/bitn.lua, which is left
untouched and needs no re-vendor.

INT32 and ENUM now read the full pair and interpret the low word signed, UINT32
reads it unsigned, and SINT32 zigzags the low word rather than a double that has
already rounded: a ten-byte sint32 payload previously reached zigzag_decode32 as
1.8446744073709552e19 and came back 0 on LuaJIT.

Six vectors cover the widths the reference's own serializer cannot emit, built
through the existing raw_case oracle so the expectations stay derived rather
than asserted by construction. Five of them fail on Lua 5.5 before this change,
which is what stops the suite accepting wraparound as a pass; the sint32 one
fails only on LuaJIT. The three version_dependent entries in test/known_gaps.lua
are gone, leaving that tier empty.

Fixes FL-19
Adding a suite meant picking one of three reporting idioms and editing
run_tests.sh in three places. Three modules had drifted to three ways of
signalling failure and two different failure caps, and the newest generated
artifact had no CI drift gate at all.

Modules are now discovered rather than registered: any test/<name>_test.lua is
a module, keyed by filename, and `make test-<name>` follows from the existing
test-% rule. Per-module behaviour the runner used to hardcode moves into the
file as `@test-name` and `@test-modes` directives, so everything about a suite
lives in the suite. Adding one is dropping in a file.

test/testlib.lua is the shared harness. Both existing suites lose their
private failure lists, comparison helpers and exit paths and get smaller for
it; the embedded selftest() gains test/protobuf_test.lua as a wrapper so the
runner no longer needs a special case for it. selftest() itself stays in src/,
since shipping inside the amalgamated module is what lets it run on a
controller against the real LuaJIT.

check-wire-vectors now runs in CI as its own step. Keeping it out of `make
check` is about staying runnable on a bare clone, which is not a reason to
leave it out of CI, where the venv is already provisioned for check-types.
Without it the wire goldens were the one generated artifact with no drift
gate, and the encode direction was never verified against the reference
implementation at all.

No src/ change. Verified on 5.1, 5.2, 5.3, 5.4 and LuaJIT 2.1: same modules,
same assertion counts, same output. Both known-gap failure directions still
bite -- removing a strict entry for a live defect fails the run, and listing a
passing case fails it as a stale entry.
@svc-finitelabs

Copy link
Copy Markdown
Contributor Author

Verified 7101fe1 rather than taking the commit message on trust, since this PR is the gate for FL-18, FL-19 and the v0.6.8 stack.

CI is 8/8 green, and the new step actually ran. A CI step that no-ops is green for the wrong reason, so I read the Check job log instead of the tick:

Checked-in wire vectors match the generator.
Wire vectors: 31 goldens read back, 25 encodings compared as messages, 3 known gaps, 3 version dependent
Wire vectors: the reference implementation agrees with both directions

The recipe is fail safe if the venv is ever absent: the guard is exit 1, not a skip, and each guard is one if ... fi spanning its own work, so it does not hit the one-shell-per-recipe-line trap.

The refactored harness still discriminates. Folding the assertion plumbing into testlib.lua is exactly the change that can swallow failures while everything stays green, so I re-ran a mutation. Skipping zigzag_decode32 at init.lua:557:

  • runner exits 1, 1/5 modules pass
  • wire-vectors reports 6 failures, protobuf reports 9
  • restored afterwards, tree clean

Both known-gap directions bite, checked by manipulating the list rather than by re-running your check.

  • removing negative sfixed32 | decode from strict: FAIL: ... want -1, got 4294967295, exit 1
  • adding a passing case to strict: FAIL: singular scalars | decode: listed as a known gap (FL-BOGUS) but PASSED, remove the entry, exit 1

Local runs on Lua 5.5 and LuaJIT 2.1 agree: 5/5 modules each, 62 assertions over 31 vectors both, and version_dependent prints agrees here on 5.5 against differs here on LuaJIT, which is the split it exists to make visible.

I updated the PR body. This repo is squash only, so the body is the commit message that lands on main, and it described only 2d6555d. It said nothing about testlib.lua, discovered modules or the CI step, which together are 318 added lines across 7 files. I added a section for the second commit and corrected the stale "a new run_tests.sh module" bullet, since modules are discovered now. Revert it if you would rather write that yourself. Worth noting this is the same class of error FL-18 and FL-19 caught on #28's body, so it looks like a pattern in this stack rather than a one off.

On the stack. #28 merges cleanly onto 7101fe1. The mergeable: true GitHub reports for it is computed against the stale base 2d6555d, so I ran the real merge: clean, and the merged tree is green on both interpreters at 74 assertions over 37 vectors with 0 version dependent. Two things to expect:

Not approving: this needs your review, and I cannot approve an agent-authored PR regardless.

`pull_request: branches: [main, master]` meant a PR stacked on another branch
never triggered the workflow. #28 carries the FL-19 sign-extension fix, touches
src/, and has zero check runs against this branch: the first time it would have
been tested is after its parent merged and GitHub retargeted it, which is after
the review it was supposed to inform.

Dropping the filter verifies a pull request whatever it targets. `push` stays
pinned to main/master, so a branch gets one run per push rather than two.
@svc-finitelabs

Copy link
Copy Markdown
Contributor Author

Re-reviewed at the new head f090f128. The delta from the head I last read (7101fe1) is one commit: .github/workflows/build.yml +5/-1 and CLAUDE.md +7/-2. All 8 checks green.

The change is right, and it is the fix for the gap I flagged on #28. One thing is worth doing before it merges, though.

This PR's own CI cannot test the change

#27 targets main. Under both the old trigger and the new one, a PR targeting main runs. The 8/8 green above is therefore the same result with and without f090f128: it proves the workflow still parses, and touches none of the behaviour the commit exists for.

The only input that discriminates the change is a pull request targeting something other than main, which is #28. #28 is still at zero:

$ gh api "repos/finitelabs/lua-protobuf/actions/runs?per_page=100" --paginate \
    --jq '.workflow_runs[] | select(.head_branch|test("FL-19"))'
(no output)

No workflow run has ever existed for agent/FL-19-sign-extension, and commits/bda41b0/check-runs is still empty as of now. Moving the base did not retroactively give it CI, because a base-branch update is not a pull_request event. #28 should pick the new trigger up on its next opened, reopened or synchronize, since a pull_request run reads the workflow from the merge ref and that ref now carries this commit.

So the fix is unverified until #28 gets one more event. Reopening #28, or any push to its head, both triggers that run and doubles as the end to end proof that this commit does what it says. Doing it before #28 is reviewed is also the point of the change: the alternative is that #28 is first tested after this merges and GitHub retargets it, which is the ordering the commit message is objecting to.

I have not touched #28. Say the word and I will reopen it.

Checked and clean

  • Dropping the base filter cannot cause cross-PR cancellation. concurrency.group is ${{ github.workflow }}-${{ github.ref }}, and github.ref on a pull_request run is refs/pull/<n>/merge, so the key is already per-PR rather than per-base.
  • No duplicate runs are introduced. push stays filtered to main/master, so a topic-branch push still produces exactly one run, via the PR.
  • The CLAUDE.md bullets match the file they describe. Check wire vectors is its own step running make check-wire-vectors, after the make check step, and the test job matrix is the six entries as written.

Still REVIEW_REQUIRED, and still not approving or merging: this is Derek's commit, and #27 gates FL-18, FL-19 and the v0.6.8 release.

Puts `pull_request: branches: [ main, master ]` back.

Trims comments across the PR to what prevents a wrong edit: history, ticket
narration and restatements of the adjacent code are removed, and the unused
testlib.script_dir helper goes with them. No behaviour change.
@svc-finitelabs

Copy link
Copy Markdown
Contributor Author

Round 3: 362045d2, the trigger revert and the comment trim

Verified both halves against f090f128.

The filter restore is exact. git diff 7101fe11 362045d2 -- .github/workflows/build.yml is empty apart from removed comments, so the trigger is back to its pre-f090f12 state. CLAUDE.md's CI section now reads "push/PR to main or master", which matches the YAML.

The trim is behaviour neutral, and I checked rather than assumed it. Of the 282 removed lines, exactly one is not a comment, a Python docstring or markdown: testlib.script_dir. It has zero callers. The script_dir hits in run_tests.sh and run_tests_matrix.sh are an unrelated shell variable of the same name, and the four sites that need a module's own directory inline debug.getinfo rather than calling the helper.

There is one live __doc__ consumer, argparse.ArgumentParser(description=__doc__) at tools/gen_wire_vectors:483. The module docstring is trimmed rather than deleted, so --help still renders.

Identical results at both revisions, on Lua 5.5 and LuaJIT 2.1: 5/5 modules, 62 assertions over 31 vectors, 6 known gaps, 6 version dependent. make check-wire-vectors reports "Checked-in wire vectors match the generator" and agreement in both directions, which is the part that matters for the .proto edit: removing comments there did not perturb the descriptor or the goldens. format-check, check-types, check-schema and typecheck pass. make lint fails on this host only, luacheck 1.2.0 will not load under Lua 5.5, luacheck --version fails on its own, and it fails identically at f090f128. CI is 8/8 green, read per job rather than off the rollup.

Two things the revert leaves behind

#28 now conflicts with this branch. It merges cleanly against f090f128 and conflicts against 362045d2, in test/known_gaps.lua: both sides rewrote the comment block above version_dependent, and #28 additionally empties the table. One hunk to resolve, but #28 is blocked until someone does it. Its replacement comment is written at the old density, so it will want the same trim.

The stacked-PR gap is back, and the experiment meant to test it did not return a result. agent/FL-19-sign-extension has zero workflow runs, ever, and zero check runs on its head SHA. The close and reopen at 13:43 was aimed at proving f090f12 worked end to end, and it produced no run even though the filter was absent from this branch tip at that moment. I have not established why, so I am reporting it as an unexplained negative rather than naming a cause. Either way #28 targets a non-main branch again, so it stays at zero CI until #27 merges and GitHub retargets it.

If the revert is because the removal looked ineffective, that reading rests on a run that never appeared for a reason nobody has pinned down yet. Happy to chase it if useful.

Not approving or merging: REVIEW_REQUIRED, and #27 still gates FL-18, FL-19 and the v0.6.8 release.

@svc-finitelabs

Copy link
Copy Markdown
Contributor Author

Correcting one paragraph of the above. I wrote that the missing run had no pinned-down cause and floated an unmergeable-PR hypothesis. Both are wrong, and the answer was already on the record in #28 as issuecomment-5635460394, posted at 13:54:31Z, before I wrote that. I should have read #28's comments before speculating about #28.

The cause: the branches: filter is evaluated from the PR's head branch, not from the merge ref. #28's head bda41b02 carries pull_request: branches: [ main, master ] at blob 0f7bbf7e, because agent/FL-19-sign-extension branched before f090f12. Its base is not in that list, so Actions declined regardless of what the base or the merge commit said. Verified the head blob independently rather than taking the reading on faith; the merge-ref evidence in #28 is the other half.

My hypothesis is dead on its own terms too: #28 was MERGEABLE at 13:43, so there was a merge ref, f0d973d4, carrying the filter-free build.yml. Actions still declined. It only became unmergeable at 13:53, when 362045d2 landed.

What this changes for the revert: f090f12 was not ineffective, it was not retroactive. A trigger change cannot reach a PR whose head predates it, so the one test aimed at it could never have passed, and the negative result was a property of the experiment rather than of the fix. Merging the base into #28's head would have exercised it. That is a reason to reconsider the revert on its merits, not a reason to undo it, and it is your call either way.

Unaffected: the trim is still behaviour neutral, CI is still 8/8, and the test/known_gaps.lua conflict 362045d2 creates for #28 still stands, control and all.

@derek-miller

Copy link
Copy Markdown
Contributor

Plan change: #28 folds into this PR, along with the FL-18 fix, a refactor onto bitn's byte helpers, and a re-vendor of lua-bitn v0.6.3 (adds bit32.to_signed, finitelabs/lua-bitn#21). Every vector then agrees with the reference, so test/known_gaps.lua and its GAP/VERSION handling are removed.

I'm doing this work on this branch. Please don't push to agent/FL-16-wire-vectors or agent/FL-19-sign-extension, and don't start FL-18 separately.

@svc-finitelabs

Copy link
Copy Markdown
Contributor Author

Standing down. No pushes to agent/FL-16-wire-vectors or agent/FL-19-sign-extension, and FL-18 is not being started.

Confirmed nothing is in flight from this side before saying so: agent/FL-19-sign-extension is still at bda41b02 (2026-09-10 20:35:55Z) and this branch is at your own 362045d2 (2026-09-11 13:53:28Z). The open question I left you in #28 issuecomment-5635603674, the conflict and the three options, is withdrawn rather than pending, and the FL-19 and FL-16 sessions have been told the same so nothing lands from a sibling. FL-18 is annotated on the tracker too, since a session pulling that ticket would not otherwise see a plan change announced on a PR.

Two things I checked because they bear on work you are about to start, then I am out of the way.

The re-vendor has no artifact yet

lua-bitn's latest tag and latest release are both v0.6.2 (2026-08-15). v0.6.3 does not exist, and finitelabs/lua-bitn#21 is still open and unmerged at 7102500a, MERGEABLE. So the vendor bump is gated on merge, tag, release, in that order.

I reviewed #21 as a comment rather than an approve, on the auto-merge precedent from lua-bitn#14 rather than anything wrong with the change. If that review is what the merge is waiting on, say so and I will convert it to an approve.

Dropping the version tier is genuinely asserted, and I would not have assumed it

This is the part of the plan that could have quietly gone wrong, so I traced it rather than trusting the shape.

version_dependent entries are printed and never asserted. test/wire_vectors_test.lua:177-182 counts the entry, notes "agrees here" or "differs here", and returns before any check. So today's green on 5.1, 5.2 and LuaJIT says nothing about the six FL-19 entries. Removing the tier converts them from narration into hard assertions, which is only safe if something actually exercises them on the interpreters where FL-19 is wrong.

It does, and the path is not obvious:

  • The Check job that prints the vector summary pins luaVersion: '5.4', so that job alone would leave the claim untested on exactly the interpreters that matter.
  • The test job is the one that covers it. Matrix is 5.1, 5.2, 5.3, 5.4, luajit-2.0, luajit-2.1, with luajit-2.1 pinned to luajit-openresty.
  • It runs make test-all. There is no test-all target; it falls through the test-% pattern rule to ./run_tests.sh all, which globs test/*_test.lua, so wire_vectors_test.lua is included, and runs every module in both native and fallback math modes.

So an FL-19 fix that is only correct on 5.3+ integer builds fails the matrix once the tier is gone, which is the behaviour you want. Worth knowing which job to read when it goes red, because the summary line you would instinctively check is the 5.4 one.

Same check for the other tier: the six strict entries are all FL-18, and that tier already fails a listed case that starts passing, so those cannot be removed early without the run going red.

The to_signed swap is exact

pb.zigzag_decode32 at src/protobuf/init.lua:483-488 and bit32.to_signed at src/bitn/bit32.lua:52-57 are the same three lines, so that site is a clean substitution with no behaviour change.

One thing to protect in review: neither masks its input, and both pass an already negative value straight through. That is required rather than lax, because on LuaJIT the raw_bxor and raw_rshift output feeding it is already signed. bit32.mask sits directly below to_signed in the same file, which makes "shouldn't this mask first" an easy review note to accept. The to_signed(-1) vector in #21 is what pins the passthrough, and it looks redundant next to to_signed(0xFFFFFFFF) until you know that.

One disposition question

#28 will not auto-close when its content lands here, since a fold-in rewrites the commits and the branch stays ahead of its base under different SHAs. Happy to close it with a pointer back to this PR once you have pushed, or to leave it to you. Not touching it either way without a word from you.

@derek-miller derek-miller changed the title test: cover the wire format differentially against the reference implementation fix: sign the 32-bit codecs and cover the wire format against the reference Sep 11, 2026
@derek-miller

Copy link
Copy Markdown
Contributor

This is now the combined PR: the wire-format coverage, #28 (FL-19), the FL-18 sfixed32 fix, the codecs on bitn's byte helpers, and vendor/bitn.lua re-vendored from the lua-bitn v0.6.3 release asset. test/known_gaps.lua and the GAP/VERSION plumbing are gone; every vector must agree with the reference. The description is rewritten to match.

Please review the whole PR at the new head rather than the delta from 362045d, and approve if you're satisfied. I'll merge once it's green and approved.

@svc-finitelabs

Copy link
Copy Markdown
Contributor Author

Reviewed the whole PR at 9d3ed76, not the delta from 362045d. Verdict: approve. No defects found. Everything below was re-derived with a different instrument from the one the PR body used, because agreeing with an author by repeating their method is not corroboration.

Independently verified

1. vendor/bitn.lua is byte-identical to the v0.6.3 release asset. Downloaded the asset itself rather than trusting the commit message: 117,756 bytes, md5 fc52ff2a289d8de42efd271776b24b82, matching the vendored file exactly.

2. Old-vs-new codec differential, my own corpus. 1,792 enumerated cases per interpreter covering encode_varint (numbers and {high, low} pairs), fixed32/fixed64 encode and decode, float and double encode, decode over assembled IEEE words, round trips, zigzag, and varint decode. Run against 68c7374 and 9d3ed76 on Lua 5.5 and LuaJIT 2.1. Exactly two behavioural deltas, both intended:

  • encode_fixed32 over [-0x80000000, -1]: was bad argument #4 to 'char', now emits two's complement (-1 gives ffffffff, -2147483648 gives 00000080).
  • Out of range now raises fixed32 value out of range: N rather than string.char's message.

Every other case byte-identical on both interpreters. That is the same conclusion as your 390,483-case run, reached from a different corpus and a different harness.

3. Mutation testing: the suite discriminates on every fix. Eight mutations, each applied to a scratch copy and reverted with a clean-tree assert.

mutation Lua 5.5 LuaJIT 2.1
INT32/ENUM back to decode_varint FAIL FAIL
drop bit32_to_signed on int32 FAIL FAIL
SINT32 back to decode_varint PASS FAIL
drop the UINT32 branch FAIL FAIL
drop sfixed32 signing FAIL FAIL
encode_fixed32 rejects negatives again FAIL FAIL
float mantissa mask 0x7FFFFF to 0xFFFFFF FAIL FAIL
double high mask 0xFFFFF to 0x7FFFF FAIL FAIL

The sint32 row is the one worth keeping. Reverting that fix is invisible on an integer-model interpreter and only fails where the number model differs, so that vector's coverage rests on the 5.1, 5.2 and LuaJIT matrix entries. Dropping any of them would retire it silently rather than loudly. Measured on 5.5 and LuaJIT 2.1 only; I could not run 5.1 through 5.4 here.

4. make check-wire-vectors discriminates on encode, and correctly does not on decode. Ran it against three mutated trees. The two encode-side mutations fail it with explicit per-vector FAILs (34 of 37 compared). The FL-19 decode revert passes it at exit 0, because that target checks golden readback plus Lua's bytes parsed by the reference, and neither exercises Lua's decode. That split is right and the decode arm lives in the six matrix jobs. Noting it only so a future reader does not take the CI step as covering both directions by itself.

5. Bytewise comparison of Lua's encodings against the reference, all 37 vectors. 21 identical, 16 differ, fully accounted for:

  • 4 packing, the documented reason
  • 4 map entries with an omitted side, where the golden is assembled and Lua re-encodes canonically
  • 6 assembled varint goldens, same
  • 2 ordering

Spot-checked the cases most likely to hide an interop bug behind a message-level comparison, and they are byte-exact against the reference: int32 = -1 and optional_nested_enum = -1 both emit the ten-byte sign-extended form, int32 = -2147483648 gives 0880808080f8ffffffff01, uint32 = 4294967295 gives five bytes, sfixed32 = -1 gives 4dffffffff. So the int32-to-int64 evolution property holds, which a semantic comparison could not have told you.

One narrowing: the PR body gives "this encoder never packs" as the reason the encode arm is semantic. That is the reason for 4 of the 16. CLAUDE.md's wording is the complete one ("normalises packing, field order and map order away"). Only the PR body is narrow, and only for this thread's benefit.

6. Emission order is not merely different from the reference, it is not stable. pb.encode iterates pairs(messageSchema.fields) at src/protobuf/init.lua:639, so under LuaJIT the "maps with string keys" vector produced two distinct encodings across twelve runs, LuaJIT randomising string hash seeds per process. Pre-existing since cf60a48, untouched by this diff, spec-legal, and invisible to a message-level comparison. Not a blocker and nothing to do here. Worth a ticket only because it means the encode arm cannot be tightened to bytewise later without fixing emission order first, so the current asymmetry is permanent rather than provisional.

7. Fuzz beyond the enumerated corpus. 400 randomised messages over the full field set (nested, corecursive, recursive, maps, packed and unpacked repeated, oneofs, every scalar at its edges), serialised by the reference, decoded by Lua, re-encoded by Lua, re-parsed by the reference, compared as messages. 73 of 400 mismatch on both Lua 5.5 and LuaJIT 2.1, and every single one is the documented subnormal flush-to-zero, categorised mechanically rather than by eye. Zero failures from any other cause.

That is consistent with the generator's "floats are exactly representable: rounding belongs to the codec tests, not here", so the corpus excludes subnormals deliberately. Saying it out loud anyway: the one open wire-format defect the repo documents is by construction the one thing the new differential suite cannot see.

8. Provenance of the vendored proto. Diffed against upstream v29.3 src/google/protobuf/test_messages_proto3.proto: 39 lines removed, 0 added. Purely subtractive, exactly the six google/protobuf/* imports and every field typed from them. All removed field numbers fall in {120} union [201,219] union [301,324], matching the header comment, and retained numbers are upstream's.

9. No known_gaps residue. Grepped known_gaps, failure_list and GAP= across *.lua, *.sh, *.md, *.yml and the Makefile. Nothing outside the unrelated VERSION= in the build recipe.

10. Generated artifacts are not stale, on a toolchain CI does not use. make check-wire-vectors regenerated both files and diffed clean under protobuf 4.25.9 and protoc 35.1, so the generator's "byte-identical on every host" claim survives a different toolchain, not just a re-run of the one that produced them. 37 goldens read back, 37 encodings compared.

11. Local gates. format-check, check-types, check-schema and typecheck all pass. All 8 CI checks green at 9d3ed76.

Three things I liked enough to record: math_fallback_test.lua aborts rather than passing vacuously when there is no oracle, and separately asserts it bound the fallbacks and not the natives, which is conditional coverage that fails when it cannot run. testlib's MAX_REPORTED cap prints the overflow count instead of truncating silently. And the removed frexp-fallback bullet was genuinely stale, since the base commit 68c7374 is #26, which fixed it.

Two doc nits, neither blocking

(a) "Neither side is hand-written" is not quite true for 11 of the 37 vectors. The claim appears in three places: the PR body, render()'s header in tools/gen_wire_vectors, and CLAUDE.md's wire-vectors section. For the raw_case vectors the golden is assembled by varint_field_wire / map_entry_wire, not emitted by the reference serialiser; what the reference does is parse it and assert it matches the declared message. That is a better design than hand-computing an expectation, and both helpers' docstrings explain it honestly and at length. It is only the three one-line summaries that overstate, and since the generated file's own header carries one of them, a future reader chasing a vector mismatch would go looking for why the reference emits a ten-byte uint32, which it never does. Something like "goldens are the reference's own serialisation, or assembled payloads the reference is asserted to parse to the declared message" would close it.

(b) CLAUDE.md's "reports 32 duplicate-doc-field warnings" is off in two ways. Measured on lua-language-server 3.19.0, the version CI pins, with test/generated taken out of ignoreDir: the tool prints Diagnosis complete, 32 problems found, and the diagnostics are 42 duplicate-doc-field plus 6 duplicate-doc-alias. So 32 is the tool's own total rather than the count of that code, and that code is not the only one. "32 problems, mostly duplicate-doc-field" would be accurate.

What I did not do

  • Could not run Lua 5.1, 5.2, 5.3 or 5.4 locally. Only 5.5 and LuaJIT 2.1.1783773675, and the 2.1 matrix entry pins luajit-openresty, so my LuaJIT is not the build CI runs.
  • Did not run make lint (luacheck 1.2.0 will not load under Lua 5.5 on this box) or make build (amalg absent). CI's Check and Build Combined Module jobs cover both and are green.
  • Did not re-derive the performance numbers in the PR body.
  • Every mutation and fuzz run happened in scratch copies; the review clone was never dirtied.

I could not post this as a formal APPROVE: the PR is authored by svc-finitelabs[bot], which is the same app identity I review as, so GitHub rejects the review as self-approval and reviewDecision cannot leave REVIEW_REQUIRED without you. Treat this comment as the approve you asked for, and the formal one still has to be yours. Not merging: that is your call and your standing rule, and nothing here is a reason to wait.

@derek-miller derek-miller left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. The bot's review at 9d3ed76 is the substantive one; GitHub rejects it as a formal approval because the PR author is the same app identity, so this is the formal half.

Its two doc nits are taken in the head commit: the three one-line summaries no longer claim every golden is the reference's own serialization (eleven are assembled payloads the reference is asserted to parse), and CLAUDE.md's typecheck note gives the tool's total instead of one code's count. Generated header regenerated; check-wire-vectors still agrees both ways.

Its item 6, pb.encode iterating pairs(fields) so emission order is not stable under LuaJIT, is real and pre-existing; leaving it out of this PR and taking it as a follow-up.

No v0.6.8 from this merge: the release is held for FL-16 Part 1.

@derek-miller
derek-miller merged commit 0c5f840 into main Sep 11, 2026
8 checks passed
@derek-miller
derek-miller deleted the agent/FL-16-wire-vectors branch September 11, 2026 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant