Repository navigation
defk-9p6.30: tape_csv, CSV to rows through a kernel tape; the cleaned copy is 2.1-2.2x the best Elixir - #75
Merged
Merged
Conversation
…rapper row stays closed Two tape forms against NimbleCSV.RFC4180 and rusty_csv 0.4.6: an [i64] tape of field events walked with binary_part, and a cleaned copy (byte 30 and 31 separators) split twice by :binary.split. The copy is faster, and still 0.81-0.93x NimbleCSV on a 20 MB export: the scan takes 28-36 ms, building the rows in Elixir 135-314 ms. rusty_csv builds the same sub-binaries in C, 3.5-4x faster than NimbleCSV. Variant timings collect the heap before each run, as Defk.Bench.report/1 does: without that, term building looked 1.5-4x cheaper. Pre-registered marginal, measured poor.
…2.2x the best Elixir In one long-lived process the verdict moved from 0.81x to 2.08x run to run with the heap the earlier runs left. A fresh process per parse is a request handler's case and repeats to 2%. Two more Elixir variants (E2: :binary.split lines and fields, quoted lines walked with :binary.match; E3: quoted pieces joined); E2 is the baseline, 5-7% faster than NimbleCSV. The cleaned copy: 2.23x, good (pre-registered marginal). rusty_csv is 2.6x faster than the copy.
…runs mix defk.examples --bench --only tape_csv --snapshot --fresh from a clean tree: quiet, 2.09x over E2, rusty_csv 49.5 ms. The header keeps its line count, so the stamped C is the committed kernels'.
DanielLuning
pushed a commit
that referenced
this pull request
Oct 20, 2026
Merged
2 tasks
DanielLuning
pushed a commit
that referenced
this pull request
Oct 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merge #74 first. This branch builds on
defk-9p6.21/numeric-loader(the same README rows, scoreboard and gitleaks pin).What changed
examples/tape_csv.exs: CSV to the rowsNimbleCSV.RFC4180.parse_string/2returns, through a kernel. Two tape forms:scan/2: an[i64]tape of field events, which Elixir walks withbinary_part/3.normalize/1: a cleaned copy (byte 31 between fields, byte 30 after rows, quotes removed). Two:binary.split/3passes turn it into rows. This is the faster form and the verdict's defk side.:binary.splitlines and fields, with quoted lines walked by:binary.match(the baseline); E3 quoted pieces joined (no gain). Native: rusty_csv 0.4.6 (precompiled Rust NIF)._tables.exs, the READMEs (regenerated), anddocs/performance.md7.11.examples/scoreboard.json: a quiet row at 71e2977 (--freshfrom a clean tree).Results
20 MB, a 200k-row export with quoted fields. Four runs; the stored one is quiet, the others are provisional.
[i64]tapeHow it was checked
mix cigreen in the worktree (2305 passed).Agreement: both tape forms, their Elixir mode, E2, E3 and rusty_csv return exactly NimbleCSV's rows on 49 inputs. 6 bad inputs raise on every side.
Touches a promised surface (kernel subset, runtime contract, public API, error codes)? Then a
CHANGELOG.mdentry, and a deprecation for a rename or removal (docs/guides/stability.md).Touches
test/stability/? A frozen file is never edited; a release adds a directory (docs/agents/stability.md).