From 0d6e7c781342732ce90889058ff0a080a400318c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20L=C3=BCning?= Date: Tue, 20 Oct 2026 15:15:12 +0200 Subject: [PATCH 1/2] defk-9p6.21: numeric_loader, a fused CSV and JSON number loader into packed f64s Two passes (count newlines or commas, then parse into one alloc) with a written-down grammar: -?[0-9]+\.[0-9]+ for CSV fields, JSON numbers without exponents, at most 18 digits and a mantissa of at most 2^53, so one division gives the bits of String.to_float/1. Everything else raises. Good on both: CSV 26 MB 2.68-2.75x over a binary-matching Elixir parser (itself 5.7-5.9x NimbleCSV + String.to_float), JSON 2.2 MB of embedding floats 3.90-4.32x over the same parser, Jason and OTP's :json. The count pass is 6-11% of the time, so no growable output is needed. --- docs/performance.md | 27 ++ examples/_tables.exs | 9 + examples/numeric_loader.exs | 759 ++++++++++++++++++++++++++++++++++++ 3 files changed, 795 insertions(+) create mode 100644 examples/numeric_loader.exs diff --git a/docs/performance.md b/docs/performance.md index 0c2c99d2..e9f9b855 100644 --- a/docs/performance.md +++ b/docs/performance.md @@ -722,6 +722,33 @@ if g(st, olval + g(st, owblock + node)) == 1, do: walk(st, fl, node, nxt, o) # slow: clauses as lists rewritten per assignment (DPLL), or a tuple per table ``` +### 2.18 Numeric CSV and JSON into packed f64s: match bytes and append to one binary; a fused kernel is still 2.7-4.3x faster + +To load numbers into packed f64s, the fastest Elixir we found parses the +digits itself with binary matching (a mantissa and a power of ten, one +division per field, the bits of `String.to_float/1` below 2^53) and +appends each f64 to one binary. Building on the library's terms costs more. + +- **Numbers:** CSV, 26 MB (400k rows x 8 columns): NimbleCSV + + `String.to_float` 788-845 ms; `:binary.split` + `binary_to_float` + 385-403 ms; `:binary.match` for each delimiter + `binary_to_float` 324 ms; + binary matching with an iolist of 8-byte binaries 199-265 ms, appending + to one binary 138.8-142.3 ms. NimbleCSV's own iolist was about 2x faster + than appending there. The kernel (two passes) 50.5-53.0 ms: 2.68-2.75x. + JSON, 2.2 MB (153,600 floats): Jason + pack 26.2-27.5 ms, OTP's + `:json` + pack 16.4-16.8 ms, binary matching 13.7-17.1 ms (best of 15: + 9.0-9.1 ms); the kernel 2.08-2.34 ms: 3.90-4.32x. +- **Two passes are cheap:** counting the rows (newlines) or values (commas) + first and parsing into one alloc costs 6% (CSV) and 11% (JSON) of the + kernel's time, so the loader needs no growable output. The count loop + vectorises only with `wrap64(acc + 1)`: with the overflow check it took + 12.9 ms instead of 2.8. +- **Kernel shape:** one byte-at-a-time state machine beat a loop over the + fields with an `Enum.reduce_while` per field's digits by 15%. +- **Conditions:** M-series Mac, five runs; the last passed the quiet test. +- **Source:** [`numeric_loader.exs`](https://github.com/DanielLuning/defk/blob/main/examples/numeric_loader.exs), `defk-9p6.21`. +- **Status:** measured (the last run); the variant lines provisional. + --- ## 3. BIFs and OTP libraries that are already native diff --git a/examples/_tables.exs b/examples/_tables.exs index a22cacfd..11274a3a 100644 --- a/examples/_tables.exs +++ b/examples/_tables.exs @@ -123,6 +123,10 @@ job: "100M slot spins on all schedulers", line: "≈1× at 1M spins: short kernel chunks stay on few schedulers" }, + "numeric_loader" => %{ + job: "numeric CSV of 26 MB into packed f64s", + line: "a JSON array of numbers {speedup:numeric_loader_json}; a fixed number grammar, and NimbleCSV or Jason for anything else" + }, "out_of_scope_word_count" => %{ job: "strings in, a map out", line: "not expressible: the compiler refuses, as it should" @@ -218,6 +222,11 @@ "sorting packed keys, nearest of 500k locations, Monte Carlo slot simulation with `:rand` bit for bit, Gorilla encoding", examples: ["sort_u64_keys", "geo_nearest", "monte_carlo_rtp", "gorilla_encode"] }, + %{ + text: + "a numeric CSV or a JSON array of numbers straight into packed f64s (NimbleCSV and Jason stay for anything else)", + examples: ["numeric_loader"] + }, %{ text: "a fuzzy-search web service, Myers bit-parallel on both sides: `/health` latency stays at Elixir's while a blocking NIF raises it", diff --git a/examples/numeric_loader.exs b/examples/numeric_loader.exs new file mode 100644 index 00000000..6b07e7dd --- /dev/null +++ b/examples/numeric_loader.exs @@ -0,0 +1,759 @@ +# numeric_loader: load a numeric CSV, or a JSON array of numbers, straight +# into packed [f64] values +# +# The job: a metrics export or an ML feature table (10-500 MB of numeric +# CSV), or an embeddings response (a JSON array of floats per vector), read +# into packed f64s that the next kernel, Nx or Explorer consumes. People +# reach for a NIF (rusty_csv: SIMD scan, "3.5x" NimbleCSV on simple input; +# ecsv; jaxon) because NimbleCSV and Jason build a binary or a float term +# per field first. The research (docs/research/string-workloads.md, ranked +# #5) found that a kernel that only scans cannot beat :binary's BIFs and +# leaves Elixir to build the terms; it pays only as a fused loader that +# parses the digits straight into packed values. This example is that form. +# General CSV or JSON to terms stays with NimbleCSV and Jason. +# +# elixir examples/numeric_loader.exs +# +# Two DEFK_RESULT lines: numeric_loader_csv (the one mix defk.examples +# reads), then numeric_loader_json; the script exits 1 itself when the JSON +# one is incorrect or misses its expected verdict. +# +# Pre-registered (ticket): good for the fused loader on numeric data. +# Expected verdicts, as measured: CSV good, JSON good. +# +# The grammar (a limit, stated before the numbers). A CSV field is +# -?[0-9]+\.[0-9]+, the form String.to_float/1 accepts without an exponent; +# a JSON number is -?(0|[1-9][0-9]*)(\.[0-9]+)?. Both: at most 18 digits +# and a mantissa of at most 2^53, so mantissa / 10^k is one correctly +# rounded division of two exact doubles, the bits String.to_float/1 gives +# (Clinger's fast path). Rows end in \n or \r\n and have exactly ncols +# fields; JSON whitespace is space, \t, \n, \r. Everything else raises +# ArgumentError: quotes, empty fields, integers in CSV, exponents, more +# digits, a nested JSON value. A loader for a wider grammar would hand the +# fields it cannot parse to String.to_float/1; that is not tried here. +# +# Two passes, no growable output: pass 1 counts newlines (CSV) or commas +# (JSON) and pass 2 parses into an alloc of that size. Pass 1 is a +# vectorised byte count: 2.8 ms of the CSV's 50 ms (6%), 0.23 ms of the +# JSON's 2.1 ms (11%), under the 20% that would call for a growable [f64]. +# Its `acc + 1` needs wrap64 (the count is at most the input length): with +# the overflow check the loop did not vectorise and took 12.9 ms. The parse +# digits use wrap64 too, safe below 18 digits (69 -> 64 ms). +# +# Kernel shapes tried: one byte-at-a-time state machine (kept), and a loop +# over the rows x ncols fields with an Enum.reduce_while over each field's +# digits (15% slower: 56.5 against 49.1 ms for pass 2). +# +# Elixir baselines (optimised; all timed in each run; the fastest is the +# baseline): +# CSV E1 NimbleCSV.parse_string + String.to_float, packed through an +# iolist (appending to one binary was about 2x slower here); +# E2 :binary.split into lines and fields + :erlang.binary_to_float; +# E3 one pass of binary matching with the kernel's algorithm and +# grammar, appending each f64 to one binary (an iolist of 8-byte +# binaries was 1.7-1.9x slower). E3 is the baseline. Also tried: +# :binary.match for the next delimiter + binary_to_float per field +# (324 ms, 2.3x E3). +# JSON J1 Jason.decode! + pack; J2 OTP's :json.decode + pack; J3 binary +# matching like E3. J3 is the baseline; J2 is within 25% of it. +# Not tried: splitting the input at newlines across schedulers (both +# sides could; the kernel yields on a normal scheduler like Elixir). +# +# Realistic sizes: CSV 26 MB (400k rows x 8 columns of 1-6 decimals, the +# low end of a metrics export), with a sweep of 4k, 40k and 1.6M rows +# (106 MB); JSON 2.2 MB (100 embeddings of 1536 floats with ten decimals, +# ", " between them), with a sweep of 1, 10 and 1000 vectors. +# +# Agreement: every Elixir side, the kernels and their Elixir mode +# (`elixir_mode: true`, Loader.Kernels.__defk_elixir__/2) give the same +# packed bytes and row count on 26 CSV inputs (1 to 2000 rows, 1 to 8 +# columns, no trailing newline, empty) and 28 JSON inputs (to 5000 +# numbers, empty arrays, whitespace, -0 and -0.0); 411 edge fields (every +# digit count to 17, zeros, -0.0, leading zeros, mantissas at 2^53, \r\n +# rows) are bit-exact with String.to_float/1. 23 bad CSV and 26 bad JSON +# inputs raise ArgumentError on every side that has the grammar; Jason +# rejects 19 of the 26 (the rest are exponents and long mantissas, this +# loader's limits). +# +# Measured (M-series Mac, OTP 29, Elixir 1.20, 2026-10-20; the reports +# best of 15 in five runs, three of them through mix defk.examples; the +# variant lines best of 5 in the two direct runs). The last stored run +# passed the quiet test for both reports (other processes 0.3-0.6 cores); +# the CSV numbers of the others are provisional (up to 1.9 cores after). +# +# CSV, 26 MB E1 788-845 ms, E2 385-403 ms, E3 (best of 15) +# 138.8-142.3 ms; defk 50.5-53.0 ms +# (pass 1 2.8-2.9 ms) 2.68-2.75x good +# Sweep 4k rows 2.8-3.2x, 40k 2.7-3.1x, 1.6M 2.75-3.15x. +# Slicing one column out of the packed result in Elixir: +# 2.9-3.4 ms. +# JSON, 2.2 MB J1 26.2-27.5 ms, J2 16.4-16.8 ms, J3 13.7-17.1 ms +# (best of 15: 9.0-9.1 ms, median 17.8-19.7; the best is +# a rare run without a GC); defk 2.08-2.34 ms +# 3.90-4.32x good +# Sweep (three runs, best of 3, noisy) 1 vector 7.4-7.9x, +# 10 vectors 4.1-8.3x, 1000 vectors (22 MB) 4.0-4.1x. +# +# Recommendation: to load numeric CSV or JSON number arrays into packed +# values, a fused kernel is 2.7x (CSV) and 3.9-4.3x (JSON) faster than the +# best Elixir, and 15-16x NimbleCSV with String.to_float. Write the grammar +# down and reject the rest. For anything with quotes, strings, mixed types or +# a tree, NimbleCSV and Jason (or :json) stay the tools: the product is +# terms, and a kernel cannot build them. + +Mix.install([ + {:defk, path: Path.expand("..", __DIR__)}, + {:nimble_csv, "~> 1.3"}, + {:jason, "~> 1.4"} +]) + +defmodule Loader.Source do + # The kernels, as a `use` block for Loader.Kernels, compiled with + # `elixir_mode: true` (native by default; `__defk_elixir__/2` is the + # agreement check's oracle). + defmacro __using__(_) do + quote location: :keep do + # 2^53: the largest mantissa a double holds exactly + @two53 9_007_199_254_740_992 + + # Pass 1 of both loaders: how many times byte b occurs. + defk count_byte(s :: [u8], b :: i64) :: i64 do + for c <- s, reduce: 0 do + acc -> if c == b, do: wrap64(acc + 1), else: acc + end + end + + # m / scale with both exact (m <= 2^53, scale a power of ten up to + # 10^18) is one correctly rounded division: the double nearest the + # decimal, as String.to_float/1 gives. + defk decimal(m :: i64, scale :: i64, neg :: bool) :: f64 do + if m > @two53, do: raise(ArgumentError, "a mantissa above 2^53") + v = m / scale + if neg, do: -v, else: v + end + + # ---- CSV: rows of ncols fields, each -?[0-9]+\.[0-9]+ ---- + # + # Fields: at most 18 digits, a mantissa of at most 2^53. Rows end in + # "\n" or "\r\n"; the last may end without one. Anything else raises + # ArgumentError: a quote, an empty field, an integer, an exponent. + # States: 0 field start, 1 after '-', 2 integer digits, 3 after '.', + # 4 fraction digits, 5 after '\r'. + defk parse_csv(s :: [u8], ncols :: i64, rows :: i64) :: [f64] do + if ncols < 1, do: raise(ArgumentError, "ncols must be at least 1") + if rows < 0, do: raise(ArgumentError, "rows must not be negative") + out = alloc(f64, rows * ncols) + n = length(s) + + {m, scale, _nd, st, neg, col, idx} = + for i <- 0..(n - 1)//1, reduce: {0, 1, 0, 0, false, 0, 0} do + {m, scale, nd, st, neg, col, idx} -> + c = s[i] + + cond do + c >= 48 and c <= 57 -> + if st == 5, do: raise(ArgumentError, "a '\\r' not followed by '\\n'") + # 18 digits keep m * 10 + 9 and scale * 10 inside i64 + if nd >= 18, do: raise(ArgumentError, "a number with more than 18 digits") + + if st >= 3 do + {wrap64(m * 10 + (c - 48)), wrap64(scale * 10), nd + 1, 4, neg, col, idx} + else + {wrap64(m * 10 + (c - 48)), scale, nd + 1, 2, neg, col, idx} + end + + c == 46 -> + if st != 2, do: raise(ArgumentError, "a '.' not after a digit") + {m, scale, nd, 3, neg, col, idx} + + c == 45 -> + if st != 0, do: raise(ArgumentError, "a '-' inside a number") + {m, scale, nd, 1, true, col, idx} + + c == 44 -> + if st != 4, do: raise(ArgumentError, "a field that is not a decimal number") + if col + 1 >= ncols, do: raise(ArgumentError, "a row with too many fields") + out[idx] = decimal(m, scale, neg) + {0, 1, 0, 0, false, col + 1, idx + 1} + + c == 10 and st == 5 -> + {0, 1, 0, 0, false, 0, idx} + + c == 10 or c == 13 -> + if st != 4, do: raise(ArgumentError, "a field that is not a decimal number") + if col != ncols - 1, do: raise(ArgumentError, "a row with too few fields") + out[idx] = decimal(m, scale, neg) + {0, 1, 0, if(c == 13, do: 5, else: 0), false, 0, idx + 1} + + true -> + raise(ArgumentError, "a byte outside the number grammar") + {m, scale, nd, st, neg, col, idx} + end + end + + if st != 0 and st != 4, do: raise(ArgumentError, "the input ends inside a field") + if st == 4 and col != ncols - 1, do: raise(ArgumentError, "a row with too few fields") + if st == 4, do: out[idx] = decimal(m, scale, neg) + out + end + + # ---- JSON: one flat array of numbers ---- + # + # Each number is -?(0|[1-9][0-9]*)(\.[0-9]+)?, at most 18 digits and + # a mantissa of at most 2^53; whitespace is space, \t, \n, \r. An + # exponent, a string, a nested value raise ArgumentError. An integer + # is the float of its value, as Jason's integer packed into an f64 + # (so "-0" is 0.0, and "-0.0" is -0.0). Returns {values, count}. + # States: 0 before '[', 1 after '[', 2 after ',', 3 after '-', + # 4 integer digits, 5 after '.', 6 fraction digits, 7 after a number + # and whitespace, 8 after ']'. + defk parse_json(s :: [u8], cap :: i64) :: {[f64], i64} do + out = alloc(f64, cap) + n = length(s) + + {_m, _scale, _nd, st, _neg, k} = + for i <- 0..(n - 1)//1, reduce: {0, 1, 0, 0, false, 0} do + {m, scale, nd, st, neg, k} -> + c = s[i] + ws = c == 32 or c == 10 or c == 13 or c == 9 + + cond do + c >= 48 and c <= 57 -> + if nd >= 18, do: raise(ArgumentError, "a number with more than 18 digits") + + cond do + st == 5 or st == 6 -> + {wrap64(m * 10 + (c - 48)), wrap64(scale * 10), nd + 1, 6, neg, k} + + st == 4 -> + if m == 0, do: raise(ArgumentError, "a number with a leading zero") + {wrap64(m * 10 + (c - 48)), scale, nd + 1, 4, neg, k} + + st >= 1 and st <= 3 -> + {c - 48, 1, 1, 4, neg, k} + + true -> + raise(ArgumentError, "a number outside the array") + {m, scale, nd, st, neg, k} + end + + c == 46 -> + if st != 4, do: raise(ArgumentError, "a '.' not after a digit") + {m, scale, nd, 5, neg, k} + + c == 45 -> + if st != 1 and st != 2, do: raise(ArgumentError, "a '-' inside a number") + {m, scale, nd, 3, true, k} + + st == 4 and (c == 44 or c == 93 or ws) -> + out[k] = float(if neg, do: -m, else: m) + {0, 1, 0, if(c == 44, do: 2, else: if(c == 93, do: 8, else: 7)), false, k + 1} + + st == 6 and (c == 44 or c == 93 or ws) -> + out[k] = decimal(m, scale, neg) + {0, 1, 0, if(c == 44, do: 2, else: if(c == 93, do: 8, else: 7)), false, k + 1} + + ws -> + if st == 3 or st == 5, do: raise(ArgumentError, "whitespace inside a number") + {m, scale, nd, st, neg, k} + + c == 44 -> + if st != 7, do: raise(ArgumentError, "a ',' not after a number") + {m, scale, nd, 2, neg, k} + + c == 93 -> + if st != 7 and not (st == 1 and k == 0), + do: raise(ArgumentError, "a ']' not after a number") + + {m, scale, nd, 8, neg, k} + + c == 91 -> + if st != 0, do: raise(ArgumentError, "a nested array") + {m, scale, nd, 1, neg, k} + + true -> + raise(ArgumentError, "a byte outside the number grammar") + {m, scale, nd, st, neg, k} + end + end + + if st != 8, do: raise(ArgumentError, "the input is not one closed array") + {out, k} + end + end + end +end + +defmodule Loader.Kernels do + use Defk, elixir_mode: true + use Loader.Source +end + +defmodule Loader.Calls do + alias Loader.Kernels + + # The two-pass loaders: pass 1 counts rows (newlines, plus a last line + # without one) or values (commas plus one), pass 2 parses into an alloc of + # that size. Returns {packed f64 values in row-major order, rows}. + def load_csv(csv, ncols) do + rows = count_rows(csv) + {Kernels.parse_csv(csv, ncols, rows), rows} + end + + def count_rows(<<>>), do: 0 + + def count_rows(csv) do + lines = Kernels.count_byte(csv, ?\n) + if :binary.last(csv) == ?\n, do: lines, else: lines + 1 + end + + def load_json(json) do + {vals, k} = Kernels.parse_json(json, Kernels.count_byte(json, ?,) + 1) + {binary_part(vals, 0, 8 * k), k} + end + + # A column of a row-major load: column j of ncols. + def column(vals, ncols, j) do + {before, rest} = {8 * j, 8 * (ncols - 1 - j)} + + for <<_::binary-size(^before), x::binary-size(8), _::binary-size(^rest) <- vals>>, + into: <<>>, + do: x + end +end + +defmodule Loader.Elixir do + NimbleCSV.define(Loader.CSV, separator: ",", escape: "\"") + + # ---- CSV ---- + + # E1: NimbleCSV to rows of binaries, String.to_float on each field, an + # iolist (appending to one binary here was about 2x slower) + def csv_nimble(csv, ncols) do + rows = Loader.CSV.parse_string(csv, skip_headers: false) + + bin = + IO.iodata_to_binary( + for row <- rows do + length(row) == ncols || raise ArgumentError, "a row with the wrong field count" + for f <- row, do: <> + end + ) + + {bin, length(rows)} + end + + # E2: :binary.split into lines, then fields, then :erlang.binary_to_float + def csv_split(csv, ncols) do + lines = :binary.split(csv, "\n", [:global, :trim]) + + bin = + for line <- lines, reduce: <<>> do + acc -> + fields = :binary.split(line, ",", [:global]) + length(fields) == ncols || raise ArgumentError, "a row with the wrong field count" + + for f <- fields, + reduce: acc, + do: (acc -> <>) + end + + {bin, length(lines)} + end + + # E3: one pass of binary matching with the kernel's algorithm and grammar, + # appending to one binary (an iolist of 8-byte binaries was 1.7-1.9x slower) + def csv_match(csv, ncols), do: field(csv, ncols, 0, <<>>, 0) + + defp field(<<>>, _ncols, 0, acc, rows), do: {acc, rows} + + defp field(<>, ncols, col, acc, rows), + do: int(rest, ncols, col, acc, rows, true, 0, 0) + + defp field(<> = s, ncols, col, acc, rows) when c in ?0..?9, + do: int(s, ncols, col, acc, rows, false, 0, 0) + + defp field(_, _, _, _, _), do: raise(ArgumentError, "a field that is not a decimal number") + + defp int(<>, ncols, col, acc, rows, neg, m, nd) when c in ?0..?9 and nd < 18, + do: int(rest, ncols, col, acc, rows, neg, m * 10 + (c - ?0), nd + 1) + + defp int(<>, ncols, col, acc, rows, neg, m, nd) + when c in ?0..?9 and nd > 0 and nd < 18, + do: frac(rest, ncols, col, acc, rows, neg, m * 10 + (c - ?0), 10, nd + 1) + + defp int(_, _, _, _, _, _, _, _), + do: raise(ArgumentError, "a field that is not a decimal number") + + defp frac(<>, ncols, col, acc, rows, neg, m, scale, nd) + when c in ?0..?9 and nd < 18, + do: frac(rest, ncols, col, acc, rows, neg, m * 10 + (c - ?0), scale * 10, nd + 1) + + defp frac(s, ncols, col, acc, rows, neg, m, scale, _nd) do + acc = <> + + case s do + <> when col + 1 < ncols -> field(rest, ncols, col + 1, acc, rows) + <> when col == ncols - 1 -> field(rest, ncols, 0, acc, rows + 1) + <> when col == ncols - 1 -> field(rest, ncols, 0, acc, rows + 1) + <<>> when col == ncols - 1 -> {acc, rows + 1} + _ -> raise ArgumentError, "a field that is not a decimal number" + end + end + + defp decimal(m, _scale, _neg) when m > 9_007_199_254_740_992, + do: raise(ArgumentError, "a mantissa above 2^53") + + defp decimal(m, scale, true), do: -(m / scale) + defp decimal(m, scale, false), do: m / scale + + # ---- JSON ---- + + # J1: Jason, then pack + def json_jason(json), do: pack(Jason.decode!(json)) + + # J2: OTP's :json, then pack + def json_otp(json), do: pack(:json.decode(json)) + + defp pack(list) when is_list(list) do + {for(x <- list, reduce: <<>>, do: (acc -> <>)), length(list)} + end + + # J3: one pass of binary matching with the kernel's grammar + def json_match(json) do + case skip_ws(json) do + <> -> + case skip_ws(rest) do + <> -> closed(tail, <<>>, 0) + rest -> jvalue(rest, <<>>, 0) + end + + _ -> + raise ArgumentError, "the input is not one closed array" + end + end + + defp skip_ws(<>) when c in [?\s, ?\n, ?\r, ?\t], do: skip_ws(rest) + defp skip_ws(s), do: s + + defp jvalue(<>, acc, k), do: jint(rest, acc, k, true) + defp jvalue(s, acc, k), do: jint(s, acc, k, false) + + defp jint(<>, _acc, _k, _neg) when c in ?0..?9, + do: raise(ArgumentError, "a number with a leading zero") + + defp jint(<>, acc, k, neg) when c in ?0..?9, + do: jdigits(rest, acc, k, neg, c - ?0, 1) + + defp jint(_, _, _, _), do: raise(ArgumentError, "a value that is not a number") + + defp jdigits(<>, acc, k, neg, m, nd) when c in ?0..?9 and nd < 18, + do: jdigits(rest, acc, k, neg, m * 10 + (c - ?0), nd + 1) + + defp jdigits(<>, acc, k, neg, m, nd) when c in ?0..?9 and nd < 18, + do: jfrac(rest, acc, k, neg, m * 10 + (c - ?0), 10, nd + 1) + + defp jdigits(s, acc, k, neg, m, _nd) when m <= 9_007_199_254_740_992 do + jnext(s, <>, k + 1) + end + + defp jdigits(_, _, _, _, _, _), do: raise(ArgumentError, "a mantissa above 2^53") + + defp jfrac(<>, acc, k, neg, m, scale, nd) when c in ?0..?9 and nd < 18, + do: jfrac(rest, acc, k, neg, m * 10 + (c - ?0), scale * 10, nd + 1) + + defp jfrac(s, acc, k, neg, m, scale, _nd), + do: jnext(s, <>, k + 1) + + defp jnext(s, acc, k) do + case skip_ws(s) do + <> -> jvalue(skip_ws(rest), acc, k) + <> -> closed(rest, acc, k) + _ -> raise ArgumentError, "a ',' or ']' expected after a number" + end + end + + defp closed(rest, acc, k) do + if skip_ws(rest) == <<>>, do: {acc, k}, else: raise(ArgumentError, "bytes after the array") + end +end + +defmodule Loader.Data do + # A metrics export: ncols columns of fixed-point decimals with 1-6 + # fraction digits and 1-6 integer digits, some negative. + def csv(rows, ncols, seed \\ 1) do + :rand.seed(:exsss, {seed, 2, 3}) + + IO.iodata_to_binary( + for _ <- 1..rows//1 do + [Enum.map_intersperse(1..ncols, ",", fn _ -> csv_field() end), ?\n] + end + ) + end + + defp csv_field do + x = (:rand.uniform() - 0.3) * :math.pow(10, :rand.uniform(6) - 1) + :erlang.float_to_binary(x, decimals: :rand.uniform(6)) + end + + # Embedding vectors as an API returns them, flattened: values near 0 with + # ten decimals, ", " between them (json.dumps' default). + def json(count, seed \\ 1) do + :rand.seed(:exsss, {seed, 5, 7}) + vals = for _ <- 1..count//1, do: :erlang.float_to_binary(:rand.normal() * 0.03, decimals: 10) + IO.iodata_to_binary([?[, Enum.intersperse(vals, ", "), ?]]) + end + + # Fields that probe the grammar's edges: every digit count, zeros, -0.0, + # leading zeros, mantissas at 2^53. + def edge_fields do + fixed = + ~w(0.0 -0.0 0.5 -0.5 007.25 1.000000000000000 123456789012345.6 9007199254740.992 -9007199254740.992 0.00000000000000001 9999999.99999999) + + random = + for _ <- 1..400 do + # 17 digits at most: a "0" is put before a fraction with no integer digits + nd = :rand.uniform(17) + fd = :rand.uniform(nd) + digits = for _ <- 1..nd, into: "", do: <> + {i, f} = String.split_at(digits, nd - fd) + i = if i == "", do: "0", else: i + sign = if :rand.uniform(2) == 1, do: "-", else: "" + f = sign <> i <> "." <> f + if mantissa(f) <= 9_007_199_254_740_992, do: f, else: "1.5" + end + + fixed ++ random + end + + defp mantissa(f), do: f |> String.replace(["-", "."], "") |> String.to_integer() +end + +alias Loader.{Calls, Data, Kernels} + +# ---- agreement ---- + +bits = fn bin -> for <>, do: x end + +raises? = fn f -> + try do + f.() + false + rescue + ArgumentError -> true + end +end + +oracle_csv = fn csv, ncols -> + rows = Calls.count_rows(csv) + {Kernels.__defk_elixir__(:parse_csv, [csv, ncols, rows]), rows} +end + +oracle_json = fn json -> + {vals, k} = Kernels.__defk_elixir__(:parse_json, [json, Kernels.count_byte(json, ?,) + 1]) + {binary_part(vals, 0, 8 * k), k} +end + +edges = Data.edge_fields() + +edge_csv = + Enum.map_join( + Enum.chunk_every(edges, 3, 3, ["1.5", "1.5"]), + "", + &(Enum.join(&1, ",") <> "\r\n") + ) + +edge_expected = for f <- edges, into: <<>>, do: <> + +csv_cases = + [ + {Data.csv(2000, 8, 7), 8}, + {Data.csv(1, 1, 3), 1}, + {"1.5", 1}, + {"1.5\n", 1}, + {"", 3}, + {edge_csv, 3} + ] ++ + for(r <- 1..20, do: {Data.csv(r, rem(r, 5) + 1, r), rem(r, 5) + 1}) + +csv_agree = + Enum.all?(csv_cases, fn {csv, ncols} -> + k = Calls.load_csv(csv, ncols) + + k == Loader.Elixir.csv_match(csv, ncols) and k == oracle_csv.(csv, ncols) and + (csv == edge_csv or + (k == Loader.Elixir.csv_nimble(csv, ncols) and k == Loader.Elixir.csv_split(csv, ncols))) + end) + +{edge_vals, _} = Calls.load_csv(edge_csv, 3) +csv_edges = bits.(binary_part(edge_vals, 0, byte_size(edge_expected))) == bits.(edge_expected) + +bad_csv = + [ + {"1", 1}, + {"1.", 1}, + {".5", 1}, + {"1e5", 1}, + {"1.0e5", 1}, + {"\"1.5\"", 1}, + {"1.5,", 1}, + {",1.5", 2}, + {"1.5,2.5\n3.5", 2}, + {"1.5\n2.5,3.5\n", 1}, + {"a", 1}, + {"1.5\r", 1}, + {"1.5\r2.5", 1}, + {"--1.0", 1}, + {"1.2.3", 1}, + {"-.5", 1}, + {"1.5\n\n", 1}, + {"1234567890.123456789", 1}, + {"9007199254740.993", 1}, + {"1.5 ", 1}, + {" 1.5", 1}, + {"1.5", 0}, + {"+1.5", 1} + ] + +csv_bad = + Enum.all?(bad_csv, fn {csv, ncols} -> + raises?.(fn -> Calls.load_csv(csv, ncols) end) and raises?.(fn -> oracle_csv.(csv, ncols) end) and + raises?.(fn -> Loader.Elixir.csv_match(csv, ncols) end) + end) + +json_cases = + [ + Data.json(5000, 2), + Data.json(1, 1), + "[]", + " [ ] ", + "[1.5]", + "[0, -0, -0.0, 3, 10]", + "\n[1.25,\t-2.5 ,3.0]\r\n" + ] ++ + ["[" <> Enum.join(Enum.reject(edges, &String.match?(&1, ~r/^-?0\d/)), ",") <> "]"] ++ + for(c <- 1..20, do: Data.json(c, c)) + +json_agree = + Enum.all?(json_cases, fn json -> + k = Calls.load_json(json) + + bits.(elem(k, 0)) == bits.(elem(Loader.Elixir.json_jason(json), 0)) and + k == Loader.Elixir.json_jason(json) and k == Loader.Elixir.json_otp(json) and + k == Loader.Elixir.json_match(json) and k == oracle_json.(json) + end) + +bad_json = + ~w(1.5 [ ] [1.5 [1.5,] [,1.5] [01] [-01.5] [1e5] [1.0E5] [1.] [.5] [-] [--1] ["1.5"] [[1.5]] [1.5]x [1.5]] [1,,2] [null] [9007199254740.993] [1234567890123456789]) ++ + ["[1 2]", "[- 1]", "[1. 5]", ""] + +json_bad = + Enum.all?(bad_json, fn json -> + raises?.(fn -> Calls.load_json(json) end) and raises?.(fn -> oracle_json.(json) end) and + raises?.(fn -> Loader.Elixir.json_match(json) end) + end) + +# Jason rejects every bad input above except the exponents and the long +# mantissas, which are this loader's limits +jason_rejects = Enum.count(bad_json, &(not match?({:ok, l} when is_list(l), Jason.decode(&1)))) + +IO.puts(""" +agreement: + csv values #{csv_agree}, edges bit-exact with String.to_float #{csv_edges}, bad input #{csv_bad} (#{length(bad_csv)} cases) + json values #{json_agree}, bad input #{json_bad} (#{length(bad_json)} cases, Jason rejects #{jason_rejects}) +""") + +# ---- inputs ---- + +ncols = 8 +csv = Data.csv(400_000, ncols) +json = Data.json(153_600) +{csv_vals, csv_rows} = Calls.load_csv(csv, ncols) +{json_vals, json_n} = Calls.load_json(json) + +best = fn f, runs -> + for(_ <- 1..runs, do: elem(:timer.tc(f), 0)) |> Enum.min() |> Kernel./(1000) |> Float.round(3) +end + +show = fn title, rows -> + IO.puts(title) + for {label, f} <- rows, do: IO.puts(" #{String.pad_trailing(label, 40)} #{best.(f, 5)} ms") +end + +show.( + "csv, #{div(byte_size(csv), 1_000_000)} MB (#{csv_rows} rows x #{ncols} columns), best of 5:", + [ + {"E1 NimbleCSV + String.to_float", fn -> Loader.Elixir.csv_nimble(csv, ncols) end}, + {"E2 :binary.split + binary_to_float", fn -> Loader.Elixir.csv_split(csv, ncols) end}, + {"E3 binary matching, one binary", fn -> Loader.Elixir.csv_match(csv, ncols) end}, + {"defk pass 1 (count rows)", fn -> Calls.count_rows(csv) end}, + {"defk pass 2 (parse)", fn -> Kernels.parse_csv(csv, ncols, csv_rows) end}, + {"defk both passes", fn -> Calls.load_csv(csv, ncols) end}, + {"column 3 of the packed values (Elixir)", fn -> Calls.column(csv_vals, ncols, 3) end} + ] +) + +show.("json, #{Float.round(byte_size(json) / 1_000_000, 1)} MB (#{json_n} numbers), best of 5:", [ + {"J1 Jason.decode! + pack", fn -> Loader.Elixir.json_jason(json) end}, + {"J2 :json.decode + pack", fn -> Loader.Elixir.json_otp(json) end}, + {"J3 binary matching, one binary", fn -> Loader.Elixir.json_match(json) end}, + {"defk pass 1 (count commas)", fn -> Kernels.count_byte(json, ?,) end}, + {"defk both passes", fn -> Calls.load_json(json) end} +]) + +# ---- the verdicts: CSV first (the one mix defk.examples reads), then JSON; +# the script fails itself if JSON misses ---- + +csv_sweep = fn rows -> + c = Data.csv(rows, ncols, 11) + {fn -> Loader.Elixir.csv_match(c, ncols) end, fn -> Calls.load_csv(c, ncols) end} +end + +_rc = + Defk.Bench.report( + name: "numeric_loader_csv", + expected: :good, + baseline: :optimized, + runs: 15, + realistic_size: "26 MB: 400k rows x 8 columns", + elixir: fn -> Loader.Elixir.csv_match(csv, ncols) end, + defk: fn -> Calls.load_csv(csv, ncols) end, + check: fn -> + csv_agree and csv_edges and csv_bad and + {csv_vals, csv_rows} == Loader.Elixir.csv_match(csv, ncols) + end, + sweep: + for(r <- [4_000, 40_000, 1_600_000], do: Tuple.insert_at(csv_sweep.(r), 0, "#{r} rows")), + sweep_runs: 3, + reason: + "the kernel parses the digits straight into packed f64s; Elixir pays a function clause per byte, and NimbleCSV a sub-binary and a String.to_float per field" + ) + +json_sweep = fn count -> + j = Data.json(count, 13) + {fn -> Loader.Elixir.json_match(j) end, fn -> Calls.load_json(j) end} +end + +rj = + Defk.Bench.report( + name: "numeric_loader_json", + expected: :good, + baseline: :optimized, + runs: 15, + realistic_size: "2.2 MB: 100 embeddings of 1536 numbers", + elixir: fn -> Loader.Elixir.json_match(json) end, + defk: fn -> Calls.load_json(json) end, + check: fn -> + json_agree and json_bad and {json_vals, json_n} == Loader.Elixir.json_match(json) + end, + sweep: + for(c <- [1536, 15_360, 1_536_000], do: Tuple.insert_at(json_sweep.(c), 0, "#{c} numbers")), + sweep_runs: 3, + reason: + "Jason and :json build a list of floats the caller then packs; the kernel writes the packed f64s and builds no term per number" + ) + +if not rj.correct or rj.verdict != rj.expected do + IO.puts("#{rj.example}: verdict #{rj.verdict}, expected #{rj.expected}, correct #{rj.correct}") + System.halt(1) +end From 01c7956a1a8f71af05d4ea4f034bcfd2d9d9e3e4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20L=C3=BCning?= Date: Tue, 20 Oct 2026 15:18:50 +0200 Subject: [PATCH 2/2] defk-9p6.21: store numeric_loader's quiet row at 0d6e7c78; README rows and ranges mix defk.examples --bench --only numeric_loader --snapshot --fresh from a clean tree: both reports quiet, CSV 2.74x, JSON 4.03x. The header's ranges cover all eight runs (CSV 2.68-2.78x, JSON 3.90-5.87x: the Elixir side's best of 15 depends on whether a run escapes GC). The header keeps its line count, so the stamped C is the committed kernels'. The sort_u64_keys gitleaks pin moves to line 35. --- .gitleaksignore | 2 +- README.md | 3 ++- docs/performance.md | 10 +++++----- examples/README.md | 21 +++++++++++++++------ examples/numeric_loader.exs | 20 ++++++++++---------- examples/scoreboard.json | 1 + 6 files changed, 34 insertions(+), 23 deletions(-) diff --git a/.gitleaksignore b/.gitleaksignore index 52a10b5a..d6583158 100644 --- a/.gitleaksignore +++ b/.gitleaksignore @@ -6,4 +6,4 @@ examples/hashes.exs:generic-api-key:866 # Keys.Kernels module's generated C ("Keys.Kernels":""), not a credential. # The fingerprint is the row's line; a scoreboard with rows added above it # moves the line. -examples/scoreboard.json:generic-api-key:34 +examples/scoreboard.json:generic-api-key:35 diff --git a/README.md b/README.md index d0592170..b30c64d1 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ measurement rules are in [`examples/README.md`](https://github.com/DanielLuning/ what is planned and what stays out by design. -Speedups over an optimised Elixir baseline, measured end to end on Apple M4 Pro on 2026-10-19 (one machine, so measure your own job). ✅ defk helps; 🟡 and ❌ use Elixir. +Speedups over an optimised Elixir baseline, measured end to end on Apple M4 Pro from 2026-10-19 to 2026-10-20 (one machine, so measure your own job). ✅ defk helps; 🟡 and ❌ use Elixir. | | The job | defk vs best Elixir | |---|---|---| @@ -42,6 +42,7 @@ Speedups over an optimised Elixir baseline, measured end to end on Apple M4 Pro | ✅ | rolling sensor statistics, HyperLogLog updates, PCM mixing with clip prevention, a byte histogram | 8.4–10× | | ✅ | base64, hex and base58 codecs (4.2× the fast64 NIF); Snappy and LZ4 block decode on untrusted input | 5.1–6.1× | | ✅ | sorting packed keys, nearest of 500k locations, Monte Carlo slot simulation with `:rand` bit for bit, Gorilla encoding | 2.4–6.0× | +| ✅ | a numeric CSV or a JSON array of numbers straight into packed f64s (NimbleCSV and Jason stay for anything else) | 2.7× | | ✅ | a fuzzy-search web service, Myers bit-parallel on both sides: `/health` latency stays at Elixir's while a blocking NIF raises it | [its own measurement](https://github.com/DanielLuning/defk/blob/main/examples/fuzzy_search/README.md) | | 🟡 | a dictionary-wide Levenshtein batch over all schedulers: Myers bit-parallel in Elixir is close | 1.5× | | 🟡 | small WebSocket frames, UTF-8 validation, substring search: the BIFs are native already | 1.0–1.5× | diff --git a/docs/performance.md b/docs/performance.md index e9f9b855..c69d565b 100644 --- a/docs/performance.md +++ b/docs/performance.md @@ -722,7 +722,7 @@ if g(st, olval + g(st, owblock + node)) == 1, do: walk(st, fl, node, nxt, o) # slow: clauses as lists rewritten per assignment (DPLL), or a tuple per table ``` -### 2.18 Numeric CSV and JSON into packed f64s: match bytes and append to one binary; a fused kernel is still 2.7-4.3x faster +### 2.18 Numeric CSV and JSON into packed f64s: match bytes and append to one binary; a fused kernel is still 2.7-4x faster To load numbers into packed f64s, the fastest Elixir we found parses the digits itself with binary matching (a mantissa and a power of ten, one @@ -733,11 +733,11 @@ appends each f64 to one binary. Building on the library's terms costs more. `String.to_float` 788-845 ms; `:binary.split` + `binary_to_float` 385-403 ms; `:binary.match` for each delimiter + `binary_to_float` 324 ms; binary matching with an iolist of 8-byte binaries 199-265 ms, appending - to one binary 138.8-142.3 ms. NimbleCSV's own iolist was about 2x faster - than appending there. The kernel (two passes) 50.5-53.0 ms: 2.68-2.75x. + to one binary 138.8-143.7 ms. NimbleCSV's own iolist was about 2x faster + than appending there. The kernel (two passes) 50.5-53.0 ms: 2.68-2.78x. JSON, 2.2 MB (153,600 floats): Jason + pack 26.2-27.5 ms, OTP's `:json` + pack 16.4-16.8 ms, binary matching 13.7-17.1 ms (best of 15: - 9.0-9.1 ms); the kernel 2.08-2.34 ms: 3.90-4.32x. + 9.0-12.4 ms, as GC lands or not); the kernel 2.08-2.34 ms: 3.90-5.87x. - **Two passes are cheap:** counting the rows (newlines) or values (commas) first and parsing into one alloc costs 6% (CSV) and 11% (JSON) of the kernel's time, so the loader needs no growable output. The count loop @@ -745,7 +745,7 @@ appends each f64 to one binary. Building on the library's terms costs more. 12.9 ms instead of 2.8. - **Kernel shape:** one byte-at-a-time state machine beat a loop over the fields with an `Enum.reduce_while` per field's digits by 15%. -- **Conditions:** M-series Mac, five runs; the last passed the quiet test. +- **Conditions:** M-series Mac, eight runs; the last passed the quiet test. - **Source:** [`numeric_loader.exs`](https://github.com/DanielLuning/defk/blob/main/examples/numeric_loader.exs), `defk-9p6.21`. - **Status:** measured (the last run); the variant lines provisional. diff --git a/examples/README.md b/examples/README.md index 4b60d154..ad473e7b 100644 --- a/examples/README.md +++ b/examples/README.md @@ -43,7 +43,7 @@ script tried and what to take from it; the numbers behind them are in each script's header. -Measured on Apple M4 Pro (12 logical cores) in 2 runs, on 2026-10-19. The newest at commit `684a3592`, OTP 29.0.3, Elixir 1.20.2: `mix defk.examples --bench --only codecs,gorilla_encode,monte_carlo_rtp,sort_u64_keys --snapshot`. A quiet measurement: other processes used at most 1.0 cores before and after it. A row from an earlier run says when it was measured, and at which commit. +Measured on Apple M4 Pro (12 logical cores) in 3 runs, from 2026-10-19 to 2026-10-20. The newest at commit `0d6e7c78`, OTP 29.0.3, Elixir 1.20.2: `mix defk.examples --bench --only numeric_loader --snapshot --fresh`. A quiet measurement: other processes used at most 1.0 cores before and after it. A row from an earlier run says when it was measured, and at which commit. | | Example | The job | defk vs best Elixir | vs the package's NIF | In one line | |---|---|---|---|---|---| @@ -61,14 +61,15 @@ Measured on Apple M4 Pro (12 logical cores) in 2 runs, on 2026-10-19. The newest | ✅ | [`byte_histogram`](byte_histogram.exs) (measured 2026-10-19 at `8fa900b7`) | histogram of 16 MB, 12 chunks each side | **8.4×** | – | windowed entropy 11×; one process vs one call 11–18× | | ✅ | [`keccak`](keccak.exs) (measured 2026-10-19 at `8fa900b7`) | Keccak-256 of 10k × 32 B per call | **7.7×** | 0.63× ex_keccak | the NIF blocks its scheduler on a 16 MB message; SHA3-256 against `:crypto` is 0.81× | | ✅ | [`image_blur`](image_blur.exs) (measured 2026-10-19 at `8fa900b7`) | grayscale + 3×3 box blur, 1080p, 12 bands each side | **7.4×** | – | one process against one call 26–29× | -| ✅ | [`codecs`](codecs.exs) | base64 round trip of 1 MB | **6.1×** | 4.2× fast64 | hex 4.4×, base58 3.5× per batch; one key per call is no faster | -| ✅ | [`sort_u64_keys`](sort_u64_keys.exs) | sort 1M packed keys | **6.0×** | – | 3.8× when the caller needs a list back | +| ✅ | [`codecs`](codecs.exs) (measured 2026-10-19 at `684a3592`) | base64 round trip of 1 MB | **6.1×** | 4.2× fast64 | hex 4.4×, base58 3.5× per batch; one key per call is no faster | +| ✅ | [`sort_u64_keys`](sort_u64_keys.exs) (measured 2026-10-19 at `684a3592`) | sort 1M packed keys | **6.0×** | – | 3.8× when the caller needs a list back | | ✅ | [`lz4_snappy_decode`](lz4_snappy_decode.exs) (measured 2026-10-19 at `8fa900b7`) | Snappy block decode of 64 KB | **5.1×** | 2.4× snappyer as installed (-O0) | slower than the same libraries built -O2; LZ4 the same picture | -| ✅ | [`monte_carlo_rtp`](monte_carlo_rtp.exs) | 100M slot spins on all schedulers | **3.9×** | – | ≈1× at 1M spins: short kernel chunks stay on few schedulers | +| ✅ | [`monte_carlo_rtp`](monte_carlo_rtp.exs) (measured 2026-10-19 at `684a3592`) | 100M slot spins on all schedulers | **3.9×** | – | ≈1× at 1M spins: short kernel chunks stay on few schedulers | | ✅ | [`per_pair_distance`](per_pair_distance.exs) (measured 2026-10-19 at `8fa900b7`) | Levenshtein on 50k short pairs, one call each | **3.6×** | – | stay in Elixir for very short strings | | ✅ | [`geo_nearest`](geo_nearest.exs) (measured 2026-10-19 at `8fa900b7`) | 10 nearest of 500k locations | **3.2×** | – | 14× in one process; a grid prefilter helps both sides | +| ✅ | [`numeric_loader`](numeric_loader.exs) | numeric CSV of 26 MB into packed f64s | **2.7×** | – | a JSON array of numbers 4.0×; a fixed number grammar, and NimbleCSV or Jason for anything else | | ✅ | [`cdcl_sat_per_call`](cdcl_sat_per_call.exs) (measured 2026-10-19 at `8fa900b7`) | one SAT solve per call, 26–28 vars | **2.6×** | 1.6× picosat | ≈1× on tiny formulas; batch the solves when one question makes several | -| ✅ | [`gorilla_encode`](gorilla_encode.exs) | Gorilla encoding of 100-point blocks | **2.4×** | 4.2× gorilla_stream's NIF | close to the bar at this size; 4.1× at 1M points | +| ✅ | [`gorilla_encode`](gorilla_encode.exs) (measured 2026-10-19 at `684a3592`) | Gorilla encoding of 100-point blocks | **2.4×** | 4.2× gorilla_stream's NIF | close to the bar at this size; 4.1× at 1M points | | ✅ | [`fuzzy_search/`](fuzzy_search) | a search service over 234k words behind Bandit | its own measurement, see its README | – | Myers on both sides; `/health` latency stays at Elixir's while a blocking NIF raises it | | 🟡 | [`batched_levenshtein`](batched_levenshtein.exs) (measured 2026-10-19 at `8fa900b7`) | one query against 234k words, 12 chunks each side | 1.5× | 13× ex_strsim per pair | Myers bit-parallel in plain Elixir is close; use it | | 🟡 | [`ws_masking`](ws_masking.exs) (measured 2026-10-19 at `8fa900b7`) | mask 125-byte WebSocket frames | 1.5× | 1.2× bandit_native | the call is the cost; 3.4× at 1 MiB, crossover 1024 B | @@ -184,6 +185,14 @@ has them in full. - [`sort_u64_keys.exs`](sort_u64_keys.exs): heapsort kernel on 1M packed keys against decode plus `:lists.sort`; still good when the caller needs a list back, with the decode counted. +- [`numeric_loader.exs`](numeric_loader.exs): a numeric CSV (26 MB) and a + JSON array of embedding floats into packed f64s, in two passes (count, + then parse into one alloc; the count is 6–11% of the time). A + written-down grammar keeps the floats bit-exact with `String.to_float/1`. + The best Elixir is a binary-matching parser that appends to one binary, + 5.7–5.9× faster than NimbleCSV with `String.to_float`; the kernel is 2.7× + that on CSV, and about 4× on JSON against Jason, OTP's `:json` and the + same parser. - [`lz4_snappy_decode.exs`](lz4_snappy_decode.exs): Snappy and LZ4 block decode from scratch on untrusted input, against snappyer, lz4_erl and nimble_lz4. Against the NIFs as installed from Hex (built with no `-O`) @@ -289,7 +298,7 @@ Nothing here is excluded on principle: each row names the condition that would r | Job | Wrapped today | Why not from scratch | What would change it | |---|---|---|---| | Signing, key exchange, secret-dependent crypto (secp256k1, ed25519, X25519, AEAD) | libsodium, secp256k1, ring via Rustler; `:crypto` | The one rule says nothing about timing: clang `-O3` may turn a select into a branch or a table lookup into a secret-dependent access, and a process dump or yielded frame holds key material. Public-input verification is allowed but gains little. | A constant-time mode: a restricted subset checked at compile time, codegen verified with dudect or ctgrind in CI, frame zeroisation, and an external audit. Until then, `:crypto`. | -| Parsers that produce trees or term streams (HTML, Markdown, tree-sitter, JSON/CSV to terms, protobuf, msgpack) | html5ever, comrak, tree-sitter, jsont, rusty_csv, protox | The product is BEAM terms and building them is the cost: `docs/research/string-workloads.md` measured a 13x gap between scanning and term creation, and a NIF builds terms serially at the same price. A kernel cannot build terms, and if it could it would be no faster. | A "tape" output (flat `[i64]` of kind/start/len events, of which a fused CSV parser is one form), plus a benchmark showing tape plus Elixir tree building beats Jason or NimbleCSV end to end. BPE tokenisation to `[i64]` ids is the exception: its product is packed. | +| Parsers that produce trees or term streams (HTML, Markdown, tree-sitter, JSON/CSV to terms, protobuf, msgpack) | html5ever, comrak, tree-sitter, jsont, rusty_csv, protox | The product is BEAM terms and building them is the cost: `docs/research/string-workloads.md` measured a 13x gap between scanning and term creation, and a NIF builds terms serially at the same price. A kernel cannot build terms, and if it could it would be no faster. | A "tape" output (flat `[i64]` of kind/start/len events, of which a fused CSV parser is one form), plus a benchmark showing tape plus Elixir tree building beats Jason or NimbleCSV end to end. BPE tokenisation to `[i64]` ids is the exception: its product is packed, and so is a numeric CSV or JSON number array loaded into f64s (`numeric_loader`, good: about 2.7× and 4×). | | Regular expressions | `:re` (PCRE in C), Rust regex via Rustler | `:re` is already native and compiled once per pattern; a kernel cannot take a pattern as data without becoming an interpreter, and fixed-string search is `:binary.match`. | A specific pattern class with a known kernel form (glob, or a simple byte DFA compiled at Elixir compile time) and a benchmark showing `:re` loses on it. | | Large codecs and encoders (brotli, zstd encode, audio and video, PDF/typst rendering) | brotli, zstd, membrane_* FFmpeg/opus bindings, typst | The cost is spec surface and tuning, not the loop: 10^4-10^5 lines with SIMD everywhere. A kernel would be a worse implementation of a solved problem. | A narrow, named need: one compression level with fixed tables (zstd level 1 is the lz4-encode ticket plus FSE) or a decoder-only requirement for untrusted input. Each reopens one row, not the category. | | Systems: databases, dataframes, tensor compilers (SQLite, DuckDB, Polars/Explorer, XLA/EXLA, LibTorch) | exqlite, duckdbex, explorer, exla, torchx | The value is the library: query planners, columnar engines, autodiff, device backends. defk is a kernel, not a system. | Interop, not rewriting: zero-copy views over their packed buffers so a per-element loop beside them runs as a kernel. `docs/research/nx-boundary.md` says where that line is; file an interop ticket when a workload asks. | diff --git a/examples/numeric_loader.exs b/examples/numeric_loader.exs index 6b07e7dd..fec94937 100644 --- a/examples/numeric_loader.exs +++ b/examples/numeric_loader.exs @@ -76,26 +76,26 @@ # loader's limits). # # Measured (M-series Mac, OTP 29, Elixir 1.20, 2026-10-20; the reports -# best of 15 in five runs, three of them through mix defk.examples; the +# best of 15 in eight runs, six of them through mix defk.examples; the # variant lines best of 5 in the two direct runs). The last stored run # passed the quiet test for both reports (other processes 0.3-0.6 cores); # the CSV numbers of the others are provisional (up to 1.9 cores after). # # CSV, 26 MB E1 788-845 ms, E2 385-403 ms, E3 (best of 15) -# 138.8-142.3 ms; defk 50.5-53.0 ms -# (pass 1 2.8-2.9 ms) 2.68-2.75x good -# Sweep 4k rows 2.8-3.2x, 40k 2.7-3.1x, 1.6M 2.75-3.15x. +# 138.8-143.7 ms; defk 50.5-53.0 ms +# (pass 1 2.8-2.9 ms) 2.68-2.78x good +# Sweep 4k rows 2.8-3.5x, 40k 2.7-3.1x, 1.6M 2.75-3.2x. # Slicing one column out of the packed result in Elixir: # 2.9-3.4 ms. # JSON, 2.2 MB J1 26.2-27.5 ms, J2 16.4-16.8 ms, J3 13.7-17.1 ms -# (best of 15: 9.0-9.1 ms, median 17.8-19.7; the best is -# a rare run without a GC); defk 2.08-2.34 ms -# 3.90-4.32x good -# Sweep (three runs, best of 3, noisy) 1 vector 7.4-7.9x, -# 10 vectors 4.1-8.3x, 1000 vectors (22 MB) 4.0-4.1x. +# (best of 15: 9.0-12.4 ms, median 17.8-19.8; the best +# is a rare run without a GC); defk 2.08-2.34 ms +# 3.90-5.87x good +# Sweep (five runs, best of 3, noisy) 1 vector 7.4-9.0x, +# 10 vectors 4.1-11.4x, 1000 vectors (22 MB) 4.0-5.5x. # # Recommendation: to load numeric CSV or JSON number arrays into packed -# values, a fused kernel is 2.7x (CSV) and 3.9-4.3x (JSON) faster than the +# values, a fused kernel is 2.7x (CSV) and 3.9-5.9x (JSON) faster than the # best Elixir, and 15-16x NimbleCSV with String.to_float. Write the grammar # down and reject the rest. For anything with quotes, strings, mixed types or # a tree, NimbleCSV and Jason (or :json) stay the tools: the product is diff --git a/examples/scoreboard.json b/examples/scoreboard.json index 18d49b10..3a4872ad 100644 --- a/examples/scoreboard.json +++ b/examples/scoreboard.json @@ -24,6 +24,7 @@ {"name":"lz4_snappy_decode","quiet":true,"results":[{"baseline":"optimized","cache":"stored","correct":true,"crossover":"4 KB","defk_gap_ms":{"max":1.89,"p99":1.89},"defk_ms":0.054,"defk_spread_ms":{"max":0.076,"median":0.056,"runs":9},"elixir_gap_ms":{"max":2.044,"p99":2.044},"elixir_ms":0.277,"elixir_spread_ms":{"max":0.386,"median":0.357,"runs":9},"example":"snappy_decode","expected":"good","native_gap_ms":{"max":1.993,"p99":1.993},"native_label":"snappyer 1.2.10, as installed (-O0)","native_ms":0.131,"native_speedup":2.43,"native_spread_ms":{"max":0.15,"median":0.135,"runs":9},"realistic_size":"64 KB block (JSON-like events)","reason":"a tag loop with checked copies beats appending binary_part copies in Elixir; the -O2 snappy C++ library is faster than the kernel","speedup":5.13,"stamp":{"c":{"LzSnappy.Kernels":"27006396f8eeab35efe1ec81d217283f"},"commit":"8fa900b7","defk":"0.1.0","measured_at":"2026-10-19T11:38:24Z","quiet":{"limit":1.0,"other_cpu":[0.11,0.06],"quiet":true,"waited_s":36}},"sweep":[{"defk_ms":0.003,"elixir_ms":0.013,"size":"4 KB","speedup":4.33},{"defk_ms":0.053,"elixir_ms":0.257,"size":"64 KB","speedup":4.85},{"defk_ms":0.912,"elixir_ms":4.241,"size":"1 MB","speedup":4.65},{"defk_ms":3.014,"elixir_ms":13.084,"size":"4 MB","speedup":4.34}],"verdict":"good"},{"baseline":"optimized","cache":"stored","correct":true,"crossover":"4 KB","defk_gap_ms":{"max":1.674,"p99":1.674},"defk_ms":0.036,"defk_spread_ms":{"max":0.045,"median":0.038,"runs":9},"elixir_gap_ms":{"max":2.015,"p99":2.015},"elixir_ms":0.206,"elixir_spread_ms":{"max":0.37,"median":0.35,"runs":9},"example":"lz4_block_decode","expected":"good","native_gap_ms":{"max":4.757,"p99":4.757},"native_label":"lz4_erl 0.2.4 (liblz4 1.7.5), as installed (-O0)","native_ms":0.083,"native_speedup":2.31,"native_spread_ms":{"max":0.095,"median":0.084,"runs":9},"realistic_size":"64 KB block (JSON-like events)","reason":"a sequence loop with checked copies beats appending binary_part copies in Elixir; an optimised liblz4 build is several times faster than the kernel","speedup":5.72,"stamp":{"c":{"LzSnappy.Kernels":"27006396f8eeab35efe1ec81d217283f"},"commit":"8fa900b7","defk":"0.1.0","measured_at":"2026-10-19T11:38:25Z","quiet":{"limit":1.0,"other_cpu":[0.06,0.04],"quiet":true,"waited_s":0}},"sweep":[{"defk_ms":0.002,"elixir_ms":0.011,"size":"4 KB","speedup":5.5},{"defk_ms":0.041,"elixir_ms":0.359,"size":"64 KB","speedup":8.76},{"defk_ms":0.649,"elixir_ms":2.981,"size":"1 MB","speedup":4.59},{"defk_ms":2.585,"elixir_ms":19.198,"size":"4 MB","speedup":7.43}],"verdict":"good"}],"run":{"command":"mix defk.examples --bench --all --snapshot","commit":"8fa900b7","elixir":"1.20.2","measured_at":"2026-10-19T11:41:41Z","otp":"29.0.3","quiet_limit":1.0},"status":"ok"}, {"name":"matmul","quiet":true,"results":[{"baseline":"optimized","cache":"stored","correct":true,"crossover":64,"defk_ms":9.453,"defk_spread_ms":{"max":9.492,"median":9.488,"runs":3},"elixir_ms":289.911,"elixir_spread_ms":{"max":291.855,"median":290.865,"runs":3},"example":"matmul","expected":"good","native_label":"Nx 0.13.1 + EXLA 0.13.1 (CPU), from packed binaries","native_ms":1.941,"native_speedup":0.21,"native_spread_ms":{"max":2.033,"median":1.982,"runs":3},"realistic_size":"256x256","reason":"good against pure Elixir, but the tensor answer is Nx: with EXLA, Nx.dot is 6x faster than defk at 256x256 and 108x at 1024x1024 (defk wins only at 64x64, where EXLA's dispatch and copies cost more than the product); for tensor work use Nx with EXLA, defk fits a custom loop beside it","speedup":30.67,"stamp":{"c":{"Matmul.Kernels":"7abfa44b1a405b06c8ebb467b7c03df3"},"commit":"8fa900b7","defk":"0.1.0","measured_at":"2026-10-19T11:39:37Z","quiet":{"limit":1.0,"other_cpu":[0.09,0.11],"quiet":true,"waited_s":0}},"sweep":[{"defk_ms":0.169,"elixir_ms":3.558,"size":64,"speedup":21.05},{"defk_ms":10.507,"elixir_ms":290.765,"size":256,"speedup":27.67},{"defk_ms":81.396,"elixir_ms":2573.575,"size":512,"speedup":31.62},{"defk_ms":579.265,"elixir_ms":28155.342,"size":1024,"speedup":48.61}],"verdict":"good"}],"run":{"command":"mix defk.examples --bench --all --snapshot","commit":"8fa900b7","elixir":"1.20.2","measured_at":"2026-10-19T11:41:41Z","otp":"29.0.3","quiet_limit":1.0},"status":"ok"}, {"name":"monte_carlo_rtp","quiet":true,"results":[{"baseline":"optimized","cache":"stored","correct":true,"crossover":1000000,"defk_gap_ms":{"max":2.473,"p99":2.244},"defk_ms":363.559,"defk_spread_ms":{"max":374.07,"median":364.982,"runs":3},"elixir_gap_ms":{"max":2.05,"p99":2.022},"elixir_ms":1418.514,"elixir_spread_ms":{"max":1429.749,"median":1429.176,"runs":3},"example":"monte_carlo_rtp","expected":"good","realistic_size":"100M spins","reason":"a seeded PRNG and table lookups per spin; the kernel runs :rand's own exsss bit for bit, so totals equal Elixir's exactly","speedup":3.9,"stamp":{"c":{"RTP.Kernels":"3f09671017c5028810c41924e992983a"},"commit":"684a3592","defk":"0.1.0","measured_at":"2026-10-19T11:59:21Z","quiet":{"limit":1.0,"other_cpu":[0.07,0.03],"quiet":true,"waited_s":0}},"sweep":[{"defk_ms":5.081,"elixir_ms":15.287,"size":1000000,"speedup":3.01},{"defk_ms":37.053,"elixir_ms":145.916,"size":10000000,"speedup":3.94},{"defk_ms":371.051,"elixir_ms":1441.392,"size":100000000,"speedup":3.88}],"verdict":"good"}],"run":{"command":"mix defk.examples --bench --only codecs,gorilla_encode,monte_carlo_rtp,sort_u64_keys --snapshot","commit":"684a3592","elixir":"1.20.2","measured_at":"2026-10-19T11:59:29Z","otp":"29.0.3","quiet_limit":1.0},"status":"ok"}, + {"name":"numeric_loader","quiet":true,"results":[{"baseline":"optimized","cache":"stored","correct":true,"crossover":"4000 rows","defk_ms":51.791,"defk_spread_ms":{"max":53.987,"median":53.259,"runs":15},"elixir_ms":141.935,"elixir_spread_ms":{"max":163.841,"median":162.481,"runs":15},"example":"numeric_loader_csv","expected":"good","realistic_size":"26 MB: 400k rows x 8 columns","reason":"the kernel parses the digits straight into packed f64s; Elixir pays a function clause per byte, and NimbleCSV a sub-binary and a String.to_float per field","speedup":2.74,"stamp":{"c":{"Loader.Kernels":"0510bca86d2d922f87862f07f2820be1"},"commit":"0d6e7c78","defk":"0.1.0","measured_at":"2026-10-20T13:17:15Z","quiet":{"limit":1.0,"other_cpu":[0.4,0.36],"quiet":true,"waited_s":0}},"sweep":[{"defk_ms":0.482,"elixir_ms":1.532,"size":"4000 rows","speedup":3.18},{"defk_ms":5.215,"elixir_ms":16.043,"size":"40000 rows","speedup":3.08},{"defk_ms":213.404,"elixir_ms":587.08,"size":"1600000 rows","speedup":2.75}],"verdict":"good"},{"baseline":"optimized","cache":"stored","correct":true,"crossover":"1536 numbers","defk_ms":2.27,"defk_spread_ms":{"max":2.352,"median":2.312,"runs":15},"elixir_ms":9.158,"elixir_spread_ms":{"max":20.702,"median":19.827,"runs":15},"example":"numeric_loader_json","expected":"good","realistic_size":"2.2 MB: 100 embeddings of 1536 numbers","reason":"Jason and :json build a list of floats the caller then packs; the kernel writes the packed f64s and builds no term per number","speedup":4.03,"stamp":{"c":{"Loader.Kernels":"0510bca86d2d922f87862f07f2820be1"},"commit":"0d6e7c78","defk":"0.1.0","measured_at":"2026-10-20T13:17:16Z","quiet":{"limit":1.0,"other_cpu":[0.4,0.45],"quiet":true,"waited_s":0}},"sweep":[{"defk_ms":0.021,"elixir_ms":0.175,"size":"1536 numbers","speedup":8.33},{"defk_ms":0.229,"elixir_ms":1.952,"size":"15360 numbers","speedup":8.52},{"defk_ms":22.658,"elixir_ms":91.352,"size":"1536000 numbers","speedup":4.03}],"verdict":"good"}],"run":{"command":"mix defk.examples --bench --only numeric_loader --snapshot --fresh","commit":"0d6e7c78","elixir":"1.20.2","measured_at":"2026-10-20T13:17:16Z","otp":"29.0.3","quiet_limit":1.0},"status":"ok"}, {"name":"out_of_scope_word_count","quiet":true,"results":[{"baseline":"first_pass","cache":"stored","correct":true,"defk_ms":null,"elixir_ms":22.637,"elixir_spread_ms":{"max":24.366,"median":22.97,"runs":5},"example":"out_of_scope_word_count","expected":"out_of_scope","reason":"maps and strings are BEAM terms; kernels only work on numbers and packed arrays","speedup":null,"stamp":{"c":{},"commit":"8fa900b7","defk":"0.1.0","measured_at":"2026-10-19T11:40:21Z","quiet":{"limit":1.0,"other_cpu":[0.36,0.41],"quiet":true,"waited_s":4}},"verdict":"out_of_scope"}],"run":{"command":"mix defk.examples --bench --all --snapshot","commit":"8fa900b7","elixir":"1.20.2","measured_at":"2026-10-19T11:41:41Z","otp":"29.0.3","quiet_limit":1.0},"status":"ok"}, {"name":"pcm_mix","quiet":true,"results":[{"baseline":"optimized","cache":"stored","correct":true,"crossover":"1 ms","defk_gap_ms":{"max":2.077,"p99":2.077},"defk_ms":0.274,"defk_spread_ms":{"max":0.358,"median":0.314,"runs":7},"elixir_gap_ms":{"max":3.245,"p99":3.139},"elixir_ms":2.649,"elixir_spread_ms":{"max":3.364,"median":2.794,"runs":7},"example":"pcm_mix","expected":"good","native_gap_ms":{"max":41.031,"p99":41.031},"native_label":"membrane_audio_mix_plugin 0.16.5 NativeAdder","native_ms":0.74,"native_speedup":2.7,"native_spread_ms":{"max":0.772,"median":0.752,"runs":7},"realistic_size":"N = 4 streams, 48 kHz stereo s16le, 10 ms chunks (1 s of audio: 100 mix calls + flush)","reason":"the sum and half-wave scan run as one kernel per chunk with the held-back queue passed back in; pure Elixir pays per sample for list cells and binary appends","speedup":9.67,"stamp":{"c":{"PCM.Kernels":"bcd69d1d3e50fe86dba41b6aa562da16"},"commit":"8fa900b7","defk":"0.1.0","measured_at":"2026-10-19T11:40:35Z","quiet":{"limit":1.0,"other_cpu":[0.24,0.31],"quiet":true,"waited_s":0}},"sweep":[{"defk_ms":0.768,"elixir_ms":2.636,"size":"1 ms","speedup":3.43},{"defk_ms":0.382,"elixir_ms":2.674,"size":"4 ms","speedup":7.0},{"defk_ms":0.251,"elixir_ms":2.594,"size":"16 ms","speedup":10.33},{"defk_ms":0.246,"elixir_ms":2.422,"size":"64 ms","speedup":9.85},{"defk_ms":0.199,"elixir_ms":2.513,"size":"250 ms","speedup":12.63},{"defk_ms":0.195,"elixir_ms":2.487,"size":"1000 ms","speedup":12.75}],"verdict":"good"},{"baseline":"optimized","cache":"stored","correct":true,"defk_ms":0.2,"defk_spread_ms":{"max":0.235,"median":0.209,"runs":7},"elixir_ms":2.617,"elixir_spread_ms":{"max":3.856,"median":3.126,"runs":7},"example":"pcm_mix_1s","expected":"good","native_label":"membrane_audio_mix_plugin 0.16.5 NativeAdder","native_ms":0.669,"native_speedup":3.35,"native_spread_ms":{"max":0.748,"median":0.747,"runs":7},"realistic_size":"N = 4 streams, 48 kHz stereo s16le, 1 s chunks (1 mix call + flush)","reason":"the same job in 1 s chunks","speedup":13.09,"stamp":{"c":{"PCM.Kernels":"bcd69d1d3e50fe86dba41b6aa562da16"},"commit":"8fa900b7","defk":"0.1.0","measured_at":"2026-10-19T11:40:35Z","quiet":{"limit":1.0,"other_cpu":[0.31,0.21],"quiet":true,"waited_s":0}},"verdict":"good"}],"run":{"command":"mix defk.examples --bench --all --snapshot","commit":"8fa900b7","elixir":"1.20.2","measured_at":"2026-10-19T11:41:41Z","otp":"29.0.3","quiet_limit":1.0},"status":"ok"}, {"name":"per_pair_distance","quiet":true,"results":[{"baseline":"optimized","cache":"stored","correct":true,"defk_ms":11.429,"defk_spread_ms":{"max":11.687,"median":11.436,"runs":5},"elixir_ms":41.271,"elixir_spread_ms":{"max":42.276,"median":41.466,"runs":5},"example":"per_pair_distance","expected":"good","reason":"one NIF call per pair: the call overhead eats most of the kernel's advantage on 8-16 byte strings","speedup":3.61,"stamp":{"c":{"Pairs.Kernels":"67c9f0fe5b68fb00c708e66d6521fe44"},"commit":"8fa900b7","defk":"0.1.0","measured_at":"2026-10-19T11:40:38Z","quiet":{"limit":1.0,"other_cpu":[0.06,0.16],"quiet":true,"waited_s":0}},"verdict":"good"}],"run":{"command":"mix defk.examples --bench --all --snapshot","commit":"8fa900b7","elixir":"1.20.2","measured_at":"2026-10-19T11:41:41Z","otp":"29.0.3","quiet_limit":1.0},"status":"ok"},