defk-9p6.29: threshold_cross, an early-exit loop in Elixir, a kernel and Nx defn with EXLA

README.md

@@ -44,6 +44,7 @@ Speedups over an optimised Elixir baseline, measured end to end on Apple M4 Pro
 | ✅ | sorting packed keys, nearest of 500k locations, Monte Carlo slot simulation with `:rand` bit for bit, Gorilla encoding | 2.4–6.0× |
 | ✅ | a numeric CSV or a JSON array of numbers straight into packed f64s (NimbleCSV and Jason stay for anything else) | 2.7× |
 | ✅ | CSV to NimbleCSV's rows through a kernel that cleans the separators for `:binary.split` (rusty_csv's NIF is faster still) | 2.1× |
+| ✅ | an early-exit loop (the first packet past a byte budget), where Nx with EXLA pays ~40 µs a call and recompiles per shape | 2.5× |
 | ✅ | a fuzzy-search web service, Myers bit-parallel on both sides: `/health` latency stays at Elixir's while a blocking NIF raises it | [its own measurement](https://github.com/DanielLuning/defk/blob/main/examples/fuzzy_search/README.md) |
 | 🟡 | a dictionary-wide Levenshtein batch over all schedulers: Myers bit-parallel in Elixir is close | 1.5× |
 | 🟡 | small WebSocket frames, UTF-8 validation, substring search: the BIFs are native already | 1.0–1.5× |

docs/performance.md

@@ -911,6 +911,30 @@ Elixir answer for BLAKE2b; BLAKE3 is not in OpenSSL.
 - **Source:** [`blake.exs`](https://github.com/DanielLuning/defk/blob/main/examples/blake.exs), `defk-52v.2`.
 - **Status:** provisional.
 
+### 3.10 Nx with EXLA on an early-exit loop: ~40 us a call, a compile per shape, and plain Elixir is faster up to ~1M elements
+
+EXLA's strength is whole-tensor work (3.8, `cosine_topk`). On a sequential
+loop that stops early it loses to a kernel at every size, and to plain
+Elixir below about a million elements.
+
+- **Numbers** (the first index where a running i64 sum passes a budget,
+  passed at 25%; per call): 16 elements Elixir 0.14 us, kernel 0.05, EXLA
+  `while` 40-48 (compiled with `Nx.Defn.compile/3`, tensors resident); 64k
+  Elixir 15.7-17.0 us, kernel 4.4-4.6, EXLA 62-63; 16M Elixir 4.3-4.5 ms,
+  kernel 1.47-1.48, EXLA 4.3-4.4. The tensor form (`cumulative_sum` +
+  `argmax`, no early exit) is 1.5 ns an element and slower than Elixir at
+  every size; building the tensor from a binary adds up to 2x at 16M.
+- **Concurrency** (4096 elements): one process EXLA 24.5k calls/s, 48
+  processes 46.8k (the host device's lock); the kernel 3.2M and 22.3M.
+- **Variable length:** 300 calls of 298 lengths: the jit compiles each new
+  shape, ~6.4 ms (1.9 s in all; 20 ms once compiled). Padding to 8
+  power-of-two buckets: 0.25 s cold, 36 ms warm. Elixir 0.8-1.0 ms, the
+  kernel 0.4 ms.
+- **Conditions:** M-series Mac, OTP 29, Elixir 1.20.2, Nx and EXLA 0.13.1
+  CPU; two runs, one beside a `mix ci`.
+- **Source:** [`threshold_cross.exs`](https://github.com/DanielLuning/defk/blob/main/examples/threshold_cross.exs), `defk-9p6.29`; [`nx-boundary.md`](https://github.com/DanielLuning/defk/blob/main/docs/research/nx-boundary.md).
+- **Status:** provisional.
+
 ---
 
 ## 4. Parallelism and schedulers

docs/research/nx-boundary.md

@@ -231,10 +231,26 @@ normal schedulers, free the frame on process death) targets exactly that gap.
 
 ## Gaps and open questions
 
-- Not measured here: EXLA-CPU per-call latency, compile time for small
-  kernels, and `while` iteration cost on each compiler. A catalog entry that
-  writes one early-exit kernel in Elixir, defk, and `defn`+EXLA would turn the
-  inferences above into numbers (fits defk-9p6.3's "measure against Nx" style).
+- **Measured since** (`examples/threshold_cross.exs`, `defk-9p6.29`,
+  2026-10-20, Nx and EXLA 0.13.1 on the CPU, provisional): one early-exit
+  job (the first index where a running sum of i64 passes a budget) in
+  Elixir, defk, `defn` with `while` and `defn` with `Nx.cumulative_sum`.
+  These inferences above are now numbers:
+  - Per-call cost: EXLA-CPU takes 37-48 us a call on a 16-element tensor
+    (compiled with `Nx.Defn.compile/3`, tensors resident), against 0.05 us
+    for the kernel and 0.14 us for Elixir.
+  - `while` iteration cost on EXLA: about 1.0 ns a step at 16M elements
+    (4.3 ms for 4.2M steps), 3x the kernel's 0.35; the `cumulative_sum` form
+    costs 1.5 ns an element and loses to plain Elixir at every size.
+  - The per-device lock: 48 concurrent callers raised EXLA's throughput
+    only 1.6-1.9x over one caller (the kernel 7x).
+  - Compile per shape: about 6.4 ms per new input length; padding to 8
+    power-of-two buckets cut 300 variable-length calls from 1.9 s to
+    0.25 s cold, at 2x the warm cost.
+  - Responsiveness: a 1 ms ticker on one normal scheduler saw the same gap
+    (about 2 ms, the floor under load) for all three: EXLA's dirty NIF does
+    not hold a normal scheduler.
+  Still not measured: Evaluator and Torchx `while` costs, and GPU.
 - Whether XLA CPU parallelises a single executable across threads internally
   (intra-op thread pool) was not traced in source; it affects how much one
   EXLA call occupies the machine beyond its one dirty scheduler.

docs/scope.md

@@ -97,9 +97,13 @@ the call must stay preemptible and stop with its process, or when it is the
 one numeric job in an app that doesn't otherwise need Nx and a compiled
 backend. Some overlap with Nx is fine: a kernel that saves you adding EXLA for
 one loop is a good use of defk. A kernel takes the same packed layout as
-`Nx.to_binary/1`, so the two work side by side. The small-input and
-early-exit cases are not measured
-against EXLA yet.
+`Nx.to_binary/1`, so the two work side by side. On an early-exit loop (the
+first element where a running sum passes a limit) a kernel was faster than
+Nx with EXLA at every size measured: about 800x at 16 elements, where EXLA
+costs some 40 us a call, and 2.9x at 16M. Plain Elixir beat EXLA up to about
+a million elements. With 48 concurrent callers EXLA's throughput rose only
+1.6-1.9x (one host device, one lock), and an input of a new length costs
+EXLA a compile of about 6 ms (provisional).
 
 **A wrapper NIF (Rustler, Zigler or an existing binding) is the right
 choice** when the value is in an existing native library rather than in a

examples/README.md

@@ -43,7 +43,7 @@ script tried and what to take from it; the numbers behind them are in each
 script's header.
 
 <!-- scoreboard:start -->
-Measured on Apple M4 Pro (12 logical cores) in 4 runs, from 2026-10-19 to 2026-10-20. The newest at commit `71e29777`, OTP 29.0.3, Elixir 1.20.2: `mix defk.examples --bench --only tape_csv --snapshot --fresh`. A quiet measurement: other processes used at most 1.0 cores before and after it. A row from an earlier run says when it was measured, and at which commit.
+Measured on Apple M4 Pro (12 logical cores) in 5 runs, from 2026-10-19 to 2026-10-20. The newest at commit `3602f17d`, OTP 29.0.3, Elixir 1.20.2: `mix defk.examples --bench --only threshold_cross --snapshot --fresh`. A quiet measurement: other processes used at most 1.0 cores before and after it. A row from an earlier run says when it was measured, and at which commit.
 
 | | Example | The job | defk vs best Elixir | vs the package's NIF | In one line |
 |---|---|---|---|---|---|
@@ -69,8 +69,9 @@ Measured on Apple M4 Pro (12 logical cores) in 4 runs, from 2026-10-19 to 2026-1
 | ✅ | [`geo_nearest`](geo_nearest.exs) (measured 2026-10-19 at `8fa900b7`) | 10 nearest of 500k locations | **3.2×** | – | 14× in one process; a grid prefilter helps both sides |
 | ✅ | [`numeric_loader`](numeric_loader.exs) (measured 2026-10-20 at `0d6e7c78`) | numeric CSV of 26 MB into packed f64s | **2.7×** | – | a JSON array of numbers 4.0×; a fixed number grammar, and NimbleCSV or Jason for anything else |
 | ✅ | [`cdcl_sat_per_call`](cdcl_sat_per_call.exs) (measured 2026-10-19 at `8fa900b7`) | one SAT solve per call, 26–28 vars | **2.6×** | 1.6× picosat | ≈1× on tiny formulas; batch the solves when one question makes several |
+| ✅ | [`threshold_cross`](threshold_cross.exs) | first packet past a byte budget, 64k packets | **2.5×** | 13× Nx + EXLA `while` | EXLA costs ~40 µs a call and scales 1.9× over 48 callers; Elixir beats it up to ~1M packets |
 | ✅ | [`gorilla_encode`](gorilla_encode.exs) (measured 2026-10-19 at `684a3592`) | Gorilla encoding of 100-point blocks | **2.4×** | 4.2× gorilla_stream's NIF | close to the bar at this size; 4.1× at 1M points |
-| ✅ | [`tape_csv`](tape_csv.exs) | CSV of 20 MB to NimbleCSV's rows, a fresh process per parse | **2.1×** | 0.38× rusty_csv | in a long-lived process 0.8–2.1× run to run; an [i64] tape of field events is slower than the cleaned copy |
+| ✅ | [`tape_csv`](tape_csv.exs) (measured 2026-10-20 at `71e29777`) | CSV of 20 MB to NimbleCSV's rows, a fresh process per parse | **2.1×** | 0.38× rusty_csv | in a long-lived process 0.8–2.1× run to run; an [i64] tape of field events is slower than the cleaned copy |
 | ✅ | [`fuzzy_search/`](fuzzy_search) | a search service over 234k words behind Bandit | its own measurement, see its README | – | Myers on both sides; `/health` latency stays at Elixir's while a blocking NIF raises it |
 | 🟡 | [`batched_levenshtein`](batched_levenshtein.exs) (measured 2026-10-19 at `8fa900b7`) | one query against 234k words, 12 chunks each side | 1.5× | 13× ex_strsim per pair | Myers bit-parallel in plain Elixir is close; use it |
 | 🟡 | [`ws_masking`](ws_masking.exs) (measured 2026-10-19 at `8fa900b7`) | mask 125-byte WebSocket frames | 1.5× | 1.2× bandit_native | the call is the cost; 3.4× at 1 MiB, crossover 1024 B |
@@ -201,6 +202,14 @@ has them in full.
   best Elixir; rusty_csv's NIF builds the same sub-binaries in C, 2.6×
   faster again. Timed in one long-lived process, the verdict moved from
   0.8× to 2.1× run to run with the heap the earlier runs left.
+- [`threshold_cross.exs`](threshold_cross.exs): the first packet whose
+  running total passes a byte budget, an early-exit loop, in Elixir, a
+  kernel and Nx `defn` on EXLA (a `while` with a stop flag, and a
+  cumulative sum). The kernel is faster than EXLA at every size, 790× at
+  16 packets and 2.9× at 16M; optimised Elixir beats EXLA up to about 1M.
+  EXLA costs ~40 µs a call, scales 1.6–1.9× over 48 callers (one host
+  lock), and compiles each new input length (~6.4 ms; power-of-two buckets
+  bring 298 compiles down to 8).
 - [`lz4_snappy_decode.exs`](lz4_snappy_decode.exs): Snappy and LZ4 block
   decode from scratch on untrusted input, against snappyer, lz4_erl and
   nimble_lz4. Against the NIFs as installed from Hex (built with no `-O`)

examples/_tables.exs

@@ -165,6 +165,11 @@
       native: "{native} rusty_csv",
       line: "in a long-lived process 0.8–2.1× run to run; an [i64] tape of field events is slower than the cleaned copy"
     },
+    "threshold_cross" => %{
+      job: "first packet past a byte budget, 64k packets",
+      native: "{native} Nx + EXLA `while`",
+      line: "EXLA costs ~40 µs a call and scales 1.9× over 48 callers; Elixir beats it up to ~1M packets"
+    },
     "utf8_valid" => %{
       job: "validate 16 MB of UTF-8",
       line: "`String.valid?` is C already"
@@ -237,6 +242,11 @@
         "CSV to NimbleCSV's rows through a kernel that cleans the separators for `:binary.split` (rusty_csv's NIF is faster still)",
       examples: ["tape_csv"]
     },
+    %{
+      text:
+        "an early-exit loop (the first packet past a byte budget), where Nx with EXLA pays ~40 µs a call and recompiles per shape",
+      examples: ["threshold_cross"]
+    },
     %{
       text:
         "a fuzzy-search web service, Myers bit-parallel on both sides: `/health` latency stays at Elixir's while a blocking NIF raises it",