Skip to content

The engine returns an error when a thread cannot be created, instead of aborting (3.12.10) - #15

Merged
TJKoury merged 1 commit into
mainfrom
engine-thread-start-error
Nov 17, 2026
Merged

TJKoury merged 1 commit into
mainfrom
engine-thread-start-error

Conversation

@TJKoury

@TJKoury TJKoury commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

What was wrong

flatsql_start created the engine's threads with std::thread. The WASM build has no exceptions, so a thread that could not be created ended the whole instance in std::thread's exception path: RuntimeError: unreachable at abort <- __libcpp_verbose_abort <- __throw_system_error <- flatsql::p4::startThreads <- flatsql_start.

3.12.9 stops a browser from reaching that point with a short pool, but any host that declines a spawn still ended the instance. Node is one example: a pool capped by maxThreads, as shown below.

What changes

  • cpp/src/p4/internal.h: the engine's threads (the writers' indexers, the service threads, the slow-work and checkpoint threads) are an EngineThread. It wraps pthread_create, whose failure is a return value. It has the same joinable() and join() that the stop paths use, so they are unchanged.
  • cpp/src/p4/mailbox.cpp startThreads: every thread is created first and waits at a start gate.
    • When all of them exist, they run.
    • If one cannot be created, the gate sends the others home before they touch anything, they are joined, and flatsql_start returns FLATSQL_E_BUSY (-6).
    • started stays false, so the engine is as it was before the call. flatsql_start may be called again, and flatsql_stop works as for an engine that never started.
  • cpp/include/flatsql/flatsql.h: flatsql_start documents the -6 status. No new status code; the ABI is otherwise unchanged.
  • wasm/index.js: status -6 from start is a FlatSQLError (-6, "FlatSQL could not create its engine threads (the host has no thread left for it); nothing was started").
  • wasm/browser-guest.mjs: -6 from flatsql_start on an armed pool terminates the pool and becomes the FlatSQLThreadPoolError the guest already gives.
  • cpp/tests/p4/t_fts_repair.cpp: its helper that restarts the slow-work thread uses EngineThread::start.
  • Tests:
    • scripts/ci-engine-thread-start-e2e.mjs (new, run by scripts/ci-e2e.sh): a Node host whose thread pool is capped below the engine's threads (maxThreads: 4, spawnWaitMs: 100).
      • Before (3.12.9's wasm): RuntimeError: unreachable in startThreads.
      • Now: FlatSQLError -6 after 183 ms, then the same store opens on an uncapped host and answers SURFACE.
    • scripts/ci-browser-threads-e2e.mjs: new case "failing". Thread workers load and then answer the pool's probe as a worker that cannot instantiate the module does. Result: FlatSQLThreadPoolError 6 ms after the last failure, one attempt, no WASM fault.
  • WASM rebuilt the documented way: FLATBUFFERS_DIR=<FlatBuffers at 8af3053e> bash scripts/build-wasm.sh --linux, which builds in an ubuntu:22.04 container with wasi-sdk 30.0.
    • Before building, the same command on main's tree reproduced main's committed wasm/flatsql.wasm byte for byte (sha256 19361edf...a175).
    • The new wasm/flatsql.wasm is 3,763,174 bytes, sha256 00c92effd74897870b45345193dfa118c43cd79e72e559d9277e8e8ae3051b63.
    • wasm/integrity.json was written by the build. The runtime and demo bundles (npm run build) and the docs copies of the engine are regenerated.
  • Docs: CHANGELOG.md 3.12.10, docs/API-3.12.md (the -6 status on any host), docs/CI.md.

Checks at a58e3ee

  • Full scripts/ci-e2e.sh with FlatBuffers at the pin: 29/29 PASS. This covers the package build, the types, the packaged CLI, the stale-memory test, the new engine-thread-start test, imports, integrity, the native build and 20 native flows, among them t_million_objects.
  • npm test: PASS.
  • npm run test:browser: website E2E PASS (all 286 SDS generators), and the thread E2E PASS (slow, broken, failing, stalled).
  • Stack scripts/check.mjs --base origin/main --e2e "cmd node scripts/ci-browser-threads-e2e.mjs": CHECK PASSED.
  • Not run here:
    • the release workflow's own rebuild on x86_64 (its build of main reproduced main's bytes, as above);
    • a native test where pthread_create fails: no host-independent way to force it. The WASM path is covered by the Node test.

Release steps (repository rules: release commit on main, tag v, npm publish only through .github/workflows/npm-publish.yml)

  1. Merge engine-thread-start-error into main. It carries the 3.12.10 version, lockfile and CHANGELOG.
  2. Wait for ci.yml on main to pass.
  3. git tag v3.12.10 <merge sha> && git push origin v3.12.10.
  4. npm-publish.yml then:
    • rebuilds the WASM with wasi-sdk 30 and checks it equals the committed bytes;
    • runs scripts/ci-e2e.sh and npm run test:browser;
    • checks the committed runtime and demo assets;
    • publishes.
  5. Confirm with npm view flatsql@3.12.10 version.

🤖 Generated with Claude Code

… instead of aborting (3.12.10)

std::thread reports a thread it cannot create by throwing; the WASM build has no exceptions, so a host without a thread for the engine ended the instance (RuntimeError: unreachable at __throw_system_error <- startThreads <- flatsql_start).

- The engine's threads are EngineThread (pthread_create; its error is a return value). startThreads creates every thread first; each waits at a start gate. When all exist they run; if one cannot be created the gate sends the others home before they touch anything, they are joined, and flatsql_start returns FLATSQL_E_BUSY with the engine as before the call (started stays false), so flatsql_start may be called again and flatsql_stop works as for an engine that never started.
- wasm/index.js: status -6 from start is a FlatSQLError with its own message; the browser guest turns it into FlatSQLThreadPoolError.
- scripts/ci-engine-thread-start-e2e.mjs (in scripts/ci-e2e.sh): a Node host capped below the engine's threads; before: RuntimeError: unreachable in startThreads; now FlatSQLError -6, and the same store opens afterwards.
- scripts/ci-browser-threads-e2e.mjs: a thread worker that loads and then fails while arming rejects within a second, one attempt.
- wasm/flatsql.wasm rebuilt with scripts/build-wasm.sh --linux (wasi-sdk 30, FlatBuffers 8af3053e); the same build of main reproduces main's committed artifact byte for byte. integrity.json, runtime and demo bundles regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@TJKoury
TJKoury merged commit 5d5401f into main Nov 17, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant