Repository navigation
The engine returns an error when a thread cannot be created, instead of aborting (3.12.10) - #15
Merged
Merged
Conversation
… instead of aborting (3.12.10) std::thread reports a thread it cannot create by throwing; the WASM build has no exceptions, so a host without a thread for the engine ended the instance (RuntimeError: unreachable at __throw_system_error <- startThreads <- flatsql_start). - The engine's threads are EngineThread (pthread_create; its error is a return value). startThreads creates every thread first; each waits at a start gate. When all exist they run; if one cannot be created the gate sends the others home before they touch anything, they are joined, and flatsql_start returns FLATSQL_E_BUSY with the engine as before the call (started stays false), so flatsql_start may be called again and flatsql_stop works as for an engine that never started. - wasm/index.js: status -6 from start is a FlatSQLError with its own message; the browser guest turns it into FlatSQLThreadPoolError. - scripts/ci-engine-thread-start-e2e.mjs (in scripts/ci-e2e.sh): a Node host capped below the engine's threads; before: RuntimeError: unreachable in startThreads; now FlatSQLError -6, and the same store opens afterwards. - scripts/ci-browser-threads-e2e.mjs: a thread worker that loads and then fails while arming rejects within a second, one attempt. - wasm/flatsql.wasm rebuilt with scripts/build-wasm.sh --linux (wasi-sdk 30, FlatBuffers 8af3053e); the same build of main reproduces main's committed artifact byte for byte. integrity.json, runtime and demo bundles regenerated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
flatsql_startcreated the engine's threads withstd::thread. The WASM build has no exceptions, so a thread that could not be created ended the whole instance instd::thread's exception path:RuntimeError: unreachableatabort <- __libcpp_verbose_abort <- __throw_system_error <- flatsql::p4::startThreads <- flatsql_start.3.12.9 stops a browser from reaching that point with a short pool, but any host that declines a spawn still ended the instance. Node is one example: a pool capped by
maxThreads, as shown below.What changes
cpp/src/p4/internal.h: the engine's threads (the writers' indexers, the service threads, the slow-work and checkpoint threads) are anEngineThread. It wrapspthread_create, whose failure is a return value. It has the samejoinable()andjoin()that the stop paths use, so they are unchanged.cpp/src/p4/mailbox.cppstartThreads: every thread is created first and waits at a start gate.flatsql_startreturnsFLATSQL_E_BUSY(-6).startedstays false, so the engine is as it was before the call.flatsql_startmay be called again, andflatsql_stopworks as for an engine that never started.cpp/include/flatsql/flatsql.h:flatsql_startdocuments the -6 status. No new status code; the ABI is otherwise unchanged.wasm/index.js: status -6 from start is aFlatSQLError(-6, "FlatSQL could not create its engine threads (the host has no thread left for it); nothing was started").wasm/browser-guest.mjs: -6 fromflatsql_starton an armed pool terminates the pool and becomes theFlatSQLThreadPoolErrorthe guest already gives.cpp/tests/p4/t_fts_repair.cpp: its helper that restarts the slow-work thread usesEngineThread::start.scripts/ci-engine-thread-start-e2e.mjs(new, run byscripts/ci-e2e.sh): a Node host whose thread pool is capped below the engine's threads (maxThreads: 4,spawnWaitMs: 100).RuntimeError: unreachableinstartThreads.FlatSQLError-6 after 183 ms, then the same store opens on an uncapped host and answers SURFACE.scripts/ci-browser-threads-e2e.mjs: new case "failing". Thread workers load and then answer the pool's probe as a worker that cannot instantiate the module does. Result:FlatSQLThreadPoolError6 ms after the last failure, one attempt, no WASM fault.FLATBUFFERS_DIR=<FlatBuffers at 8af3053e> bash scripts/build-wasm.sh --linux, which builds in an ubuntu:22.04 container with wasi-sdk 30.0.wasm/flatsql.wasmbyte for byte (sha256 19361edf...a175).wasm/flatsql.wasmis 3,763,174 bytes, sha256 00c92effd74897870b45345193dfa118c43cd79e72e559d9277e8e8ae3051b63.wasm/integrity.jsonwas written by the build. The runtime and demo bundles (npm run build) and the docs copies of the engine are regenerated.CHANGELOG.md3.12.10,docs/API-3.12.md(the -6 status on any host),docs/CI.md.Checks at a58e3ee
scripts/ci-e2e.shwith FlatBuffers at the pin: 29/29 PASS. This covers the package build, the types, the packaged CLI, the stale-memory test, the new engine-thread-start test, imports, integrity, the native build and 20 native flows, among themt_million_objects.npm test: PASS.npm run test:browser: website E2E PASS (all 286 SDS generators), and the thread E2E PASS (slow, broken, failing, stalled).scripts/check.mjs --base origin/main --e2e "cmd node scripts/ci-browser-threads-e2e.mjs": CHECK PASSED.pthread_createfails: no host-independent way to force it. The WASM path is covered by the Node test.Release steps (repository rules: release commit on main, tag v, npm publish only through
.github/workflows/npm-publish.yml)engine-thread-start-errorintomain. It carries the 3.12.10 version, lockfile and CHANGELOG.ci.ymlon main to pass.git tag v3.12.10 <merge sha> && git push origin v3.12.10.npm-publish.ymlthen:scripts/ci-e2e.shandnpm run test:browser;npm view flatsql@3.12.10 version.🤖 Generated with Claude Code