diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 6afac24..b5633dc 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -13,7 +13,7 @@ jobs: - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 with: - node-version: '24.19.0' + node-version: '24.21.0' cache: npm - name: Install pinned Rust run: rustup toolchain install 1.99.0 --profile minimal diff --git a/README.md b/README.md index 134e16f..368ef42 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,7 @@ See [verification notes](docs/verification.md#compiler-pin-moved-to-the-v0660-re | --- | --- | --- | --- | | Linux x86_64 native HTTP | Passed locally | 18 cases passed over HTTP | Not deployed | | macOS arm64 native HTTP | Passed locally with the `--release` installer | 18 cases passed over HTTP | Not applicable | -| Wasm + generated JS, Node 24.19.0 | Passed locally | Same 18 cases + 1,000 repeated string calls | Not applicable | +| Wasm + generated JS, Node 24.21.0 | Passed locally | Same 18 cases + 1,000 repeated string calls; `/notes` through the JSPI store hooks | Not applicable | | Workers, Wrangler 4.147.0 / local workerd | Dry-run bundle passed; real `wrangler deploy` uploaded | Same 18 cases passed over HTTP locally and on the workers.dev edge | Deployed temporarily with Wrangler and with [Terraform](providers/cloudflare-workers/terraform/), verified, deleted | | ConoHa Docker / Compose | Image built and Compose started on macOS arm64 (Docker 29.6.1) and on a ConoHa VPS, x86_64 (Docker 29.2.1, Compose v5.0.2) | Same 18 cases passed against the container on both | VPS created with [Terraform](providers/conoha/terraform/), verified, destroyed | | Google Cloud Run container | linux/amd64 image built (QEMU on Apple silicon), pushed by digest; `replace --dry-run` and deploy passed | Same 18 cases passed from a VM inside the VPC with an ID token | Deployed temporarily with internal ingress + IAM, by gcloud and by [Terraform](providers/google-cloud-run/terraform/), verified, deleted | @@ -138,37 +138,58 @@ authentication emulators. [src/api.almd](src/api.almd) owns path routing, input validation, response status, JSON encoding, method errors and what to store. Its public boundary is strings in and one JSON envelope out: `step(method, target, body, reads) -> String` -(`handle(method, target, body)` is the same without storage). +(`handle(method, target, body)` is the same without storage). Each host runs +`step` to the end with its own storage, in Almide. - [src/native.almd](src/native.almd) maps Almide HTTP requests/responses, and performs storage itself -- [src/wasm.almd](src/wasm.almd) exposes the same functions to generated JS +- [src/wasm.almd](src/wasm.almd) runs `step` with two storage hooks and exports `serve` (and `handle`) to generated JS - [worker.js](providers/cloudflare-workers/worker.js) maps Workers Request/Response, with KV as the store - [adapters/node-wasm.mjs](adapters/node-wasm.mjs) shares one lazy Wasm initializer across Node function adapters -- [adapters/step.mjs](adapters/step.mjs) runs `step` for the JS hosts; [adapters/gcs-store.mjs](adapters/gcs-store.mjs) is Cloud Storage for Node +- [adapters/store.mjs](adapters/store.mjs) binds the hooks to a store for the JS hosts; [adapters/gcs-store.mjs](adapters/gcs-store.mjs) is Cloud Storage for Node - [tests/cases.mjs](tests/cases.mjs) and [tests/notes.mjs](tests/notes.mjs) are the shared expected-behavior fixtures The HTTP response contains the envelope's `body`; `status` and optional `allow` become HTTP metadata. The host adapters do not duplicate the application logic. -### Storage: Almide decides, the host performs +### Storage: Almide drives every route -The generated JS host has no asynchronous I/O, and a stock Wasm build refuses -Almide's HTTP client (`http.get` is E081 on `--target wasm`), while Workers KV, -`fetch` and Cloud Storage are asynchronous. So the shared code describes storage -instead of doing it: +`step` describes storage instead of doing it, so that each host can run it to the +end with its own store: -1. The host calls `step(method, target, body, "{}")`. -2. If the envelope has `"read": ["notes"]`, the host reads those keys and calls - `step` again with `reads` = `{"notes": "" | null}`. +1. Call `step(method, target, body, "{}")`. +2. If the envelope has `"read": ["notes"]`, read those keys and call `step` again + with `reads` = `{"notes": "" | null}`. 3. The envelope with `"status"` is final. If it has `"write": {"key", "value"}`, - the host stores it before answering; a failed read or write answers 503 + store it before answering; a failed read or write answers 503 `storage_unavailable`. -| Route | Store | Who performs it | +This loop is Almide code on every route. On native it calls `fs` or +`http.request` directly. On the JS hosts, Workers KV, `fetch` and Cloud Storage +only return Promises, so [src/wasm.almd](src/wasm.almd) declares two hooks, +`store_get(key)` and `store_put(key, value)`, as ordinary functions whose +`@extern` carries `returns: promise`. The generated JS then suspends +the module through JSPI (`WebAssembly.Suspending` / `promising`) until each hook +settles. Only `serve`, which reaches the hooks, returns a Promise; `handle` stays +synchronous. The JS side binds each hook to one store call +([adapters/store.mjs](adapters/store.mjs)) and does nothing else. + +JSPI is on by default in workerd, Node 25+ and Node **24.20.0+**; Node 24.19.0 and +earlier lack it, and `init()` then refuses with a message saying so. A managed +"Node 24" runtime is not enough by itself: on 2026-10-04 Google's `nodejs24` image +was still 24.19.0, so Cloud Run functions was verified on `nodejs26` (beta, Node +26.7.0). Check the patch version of the Lambda and Azure Functions runtimes +before relying on `/notes` there. + +`returns: promise` is not in the pinned v0.66.0 release. It ships in v0.67.0 +([almide/almide#3353](https://github.com/almide/almide/issues/3353), +[#3371](https://github.com/almide/almide/issues/3371)). Until then, build Almide's +`develop` and point `ALMIDE_BIN` at it. + +| Route | Store | Who calls it | | --- | --- | --- | | Native (ConoHa, Cloud Run container) | `STORE_DIR` files, or `GCS_BUCKET` objects | Almide: `fs`, or `http.request` with a metadata-server token | -| Cloudflare Workers | KV binding `NOTES` | `worker.js` | -| Cloud Run functions | `GCS_BUCKET` objects | `adapters/gcs-store.mjs` (`fetch`, no SDK) | +| Cloudflare Workers | KV binding `NOTES` | Almide, through the `store_get` / `store_put` hooks bound to `env.NOTES` | +| Cloud Run functions | `GCS_BUCKET` objects | Almide, through the hooks bound to `adapters/gcs-store.mjs` (`fetch`, no SDK) | | Lambda, Azure Functions | none configured | `/notes` answers 503 | `/notes` keeps one JSON list under the key `notes`: `POST {"text": ...}` (1–280 diff --git a/adapters/node-wasm.mjs b/adapters/node-wasm.mjs index 72c7085..d780f23 100644 --- a/adapters/node-wasm.mjs +++ b/adapters/node-wasm.mjs @@ -1,10 +1,11 @@ // Common Node host for the three function adapters. The compiler-generated JS -// owns the Wasm ABI; every request uses the same synchronous Almide function. +// owns the Wasm ABI; Almide's serve makes the storage calls through the hooks. import { readFile } from 'node:fs/promises'; -import { init, step } from '../build/app.js'; -import { runStep } from './step.mjs'; +import { init, serve } from '../build/app.js'; +import { storeHost } from './store.mjs'; import { gcsStore } from './gcs-store.mjs'; +const host = storeHost(); let initialization; // GCS_BUCKET selects Cloud Storage; without it, requests that need storage get 503. @@ -15,7 +16,7 @@ export async function callApi(method, target, body = '', { store = defaultStore throw new TypeError('callApi expects method, target and body strings'); } initialization ??= readFile(new URL('../build/app.wasm', import.meta.url)) - .then(bytes => init(bytes)); + .then(bytes => init(bytes, host.hooks)); await initialization; - return runStep(step, method, target, body, store); + return host.serveWith(serve, store, method, target, body); } diff --git a/adapters/step.mjs b/adapters/step.mjs deleted file mode 100644 index 1cd3fc9..0000000 --- a/adapters/step.mjs +++ /dev/null @@ -1,38 +0,0 @@ -// Runs the shared Almide `step` to a final envelope. Almide decides what to read -// and write; the host performs it. `store` is { get(key) -> string | null, -// put(key, value) } (sync or async), or null when the deployment has none. -// No Node or provider APIs here, so Workers and the Node adapters share it. -export const UNAVAILABLE = Object.freeze({ status: 503, body: { error: 'storage_unavailable' } }); - -export async function runStep(step, method, target, body, store) { - const reads = {}; - for (let round = 0; ; round++) { - const envelope = JSON.parse(step(method, target, body, JSON.stringify(reads))); - if (Array.isArray(envelope.read)) { - if (!store || round >= 2) return UNAVAILABLE; - try { - for (const key of envelope.read) reads[key] = (await store.get(key)) ?? null; - } catch (error) { - console.error('storage read failed:', error?.message ?? error); - return UNAVAILABLE; - } - continue; - } - const { write, ...result } = envelope; - if (write) { - if (!store) return UNAVAILABLE; - try { - await store.put(write.key, write.value); - } catch (error) { - console.error('storage write failed:', error?.message ?? error); - return UNAVAILABLE; - } - } - return result; - } -} - -export function memoryStore(initial = {}) { - const data = new Map(Object.entries(initial)); - return { data, get: key => data.get(key) ?? null, put: (key, value) => { data.set(key, value); } }; -} diff --git a/adapters/store.mjs b/adapters/store.mjs new file mode 100644 index 0000000..73a9707 --- /dev/null +++ b/adapters/store.mjs @@ -0,0 +1,55 @@ +// Binds the Almide storage hooks (src/wasm.almd) to a key -> string store. Almide +// decides what to read and write and makes each call itself; the generated JS +// suspends it through JSPI while the store's promise settles. `store` is +// { get(key) -> string | null, put(key, value) } (sync or async), or null when +// the deployment has none. No Node or provider APIs here, so Workers and the +// Node adapters share it. + +export function storeHost() { + let current = null; + let chain = Promise.resolve(); + + const hooks = { + js: { + async store_get(key) { + if (!current) return JSON.stringify({ error: 'no storage configured' }); + try { + return JSON.stringify({ value: (await current.get(key)) ?? null }); + } catch (error) { + return JSON.stringify({ error: String(error?.message ?? error) }); + } + }, + async store_put(key, value) { + if (!current) return 'no storage configured'; + try { + await current.put(key, value); + return ''; + } catch (error) { + return String(error?.message ?? error) || 'write failed'; + } + }, + }, + }; + + // The store is bound for one serve call at a time, so a call that is + // suspended on its store cannot see another request's store. + function serveWith(serve, store, method, target, body) { + const turn = chain.then(async () => { + current = store; + try { + return JSON.parse(await serve(method, target, body)); + } finally { + current = null; + } + }); + chain = turn.catch(() => {}); + return turn; + } + + return { hooks, serveWith }; +} + +export function memoryStore(initial = {}) { + const data = new Map(Object.entries(initial)); + return { data, get: key => data.get(key) ?? null, put: (key, value) => { data.set(key, value); } }; +} diff --git a/docs/verification.md b/docs/verification.md index 11aa2ca..624363e 100644 --- a/docs/verification.md +++ b/docs/verification.md @@ -302,6 +302,69 @@ Live: Not shown: behavior under concurrent writers. The single-key read-modify-write has no conditional write, so concurrent instances can lose a note. +## /notes on JS hosts: Almide calls the store through JSPI + +Date: 2026-10-04. Compiler: Almide `fix-3353` at `e9eb4bb90` (unreleased, +[almide/almide#3353](https://github.com/almide/almide/issues/3353)), built locally +and passed as `ALMIDE_BIN`. Node 24.21.0, Wrangler 4.147.0. + +The Wasm route used to return `step`'s reads and write to the JS host, which +performed them (`adapters/step.mjs`). Now `src/wasm.almd` runs the same loop as +the native host and calls the hooks `store_get` / `store_put` itself. Their +`@extern` carries `returns: promise`; the generated `app.d.ts` has +`serve(...): Promise` and `handle(...): string`. +`adapters/store.mjs` binds the hooks to a store. + +Local: + +1. `npm test`: 158/158. Under Node 22 (no JSPI) the storage tests fail, and + `init()` refuses with a message naming the missing JSPI +2. `npm run test:workers` (local workerd, fresh KV state): 29/29 +3. 20 concurrent `POST /notes` to local workerd: 20 × 201, and all 20 notes stored + with distinct `n` (one isolate runs one call at a time) +4. `npm run check:workers`: bundle 41.05 KiB, `NOTES` bound + +Live (Cloudflare Workers, deployed with `wrangler deploy`, then deleted): + +1. KV namespace provisioned from the id-less binding; 18 cases + the scenario + passed (29/29) +2. `wrangler tail`: 22 requests, every outcome `ok`, 0 exceptions +3. 20 concurrent `POST /notes`: 20 × 201, but the list grew by only 5. Requests + spread over several isolates, and each does a read-modify-write of one KV key + with no conditional write. This is the limitation the root README describes, + now measured. It is unchanged by this route +4. `wrangler delete` and `wrangler kv namespace delete`; the URL then answered + Cloudflare error 1042 (no Worker) + +Live (Cloud Run functions, `providers/google-cloud-functions/terraform`, a new +disposable project deleted afterwards; probe VM with no external IP, Private +Google Access, IAP SSH, as in the earlier Google runs): + +1. Apply with the default `nodejs24`: 16 resources in 161 s. Every request + answered 500, and the log showed the generated refusal: "this module awaits + async JS imports (store_get, store_put) through JSPI, and this runtime has no + WebAssembly.Suspending / WebAssembly.promising". Google's `nodejs24` image tags + ran up to `nodejs24_20260926_24_19_0_RC00`, which is Node 24.19.0. Locally, + JSPI is on by default from 24.20.0 (24.0.0 through 24.19.0: off; 24.20.0, + 24.21.0, 25.9.0, 26.10.0: on). On 24.19.0, `--experimental-wasm-jspi` enables + it, but Node refuses that flag in `NODE_OPTIONS`, and + `v8.setFlagsFromString` at run time does not install the API +2. Terraform gained `var.runtime` (default `nodejs24`), and `deploy.sh` gained + `GCP_BASE_IMAGE`. Re-applied with `runtime = nodejs26` (beta, image + `nodejs26_20260929_26_7_0_RC00`): 1 changed in 65 s +3. From the VM with an ID token: 18/18 as octet-stream, the `/notes` scenario + 11/11, and as JSON the same 3 known framework rejections. Without a token: 403 +4. The bucket's `notes` object held exactly the two saved notes + (`application/json`, 53 bytes), written by Almide through `store_put` +5. 20 concurrent `POST /notes` (max 3 instances, concurrency 1): 17 × 201 and + 3 × 503. The 503s were Almide's `storage_unavailable` after Cloud Storage + answered 429 to rapid writes of one object. Almide logged + `storage write failed: GCS write: HTTP 429` itself. 14 of the 17 accepted + notes were kept; the rest were lost to the read-modify-write race +6. `terraform destroy` removed 16 resources; the project was deleted + +CI's Node was 24.19.0, so it moved to 24.21.0. `engines` now says `>=24.20.0`. + ## Terraform for Cloudflare and Google Date: 2026-10-04, Terraform 1.14.9, providers cloudflare 5.26.0, google 8.5.0, @@ -354,7 +417,7 @@ the operator's /32 only); the provider installed with `go install` and ## Not established - Native x86_64 Docker build outside the ConoHa VPS -- `/notes` under concurrent writers on any route +- `/notes` under concurrent writers is safe on any route (measured on Workers: writes are lost across isolates) - ConoHa TLS/reverse proxy, restart behavior, production load, or plans smaller than `g2l-t-c4m4` - Azure Container Apps or ECS Fargate provider validation/deployment diff --git a/package.json b/package.json index 1bce0ff..cd87bcb 100644 --- a/package.json +++ b/package.json @@ -3,7 +3,7 @@ "private": true, "type": "module", "engines": { - "node": ">=22.0.0" + "node": ">=24.20.0" }, "scripts": { "build": "bash scripts/build.sh", diff --git a/providers/cloudflare-workers/README.md b/providers/cloudflare-workers/README.md index e6177cd..2156cff 100644 --- a/providers/cloudflare-workers/README.md +++ b/providers/cloudflare-workers/README.md @@ -34,11 +34,12 @@ for the exact evidence. ## /notes and KV -`/notes` is stored in the KV binding `NOTES`. The shared Almide `step` says which -key to read and what to write; `worker.js` performs it with `env.NOTES` (see the -root README, "Storage: Almide decides, the host performs"). The generated JS -host's environment is not a transparent bridge to Workers bindings, so the -adapter, not Almide, touches KV. +`/notes` is stored in the KV binding `NOTES`. Almide reads and writes it itself: +`src/wasm.almd` calls the hooks `store_get` / `store_put`, and `worker.js` binds +them to `env.NOTES` ([adapters/store.mjs](../../adapters/store.mjs)). KV only +returns Promises, so the hooks' `@extern` carries `returns: promise`, and the generated +JS suspends the module through JSPI until each KV call settles (root README, +"Storage: Almide drives every route"). `wrangler.jsonc` names the binding without an `id`, so `wrangler deploy` provisions a namespace called `almide-cloud-example-notes`, and `wrangler dev diff --git a/providers/cloudflare-workers/worker.js b/providers/cloudflare-workers/worker.js index d6d8b1e..b031dbc 100644 --- a/providers/cloudflare-workers/worker.js +++ b/providers/cloudflare-workers/worker.js @@ -1,9 +1,10 @@ import module from '../../build/app.wasm'; -import { init, step } from '../../build/app.js'; -import { runStep } from '../../adapters/step.mjs'; +import { init, serve } from '../../build/app.js'; +import { storeHost } from '../../adapters/store.mjs'; // One init per isolate. No filesystem/URL fallback and no handwritten Wasm ABI. -const ready = init(module); +const host = storeHost(); +const ready = init(module, host.hooks); // The NOTES KV namespace is the store; without the binding, /notes answers 503. const kvStore = kv => kv && { get: key => kv.get(key), put: (key, value) => kv.put(key, value) }; @@ -14,7 +15,7 @@ export default { const url = new URL(request.url); const body = request.method === 'GET' || request.method === 'HEAD' ? '' : await request.text(); - const response = await runStep(step, request.method, url.pathname, body, kvStore(env.NOTES)); + const response = await host.serveWith(serve, kvStore(env.NOTES), request.method, url.pathname, body); return Response.json(response.body, { status: response.status, ...(response.allow ? { headers: { Allow: response.allow } } : {}), diff --git a/providers/google-cloud-functions/README.md b/providers/google-cloud-functions/README.md index 7ac2cd0..6bbccc5 100644 --- a/providers/google-cloud-functions/README.md +++ b/providers/google-cloud-functions/README.md @@ -45,8 +45,14 @@ export GCS_BUCKET=YOUR_EXISTING_BUCKET With `GCS_BUCKET`, `deploy.sh` adds `--set-env-vars GCS_BUCKET=...` and [adapters/gcs-store.mjs](../../adapters/gcs-store.mjs) stores `/notes` as the object `notes` with `fetch` and the metadata-server token (no SDK dependency). Grant the -runtime account `roles/storage.objectUser` on that bucket only. The shared Almide -`step` decides the read and the write; the Node host performs them. +runtime account `roles/storage.objectUser` on that bucket only. Almide makes the +read and the write itself through the `store_get` / `store_put` hooks, which the +Node host binds to that store. The hooks need JSPI, which is on by default from +Node 24.20.0. On 2026-10-04 Google's `nodejs24` image was 24.19.0 (no JSPI), and +every request failed at `init()` with the generated message naming the missing +`WebAssembly.Suspending`. Until that image moves to 24.20 or later, deploy with +`nodejs26` (beta): `GCP_BASE_IMAGE=nodejs26` for `deploy.sh`, or +`-var runtime=nodejs26` for Terraform. From the repository root, `bash providers/google-cloud-functions/deploy.sh` only prints a reviewed, shell-escaped command. Adding `--execute` runs it and diff --git a/providers/google-cloud-functions/deploy.sh b/providers/google-cloud-functions/deploy.sh index 945b5b9..870dac9 100755 --- a/providers/google-cloud-functions/deploy.sh +++ b/providers/google-cloud-functions/deploy.sh @@ -15,7 +15,7 @@ source_dir="$root/build/packages/google-cloud-functions" echo 'Build and package google-cloud-functions first; see README.md' >&2; exit 1; } command=(gcloud run deploy "$GCP_SERVICE" --project "$GCP_PROJECT" --region "$GCP_REGION" - --source "$source_dir" --function almideApi --base-image nodejs24 + --source "$source_dir" --function almideApi --base-image "${GCP_BASE_IMAGE:-nodejs24}" --service-account "$GCP_SERVICE_ACCOUNT" --build-service-account "$GCP_BUILD_SERVICE_ACCOUNT" --no-allow-unauthenticated --invoker-iam-check --ingress internal --concurrency 1 --min-instances 0 --max-instances 3 --memory 256Mi --cpu 1 --timeout 30s) diff --git a/providers/google-cloud-functions/terraform/main.tf b/providers/google-cloud-functions/terraform/main.tf index 699042d..4333376 100644 --- a/providers/google-cloud-functions/terraform/main.tf +++ b/providers/google-cloud-functions/terraform/main.tf @@ -78,7 +78,7 @@ resource "google_cloudfunctions2_function" "api" { location = var.region build_config { - runtime = "nodejs24" + runtime = var.runtime entry_point = "almideApi" service_account = google_service_account.build.id source { diff --git a/providers/google-cloud-functions/terraform/variables.tf b/providers/google-cloud-functions/terraform/variables.tf index bf362b7..f2eb5e7 100644 --- a/providers/google-cloud-functions/terraform/variables.tf +++ b/providers/google-cloud-functions/terraform/variables.tf @@ -8,6 +8,14 @@ variable "region" { default = "asia-northeast1" } +variable "runtime" { + # JSPI (the async storage hooks) is on by default from Node 24.20. Google's + # nodejs24 image was 24.19.0 on 2026-10-04; nodejs26 (beta) had 26.7.0. + description = "Cloud Run functions runtime; it must provide JSPI" + type = string + default = "nodejs24" +} + variable "name" { description = "Function name and prefix of what is created with it" type = string diff --git a/scripts/build.sh b/scripts/build.sh index 494ccde..cca436f 100755 --- a/scripts/build.sh +++ b/scripts/build.sh @@ -13,4 +13,5 @@ fi mkdir -p build "$almide" check src/native.almd "$almide" build src/native.almd -o build/server +# The storage hooks are async on every JS host: the glue suspends through JSPI. "$almide" build src/wasm.almd --target wasm --host js -o build/app.wasm diff --git a/scripts/package-faas.mjs b/scripts/package-faas.mjs index 806c6b6..1fe1459 100644 --- a/scripts/package-faas.mjs +++ b/scripts/package-faas.mjs @@ -16,7 +16,7 @@ const metadata = JSON.stringify({ provider, source: 'almide-cloud-examples' }) + // Check all required inputs before touching a previous generated package. for (const path of [ - 'adapters/node-wasm.mjs', 'adapters/step.mjs', 'adapters/gcs-store.mjs', 'build/app.js', 'build/app.wasm', + 'adapters/node-wasm.mjs', 'adapters/store.mjs', 'adapters/gcs-store.mjs', 'build/app.js', 'build/app.wasm', `providers/${provider}/package.json`, `providers/${provider}/package-lock.json`, ]) await stat(join(root, path)); @@ -37,7 +37,7 @@ try { await mkdir(output, { recursive: true }); await writeFile(join(output, marker), metadata); -for (const path of ['adapters/node-wasm.mjs', 'adapters/step.mjs', 'adapters/gcs-store.mjs', 'build/app.js', 'build/app.wasm', 'licenses', 'LICENSE']) { +for (const path of ['adapters/node-wasm.mjs', 'adapters/store.mjs', 'adapters/gcs-store.mjs', 'build/app.js', 'build/app.wasm', 'licenses', 'LICENSE']) { const dest = join(output, path); await mkdir(resolve(dest, '..'), { recursive: true }); await cp(join(root, path), dest, { recursive: true }); diff --git a/src/wasm.almd b/src/wasm.almd index 642eb93..ffa049e 100644 --- a/src/wasm.almd +++ b/src/wasm.almd @@ -1,9 +1,73 @@ import api +import json +// --- Storage on JS hosts ----------------------------------------------------- +// The host binds these two hooks to its store (Workers KV, Cloud Storage, or +// memory). Its I/O is asynchronous; `returns: promise` marks both hooks, so +// the glue suspends this module through JSPI until each settles and they read +// here as ordinary calls. +// +// store_get answers {"value": } or {"error": }. +// store_put answers "" when the write is made, otherwise the error message. + +@extern(wasm, "js", "store_get", returns: promise) +local fn store_get(key: String) -> String + +@extern(wasm, "js", "store_put", returns: promise) +local fn store_put(key: String, content: String) -> String + +local fn read_key(key: String) -> Result[Value, String] = match json.parse(store_get(key)) { + err(e) => err("store_get: " + e), + ok(reply) => match json.get_path(reply, json.field(json.root(), "value")) { + some(stored) => ok(stored), + none => err(json.get_string(reply, "error") ?? "store_get: no value"), + }, +} + +local fn read_all(keys: List[String], acc: List[(String, Value)]) -> Result[List[(String, Value)], String] = + match list.first(keys) { + none => ok(acc), + some(key) => match read_key(key) { + err(e) => err(e), + ok(stored) => read_all(list.drop(keys, 1), list.flatten([acc, [(key, stored)]])), + }, + } + +local fn unavailable() -> String = "{\"status\":503,\"body\":{\"error\":\"storage_unavailable\"}}" + +// Runs api.step to a final envelope as the native host does: reads what it asks +// for (at most twice), makes its write, and returns the envelope without it. +local fn run(method: String, target: String, body: String, reads: List[(String, Value)], round: Int) -> String = { + let reply = api.step(method, target, body, json.stringify(value.object(reads))) + match json.parse(reply) { + err(_) => reply, + ok(envelope) => match json.get_array(envelope, "read") { + some(keys) => if round >= 2 then unavailable() else { + let names = list.filter_map(keys, (k) => match value.as_string(k) { ok(s) => some(s), err(_) => none }) + match read_all(names, reads) { + ok(more) => run(method, target, body, more, round + 1), + err(e) => { eprintln("storage read failed: ${e}"); unavailable() }, + } + }, + none => match json.get_path(envelope, json.field(json.root(), "write")) { + none => reply, + some(w) => { + let failure = store_put(json.get_string(w, "key") ?? "", json.get_string(w, "value") ?? "") + if failure == "" then json.stringify(json.remove_path(envelope, json.field(json.root(), "write"))) + else { eprintln("storage write failed: ${failure}"); unavailable() } + }, + }, + }, + } +} + +// Pure routes: never touches storage, so it stays a synchronous export. pub fn handle(method: String, target: String, body: String) -> String = api.handle(method, target, body) -pub fn step(method: String, target: String, body: String, reads: String) -> String = - api.step(method, target, body, reads) +// Every route, storage included. It reaches the async hooks, so the generated +// JS exports it as a Promise. +pub fn serve(method: String, target: String, body: String) -> String = + run(method, target, body, [], 0) fn main() -> Unit = {} diff --git a/tests/contract.test.mjs b/tests/contract.test.mjs index 36a7c6b..1f10e27 100644 --- a/tests/contract.test.mjs +++ b/tests/contract.test.mjs @@ -3,14 +3,18 @@ import { test } from 'node:test'; import { readFile, mkdtemp, rm } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; -import { init, handle } from '../build/app.js'; +import { init, handle, serve } from '../build/app.js'; +import { storeHost, memoryStore } from '../adapters/store.mjs'; import { cases } from './cases.mjs'; import { freePort, startServer, verifyHttp } from './http-harness.mjs'; import { verifyNotes, httpCall } from './notes.mjs'; +const host = storeHost(); +const call = (store, method, path, body = '') => host.serveWith(serve, store, method, path, body); + test('generated Wasm JS contract (compiled-module entry)', async t => { const module = await WebAssembly.compile(await readFile(new URL('../build/app.wasm', import.meta.url))); - await init(module); + await init(module, host.hooks); for (const c of cases) { await t.test(c.name, () => { assert.deepEqual(JSON.parse(handle(c.method, c.path, c.body ?? '')), { status: c.status, body: c.json, ...(c.allow ? { allow: c.allow } : {}) }); @@ -32,25 +36,23 @@ test('native HTTP contract', async t => { await verifyHttp(t, server.base); }); -test('generated Wasm notes: the host performs the reads and writes step asks for', async t => { - const { step } = await import('../build/app.js'); - const { runStep, memoryStore } = await import('../adapters/step.mjs'); +test('generated Wasm notes: Almide reads and writes the store through async hooks', async t => { const store = memoryStore(); await verifyNotes(t, async (method, path, body) => { - const r = await runStep(step, method, path, body, store); + const r = await call(store, method, path, body); return { status: r.status, allow: r.allow ?? null, json: r.body }; }); await t.test('stored document is the JSON list of notes', () => { assert.deepEqual(JSON.parse(store.data.get('notes')), [{ n: 1, text: 'hello' }, { n: 2, text: '世界 🌏' }]); }); await t.test('no store answers 503, and a corrupt document 500', async () => { - assert.deepEqual(await runStep(step, 'GET', '/notes', '', null), { status: 503, body: { error: 'storage_unavailable' } }); - const corrupt = await runStep(step, 'GET', '/notes', '', memoryStore({ notes: 'not json' })); + assert.deepEqual(await call(null, 'GET', '/notes'), { status: 503, body: { error: 'storage_unavailable' } }); + const corrupt = await call(memoryStore({ notes: 'not json' }), 'GET', '/notes'); assert.deepEqual(corrupt, { status: 500, body: { error: 'store_corrupt' } }); }); await t.test('only the newest 50 notes are kept', async () => { const many = memoryStore(); - for (let i = 1; i <= 55; i++) await runStep(step, 'POST', '/notes', JSON.stringify({ text: `n${i}` }), many); + for (let i = 1; i <= 55; i++) await call(many, 'POST', '/notes', JSON.stringify({ text: `n${i}` })); const kept = JSON.parse(many.data.get('notes')); assert.equal(kept.length, 50); assert.deepEqual([kept[0], kept.at(-1)], [{ n: 6, text: 'n6' }, { n: 55, text: 'n55' }]); diff --git a/tests/faas.test.mjs b/tests/faas.test.mjs index e28a2ea..177895e 100644 --- a/tests/faas.test.mjs +++ b/tests/faas.test.mjs @@ -115,7 +115,7 @@ test('warm and overlapping invocations remain isolated', async () => { test('Node function host runs the notes scenario against an injected store', async t => { const { callApi } = await import('../adapters/node-wasm.mjs'); - const { memoryStore } = await import('../adapters/step.mjs'); + const { memoryStore } = await import('../adapters/store.mjs'); const store = memoryStore(); const { verifyNotes } = await import('./notes.mjs'); await verifyNotes(t, async (method, path, body) => { diff --git a/tests/package-faas.test.mjs b/tests/package-faas.test.mjs index 1f5717b..89cb3d4 100644 --- a/tests/package-faas.test.mjs +++ b/tests/package-faas.test.mjs @@ -10,7 +10,7 @@ async function fixture(t) { t.after(() => rm(root, { recursive: true, force: true })); const provider = 'aws-lambda'; const files = { - 'adapters/node-wasm.mjs': '// adapter\n', 'adapters/step.mjs': '// step\n', + 'adapters/node-wasm.mjs': '// adapter\n', 'adapters/store.mjs': '// store\n', 'adapters/gcs-store.mjs': '// gcs\n', 'build/app.js': '// generated\n', 'build/app.wasm': 'fixture bytes', 'LICENSE': 'fixture license', 'licenses/Almide-MIT.txt': 'third-party notice',