[lukebuehler]

This PR moves the environment stack to protocol version 4 and simplifies the environment model: Lightspeed keeps the inventory and decides when environments are created and closed, and providers own everything that happens while a machine runs. It is a breaking change. Databases need a reset, and envd, providers and the runtime must be upgraded together (see Deployment).

Protocol and envd

• Versions. Two exact-match version numbers replace the single protocol version: envd (data plane and registration) and controller, both now 4.
• Wire format. All three sockets use one JSON-RPC 2.0 envelope with integer error codes. The error set is closed, each error has a kind (data.kind), and retryability follows from the kind.
• Handshake and decoding. initialize is enforced first on every socket, and decoding is strict.
• Contract. A method manifest drives dispatch, the JSON Schema/OpenRPC contract export (crates/environment-protocol/contract/) and per-method fixtures.
• Registration channel. It is JSON-RPC too:
  • The gateway's first message is the frozen registration/challenge, and the signature binds the gateway origin.
  • registration/openStream replaces dataUrl, so daemons derive the data URL from the gateway they dialed.
  • New registration/goodbye and registration/closed {superseded | environmentClosed}.
  • A heartbeat deadline on both sides.
• Process and job output.
  • Reads use byte offsets, with explicit gaps, tail reads, returnOn, an optional ack, and the daemon-owned cursor kept as the default.
  • There is one exit outcome with named signals, plus terminate with escalation.
  • Input modes are none, bytes, pipe or a PTY.
• Filesystem.
  • One confined resolver under the root, so links are followed only while they stay inside it.
  • Atomic fs/writeFile with ifMatch, and a new fs/rename.
  • One gitignore-style pattern grammar, and fs/scan results reported per root.
  • transfer/* split into separate methods with a failed phase.
• envd hardening. Fixes for the defects found in review:
  • output lost at exit or consumed by a dropped connection;
  • job paging that skipped bytes;
  • a resubmitted job replaying an old run;
  • stdin and PTY writes that could deadlock or freeze the daemon;
  • quadratic job-output rewrites;
  • special files that could OOM envd;
  • search and glob running out of budget inside .git;
  • the state dir living inside the agent's working directory (it now defaults to the XDG state dir);
  • and others.
• Removed. Dead surface is deleted: process notifications, fs/copy, fs/capture, fs/materialize, job artifacts, job/list, process/resize, and the http/stdio/ssh transports.

Environment model

• State. One 5-value state (provisioning, open, closing, closed, failed) with a close_reason replaces the 10-value status. The display status (paused, away, booting, …) is derived. The incarnation table is gone.
• Ownership. Providers own power, idle, wake and ingress. Lightspeed no longer stores a power intent or runs the power reaper, power and ingress convergence, or the provider-facts refresh.
  • A dial to a sleeping target wakes it, and providers answer typed refusals (503 with Retry-After, 410, 409).
  • environments/power/put and environments/ingress/put are synchronous pass-throughs.
  • The idle policy is a template default plus an optional create-time override.
• Reconciler. It retries only two transitions: create until the provider accepts, then one data handshake (with an exact version check) opens the row; and close until the target is gone. Retries back off per row, there is a 30-minute provisioning deadline, and a slow display sweep records provider observations.
• One use gate. Every caller gets Use, Wait or Gone. A disconnected registered environment waits up to 2 minutes for persistent keys and not at all for ephemeral ones.
• Registered environments.
  • Liveness is written only on connect and disconnect, and closes reach daemons in-process.
  • Accepted registration URLs are configurable (LIGHTSPEED_ENVIRONMENT_REGISTRATION_URLS).
  • envd gains --on-closed=exit|park and an opt-in for plain ws://.
  • Keys can be updated and rotated (with a child secrets table), and environments can be updated.
  • Lists hide closed rows by default and page with a cursor.
• Providers. The provider credential is optional on every scheme. Bindings are keyed by provider id. Adopt is removed.
• Incus provider. It runs its own idle loop over env/getIdle, wakes on dial, takes an optional credentialFile, and names resources per universe.
• Per-session environments. The provision attachment was prototyped and dropped again: attachments stay environmentId | inherit.

UI, API and docs

• API. Environment methods go 32 → 33: removed environments/idle-policy/put and deployment/environments/adopt; added environments/update, registration-keys/update and registration-keys/rotate. The API contract and the TypeScript clients are regenerated.
• Environments page.
  • It is rebuilt on the inventory table from Unify setup tables, refine session activity, and adopt IBM Plex #123 (merged from main), showing state, lifecycle messages and provider observations.
  • Power and ingress buttons are pass-throughs.
  • The create dialog has a custom idle policy, there are key Edit/Rotate/Revoke dialogs, and external environments have a Close button.
• Docs.
  • New: reference/environment-protocol.md, the normative spec for provider authors, and environments/registered-daemons.md, with StatefulSet, Deployment and systemd examples.
  • Rewritten: the provider guide.
  • Updated: the environment pages and reference/environment-variables.md.
• Small extras.
  • scripts/dev/env.sh exports LIGHTSPEED_MCP_PRIVATE_NETWORKS (loopback).
  • The default text weight is 450 on standard-density screens, so IBM Plex renders crisper on Windows.
  • The runs_live assertions now hold despite the core blob tools; they were already failing before this branch.

Deployment

• Database reset required. store-pg/migrations/005_environments.sql is rewritten in place, and the schema revision stays 11. A database migrated from main fails the checksum check. Reset it (./dev.sh reset), or purge the environment tables and remigrate.
• Upgrade envd, providers and the runtime together. Both protocol numbers are exact-match, so envd 4, controller 4 and this runtime must match. Rebuild Incus images.
  • Registered daemons re-register, because the default state dir moved and keys are not migrated.
  • Terminate in-flight EnvironmentJobWorkflow runs: their history types changed.
• Configuration.
  • LIGHTSPEED_ENVIRONMENT_PUBLIC_URL is removed.
  • An environment-gateway on a non-loopback bind needs LIGHTSPEED_PUBLIC_BASE_URL, which is the signed registration origin, or LIGHTSPEED_ENVIRONMENT_REGISTRATION_URLS.
• Run exactly one environment-gateway replica. API gateways can still scale.
• Repositories outside this one. The Harbor adapter's receipt reader must follow the typed receipt (incarnationId is gone), and third-party providers must move to controller protocol 4.

Testing

• cargo test --workspace: 2,609 passed, 0 failed.
• The exact CI Clippy command (-D warnings), cargo fmt --check, npm run typecheck, the web tests (960), the web and demo builds, check:docs and verify-metadata.sh are all clean.
• Live suites against the dev stack:
  • store-pg;
  • environment registration and provider;
  • vfs transfer;
  • profiles;
  • code capabilities, tools, worker process and workflow;
  • subagents;
  • bots;
  • sessions;
  • runs_live (9/9, twice).
• Not run: incus_live (it needs a real Incus server) and runs_live_slow.

Known gaps

• Untested against real Incus: the Incus provider's freeze and stop, its idle loop and wake-on-dial.
• Universe deletion: two foreign keys in 004_auth.sql (mcp_servers_auth_grant_fk, auth_providers_credential_secret_fk) may still block deleting a universe. This predates this branch.
• Debug test binaries: in large ones a panic aborts with "failed to initiate panic, error 5" because there is no unwind info. Run single tests with --nocapture to see the message.