Skip to content

refactor(stdlib): drop Option/Unit workarounds in lookups, LRU and command hosts - #39

Merged
fffonion merged 16 commits into
masterfrom
feat/option-result-core-api-cleanup
Nov 15, 2026
Merged

fffonion merged 16 commits into
masterfrom
feat/option-result-core-api-cleanup

Conversation

@fffonion

Copy link
Copy Markdown
Collaborator

Summary

First Core scope of the Option/Result downstream follow-up: remove the historical workarounds that stood in for Option/Unit, keeping the accepted language implementation unchanged.

  • W02 — public lookups no longer use -1: stdlib::rss::collections::index_of and stdlib::rss::bytes::index_of now return Option<int> (Some(index) / None); contains stays bool and matches the Option. Internal loop counters and private parser sentinels are untouched.
  • W01 — LRU no longer duplicates presence: peek returns Option<V>; get keeps cache plus Option<V>; oldest/newest/entry_report return Option<LruEntryRow>. LruPeekResult and LruEntryReport are deleted; list order, capacity and eviction logic are unchanged.
  • W13 — command-style hosts report logical Unit: runtime::sleep, aot::compile, aot::clear, timer::at and timer::every return Unit instead of a fixed success bool. Real queries (aot::is_compiled, exec_count, timer counts, callback premature) keep their types.
  • pd-vm-wasm sleep completion now returns the Unit carrier (Null). The wasm binding is name-based and installs no import schema, so the old Bool(true) leaked silently into the guest stack; new tests assert the carrier value, not just error.is_none().

Breaking changes for downstream consumers

  • index_of(...) == -1 must become a match on Option.
  • LRU callers must stop reading .found; peek/entry_report are Options.
  • Statements that assigned the old success bool from runtime::sleep / aot::* / timer::at|every must drop the value.

Follow-up scopes in the tracking plan migrate spec, playground, website and every consumer repository.

Verification

Executed on 1989e0a6 (inner exit codes):

  • cargo test --locked -p pd-vm-wasm --features runtime — 64 passed
  • cargo test --locked --test builtins_tests — 35 passed (incl. stdlib_{collections,bytes,lrucache}_spec_passes)
  • cargo test --locked --lib builtins::runtime — 73 passed
  • cargo test --locked --test timer_host_tests — 40 passed
  • cargo test --locked --test timer_host_arch_tests — 8 passed
  • cargo test --locked --test standard_host_descriptor_arch_tests — 15 passed (also with sqlite, http-client, http-client,sqlite)
  • cargo test --locked --test compiler_tests — 363 passed
  • cargo test --locked --test vm_tests — 167 passed
  • cargo fmt --all -- --check, cargo clippy --all-targets --features runtime -- -D warnings — clean

Red evidence for the wasm fix was captured by reverting the completion value: two new cases failed with One(Bool(true)) vs One(Null) and ["true"] vs ["null"].

Independent review: one severe blocker (wasm sleep return) found and closed; scoped re-review accepted with non-blocking notes.

Deliberately out of scope

  • pd-vm-wasm/examples/rss-lrucache-example.rss keeps its self-contained found shell until the canonical example scope.
  • IO Result contract and typed json::decode Result follow in later Core scopes.

Do not merge: this PR is part of a staged delivery and downstream pins are updated after acceptance.

runtime::sleep, aot::compile, aot::clear and timer::at/every no longer
return a fixed success boolean. A successful runtime::sleep and AOT
command is the logical Unit result (VmResult<()>), and the owned timer
registration returns the Unit carrier instead of `true`.

The timer catalog declares HostTypeSchema::Unit for both registration
commands, so the timer and composed standard catalog identities are
re-pinned; aot::is_compiled, aot::exec_count, timer counts and the
callback `premature` flag keep their real boolean/integer results.

Guest call sites drop the unused confirmation value: the complex example
sleeps as a statement, the type-inference case compares the propagated
Unit result, and the JIT binding tests bind the wait to a typed Unit
local. The timer architecture guard still rejects nullable routes and
now checks the registration Unit result explicitly.
The browser async bridge completed `runtime::sleep` with `Value::Bool(true)`,
so the playground left a success `true` in the guest value slot while Core
declares the logical Unit result. Wasm binds hosts by name and installs no
import schema, so the Core return check never ran there and the divergence
stayed silent.

- Complete the deadline poll with the Unit carrier (`Value::Null`), matching
  `timer::at` / `timer::every` and the other command hosts. No other wasm
  command host still returns an old success confirmation.
- Cover the bridge return and the guest-visible value slot, and pin the Unit
  slot in the embedded-stdlib + named-runtime-import case. Before this fix the
  new assertions report `left: One(Bool(true))` / `left: ["true"]`.
- Assert the `Option::Some` payload directly in the collections-iter example,
  the bytes spec and the lrucache spec instead of threading a `-1` sentinel
  that a missing payload would satisfy.
- Note `runtime::sleep` and `aot::compile` / `aot::clear` in the standard
  catalog fingerprint comment, and assert the Unit carrier on timer
  registration success.
A namespaced builtin's guest return type was inferred from the coarse
`ValueType` of its Rust signature, so a `Result<T, E>` contract stayed
invisible to `?` and `match`: `io::open(...)` typed as `int` even though
its catalog contract returns `resource<io.file>`.

The runtime dispatch now exposes `guest_return_schema`, the
`HostTypeSchema` the builtin's published descriptor declares — a declared
contract when the function has one, otherwise the signature-derived
schema. It is emitted into the dispatch unit because that unit owns the
private backend modules the descriptors are declared in. Only `Result`
carriers are surfaced, so every other builtin keeps the existing
`ValueType` inference and no guest type moves until a builtin actually
returns a `GuestResult`.
The IO guest surface could not distinguish "empty" from "failed": an EOF
read_line returned "", every flush/close returned a fixed `true`, and an
open on a missing file aborted the VM.

Every fallible IO call now declares `Result<T, string>`:
`io.open`/`io.popen` -> `Result<resource<io.file>, string>`,
`io.read_all` -> `Result<string, string>`,
`io.read_line` -> `Result<Option<string>, string>`,
`io.write` -> `Result<int, string>` (bytes written),
`io.flush`/`io.close` -> `Result<(), string>`, and `io.exists` keeps its
Boolean. `read_line` drops the single trailing newline, so EOF is
`Ok(None)`, a real empty line is `Ok(Some(""))`, and an empty file reads
as `Ok("")`.

Only ordinary file-system outcomes take the `Err` branch. Invalid or
closed handles, resource borrow/ownership/scope failures, policy and
capability denials, read/write limits, unsupported modes and wasm
rejections stay fatal `VmError`s. A failed close still retires its scope
resource: the blocking backend falls back to the drop path (which signals
the process tree), and the async backend detaches the shared handle
before retirement.

The three backends share the contracts declared in `io/mod.rs`, including
new contracts for popen/read_line/write/flush, so the descriptor schema
cannot drift per target. The async adapter returns its carriers through
`HostFutureOutput<Value>` plus `publish_owned`, because an owned resource
inside a `GuestResult` cannot cross a future publication.

`stdlib/rss/io.rss` propagates with `?` and no longer swallows close or
flush failures; the stdlib, blocking, async and descriptor-install suites
migrate to the new contract and pin the recoverable outcomes by name.
EOF and empty-file assertions, the write byte count, and the Unit
close/flush results replace the former sentinel checks.
The five `oldest`/`newest`/`peek` checks compared against an artificial
`-1` fallback, so a wrong `Option` branch would have been indistinguishable
from a real value of -1. Each lookup now asserts the expected payload in
its `Some` arm and fails in `None`, with no intermediate sentinel.
`io::close` on the async backend routed every `close_shared_io_handle_with`
error through the guest `Err` branch after detaching the handle and retiring
the scope. That made an already-closed slot (no handle left to flush, signal
or retire) look like a recoverable flush failure and pushed the scope through
a second retirement.

Classify the failure where it is produced instead: `CloseFailure::RuntimeRequired`
and an empty slot stay fatal `VmError`s before any mutation, and only
`CloseFailure::Cleanup` detaches the handle so `Drop` still signals the process
tree, the scope still retires, and the guest sees the message. This matches the
blocking backend's order of "fatal empty slot first, then cleanup outcome".

Also migrate the remaining call sites to the Result contract this branch
introduced: `tests/pr24_resource_tests.rs` (open/close ownership flows and the
`io::open` / `io::close` schema assertions) and both guest scripts in
`tests/vm/io_http_coexistence_tests.rs`. Document that `io::read_line` strips
only the trailing `\n` (so `\r\n` leaves the `\r`) and that the async
`publish_owned` validation covers the `Ok` path only, and reword the stale
"close stays Boolean" assertion message.

New coverage: a closed slot stays fatal and never reaches the cleanup closure,
a cleanup failure detaches the handle and cannot be retried, a missing file is a
guest error on the async backend, and a retried guest close is rejected.
The previous commit fixed `builtin_io_close` and covered the helper it calls,
so a revert of the call site kept the suite green. Add the missing caller-level
coverage and correct two claims the review found overstated.

`builtin_close_keeps_an_empty_slot_resource_registered_and_fatal` installs a
registered `io.file` resource whose internal handle is already gone and drives
`builtin_io_close`'s own body. It asserts the fatal `io handle is closed`
`VmError` and that nothing was retired: the resource count is unchanged, the
token still resolves to the same live resource (so its `Drop` never ran), and
the scope is not closing. Neither new test passes against the pre-fix caller.

`runtime_required_close_retains_the_process_handle` covers the branch the
review found untested: a popen close polled outside any Tokio runtime fails with
`RuntimeRequired` and still leaves the handle owned, so a retry under a live
runtime can close and reap the retained child.

The empty-slot state is not reachable from guest code once the fix is in place
(only the old double-retirement produced it), which is why the test installs it
directly rather than driving a script.

Accuracy fixes: the retried-close test only covers the stale-capture path in
`io_handle_for_resource` (`resource_stale`), not the cleanup-failure path; the
descriptor test's assertion message no longer claims the stack-top `Bool(true)`
is close's `Unit` branch; and the docs no longer say the guest `Err` branches
skip validation entirely — they skip `publish_owned` only and still pass
`validate_host_call_return`.
Descriptors could not express a stack adapter whose pending operation belongs
to the generic VM operation/stream registries, so a downstream composed against
this revision had to drop either the guest unions or the descriptor adapters.
The classes were already present for the other dispatch families; only the
runtime-owned pair was missing:

* `HostBindingKind::StaticStackRuntimeOwned`, documented as the stack class
  whose pending operation is resolved from the generic registries rather than
  from a registered per-function operation driver.
* `HostAdapterDescriptor::StaticStackRuntimeOwned(StaticHostStackFunction)`, so
  the adapter descriptor stays `Copy` and `'static` like its siblings.
* `HostFunctionRegistry::mark_exact_runtime_owned_pending`, the registration
  hook the install path calls after the import registers.

`install_descriptor_adapter` registers the adapter exactly like `StaticStack`
and marks the import runtime-owned afterwards; the hook fails closed with
`cannot mark unregistered host import '<name>' as runtime-owned` and records no
per-import state, because submitted operations are already keyed by their
reserved host-operation id. No parallel op-driver table is revived, and the
binding/adapter mismatch diagnostic is unchanged.

Tests cover the install through a module, an explicit descriptor list and a
caller catalog snapshot, the unregistered-name rejection, the plain `StaticStack`
regression, and both mismatch directions.
…ontracts

The `io` module now reports its outcomes as `Result` instead of sentinel
values, which changes its guest schema and therefore its catalog identity. The
arch guard still carried the pre-`Result` pins, so `io` and every composed
standard catalog failed byte-for-byte comparison on a clean tree. `io` is
composed into every build, so all four composed identities move with it.

Re-pinned from the values the guard itself reports for each feature set:
`io` `c8877c93e38dd831`; standard base `8bf67281072b7f1d`, sqlite
`34b88854300802c8`, http `48b693a20f1b92d0`, http+sqlite `2cd4d66a50e0cc47`.
No schema, adapter, or IO behavior is touched.
The runtime side already understood `StaticStackRuntimeOwned`, but the attribute
macro could not emit it: `#[pd_host_function]` accepted `name` and `contract`
only, so a function whose pending operation belongs to the generic VM
operation/stream registries had no declarative spelling. No downstream in this
tree uses it yet, but the gap made the macro line strictly weaker than the
descriptor line it is supposed to mirror, and the next composed module would
have hit a parse error instead of a descriptor.

The attribute now accepts a `runtime_owned_pending` path flag end to end:

* `parse_function_args` returns it alongside `name`/`contract`, and the
  unsupported-argument diagnostic names it.
* The flag is threaded through `generate_host_function_descriptor` into
  `generated_adapter_tokens`, which both the inferred and the contract-declared
  paths now share instead of duplicating the binding match.
* It is only accepted on a stack-dispatch signature. Any other binding class
  fails closed with the descriptor-line wording, character for character:
  `runtime_owned_pending requires a stack-dispatch signature (`&mut Vm` or a
  resource parameter)`.
* On a stack signature it swaps the binding class to
  `HostBindingKind::StaticStackRuntimeOwned` and the adapter to
  `HostAdapterDescriptor::StaticStackRuntimeOwned(<adapter>)`. The adapter
  function itself stays the plain stack adapter, so install and dispatch are
  unchanged; only the marker records that pending resolution is generic.

The one deliberate difference from the descriptor line: the flag also applies to
the inferred (no `contract`) path, where it was silently ignored. Implementing it
once in the shared helper makes both paths behave the same and turns a silent
no-op into either a working binding or the exact arity error.

Tests: `runtime_owned_pending_flag_selects_the_pending_owning_stack_adapter`,
`runtime_owned_pending_flag_requires_a_stack_dispatch_signature` (exact message),
and `stack_signature_without_the_flag_still_generates_a_plain_stack_adapter` as
the regression that a non-opted-in function keeps `StaticStack`. The runtime-owned
integration case also grows the missing restricted-registry assertion: a catalogue
that does not declare the import still rejects the module with
`missing catalog member 'demo::await_signal'` and leaves the registry generation
untouched.
Copy syntaxes/rustscript.tmLanguage.json byte-for-byte from the spec
repository (master c2161b31) so the Core extension highlights the same
lexical surface: unit literal, Option/Result constructors, propagation
operator and optional access operator, with the removed null literal.
…lt contracts

The LRU example now returns Option<V> from peek, drops the found flag from
get, and returns Option<LruEntryRow<K, V>> from oldest/newest instead of the
removed LruPeekResult/LruEntryReport wrappers. The complex example documents
runtime::sleep as a Unit statement. A new self-contained Option/Result
example prints one named line per contract check, and
tests/option_result_downstream_examples.rs compiles and runs it, the LRU and
the complex examples through the hidden stdlib modules, asserting the named
output rather than only the exit status.
The dispatch flag is now exercised without a declared contract: a
&mut Vm signature still selects StaticStackRuntimeOwned while keeping the
inferred schema, an argument-only signature is rejected, and a trybuild
case pins the stack-dispatch diagnostic.
encode_program only needs bytecode and schema types, but both vmbc and its
re-exports were gated behind `runtime`, forcing compiler-only consumers such as
IronRust to enable the whole VM. Publish vmbc unconditionally and import the
program types from `bytecode`; the optional-schema validator it uses moves out
from behind the same gate.
@fffonion
fffonion force-pushed the feat/option-result-core-api-cleanup branch from 905e364 to 8db4583 Compare November 15, 2026 12:45
@fffonion
fffonion merged commit baa7b58 into master Nov 15, 2026
8 checks passed
@fffonion
fffonion deleted the feat/option-result-core-api-cleanup branch November 15, 2026 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant