Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ ngx-rust-rev = "cda9d8372ccd2e139ef2b03e5806e019e7796187"
nginx-version = "1.30.4"
nginx-tarball-sha256 = "4261dc90e9e47c1c4041276e9aaa3d48ebe2e664f728e14fa95ae6c67d57a08b"
rustscript-git = "https://github.com/rustscript-lang/rustscript"
rustscript-rev = "f5f71ebc2f38e276c35f2aebaf63f4e2780efe99"
rustscript-rev = "b1d6cffede77f49410bf63525f30b9a46b02dc01"

[lib]
crate-type = ["cdylib", "rlib"]
Expand All @@ -27,7 +27,7 @@ sha1 = { version = "0.10", default-features = false }
sha2 = { version = "0.10", default-features = false }
nginx-sys = { git = "https://github.com/nginx/ngx-rust", rev = "cda9d8372ccd2e139ef2b03e5806e019e7796187" }
ngx = { git = "https://github.com/nginx/ngx-rust", rev = "cda9d8372ccd2e139ef2b03e5806e019e7796187", default-features = false, features = ["std"] }
pd-host-function = { git = "https://github.com/rustscript-lang/rustscript", rev = "f5f71ebc2f38e276c35f2aebaf63f4e2780efe99" }
pd-host-function = { git = "https://github.com/rustscript-lang/rustscript", rev = "b1d6cffede77f49410bf63525f30b9a46b02dc01" }
tokio = { version = "1", default-features = false, features = ["rt", "rt-multi-thread", "time", "net", "sync", "io-util"] }
rustls = { version = "0.23", default-features = false, features = ["std", "tls12", "ring"] }
tokio-rustls = { version = "0.26", default-features = false, features = ["ring"] }
Expand All @@ -36,7 +36,7 @@ rustls-pemfile = { version = "2", default-features = false, features = ["std"] }
rustls-native-certs = "0.8"
webpki-roots = "0.26"
zeroize = { version = "1", default-features = false, features = ["alloc"] }
vm = { package = "pd-vm", git = "https://github.com/rustscript-lang/rustscript", rev = "f5f71ebc2f38e276c35f2aebaf63f4e2780efe99", default-features = false, features = ["runtime", "async", "http-client", "sqlite"] }
vm = { package = "pd-vm", git = "https://github.com/rustscript-lang/rustscript", rev = "b1d6cffede77f49410bf63525f30b9a46b02dc01", default-features = false, features = ["runtime", "async", "http-client", "sqlite"] }

[features]
default = ["export-modules"]
Expand Down
22 changes: 11 additions & 11 deletions docs/api/shared-dict.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@ bytes. Duplicate names (same or different size) are rejected. The zone
is NGINX shared memory with a slab-backed rbtree/LRU store, the
`init(shm_zone, old_data)` reload contract, and a cross-worker mutex.
`ngx::shared::dict(name)` returns a typed handle; later methods take
that handle. Multi-return OpenResty values use fixed result structs at the
RustScript boundary. The current host catalog exposes these return slots as
`unknown`, so callers declare the matching struct and annotate the assignment;
field access then uses `.value` / `.flags` / `.stale` / `.ok` / `.error` /
`.forcible`. The VM's internal carrier is not part of the script-facing type.
that handle. Multi-return OpenResty values use host-catalog named structs
(`SharedGetResult`, `SharedStoreResult`, `SharedSafeStoreResult`,
`SharedIncrResult`, `SharedNumberResult`). Do not redeclare them in
guest scripts. `SharedGetResult.value` is `unknown?` because stored
values are heterogeneous; numeric results use `number?`.
Truly dynamic-key collections remain `map` fields inside typed envelopes, such
as `DecodedArgsResult.args` and `RequestHeadersResult.headers`.
Optional `exptime` and `flags` are explicit `null`. Values are bool, int,
Expand All @@ -22,8 +22,8 @@ null.
The shared-dict result shapes are:

```rss
struct SharedGetResult<T> {
value: T?,
struct SharedGetResult {
value: unknown?,
flags: int,
stale: bool,
error: string?
Expand Down Expand Up @@ -69,8 +69,8 @@ request-owned VMs.
| OpenResty API | RustScript callable | Status | Phases | Limits | Semantic differences | Acceptance |
| --- | --- | --- | --- | --- | --- | --- |
| `ngx.shared.DICT` | `ngx::shared::dict(name: string) -> resource` | adapted | `PHASE_DICT_LOOKUP` | unknown zone is a host error | Typed `ngx.shared.dict` handle, not `ngx.shared.NAME` field | `tests/ngx_shared_dict.rs` |
| `ngx.shared.DICT.get` | `ngx::shared::get(dict, key) -> SharedGetResult<T>` | adapted | `PHASE_DICT_GET` | missing → `{value: null}` | Fields `value`, `flags`, `stale`, `error` | `tests/ngx_shared_dict.rs` |
| `ngx.shared.DICT.get_stale` | `ngx::shared::get_stale(dict, key) -> SharedGetResult<T>` | adapted | `PHASE_DICT_GET` | stale expired values | `stale` is bool | `tests/ngx_shared_dict.rs` |
| `ngx.shared.DICT.get` | `ngx::shared::get(dict, key) -> SharedGetResult` | adapted | `PHASE_DICT_GET` | missing → `{value: null}` | Fields `value`, `flags`, `stale`, `error` | `tests/ngx_shared_dict.rs` |
| `ngx.shared.DICT.get_stale` | `ngx::shared::get_stale(dict, key) -> SharedGetResult` | adapted | `PHASE_DICT_GET` | stale expired values | `stale` is bool | `tests/ngx_shared_dict.rs` |
| `ngx.shared.DICT.set` | `ngx::shared::set(dict, key, value, exptime, flags) -> SharedStoreResult` | adapted | `PHASE_DICT_WRITE` | LRU up to 30 | Fields `ok`, `error`, `forcible` | `tests/ngx_shared_dict.rs` |
| `ngx.shared.DICT.safe_set` | `ngx::shared::safe_set(dict, key, value, exptime, flags) -> SharedSafeStoreResult` | adapted | `PHASE_DICT_WRITE` | no eviction | Fields `ok`, `error` | `tests/ngx_shared_dict.rs` |
| `ngx.shared.DICT.add` | `ngx::shared::add(dict, key, value, exptime, flags) -> SharedStoreResult` | adapted | `PHASE_DICT_WRITE` | exists → error | Fields `ok`, `error`, `forcible` | `tests/ngx_shared_dict.rs` |
Expand All @@ -84,8 +84,8 @@ request-owned VMs.
| `ngx.shared.DICT.free_space` | `ngx::shared::free_space(dict) -> int` | adapted | `PHASE_DICT_WRITE` | approximate | Process-local: capacity − used; shm: `pfree * pagesize` | `tests/ngx_shared_dict.rs` |
| `ngx.shared.DICT.lpush` | `ngx::shared::lpush(dict, key, value) -> SharedNumberResult` | adapted | `PHASE_DICT_WRITE` | list len ≤ 65536; no LRU eviction | Fields `value`, `error`; bool/null/map → `bad value type`; scalar conflict → `value not a list` without mutating the node | `tests/ngx_shared_dict_list.rs` |
| `ngx.shared.DICT.rpush` | `ngx::shared::rpush(dict, key, value) -> SharedNumberResult` | adapted | `PHASE_DICT_WRITE` | same | Same result type as lpush; appends | `tests/ngx_shared_dict_list.rs` |
| `ngx.shared.DICT.lpop` | `ngx::shared::lpop(dict, key) -> SharedGetResult<T>` | adapted | `PHASE_DICT_WRITE` | decode before unlink | Fields `value`, `flags`, `stale`, `error`; empty/missing → `{value: null}`; decode failure → `{error: "bad value type"}` and the item stays queued | `tests/ngx_shared_dict_list.rs` |
| `ngx.shared.DICT.rpop` | `ngx::shared::rpop(dict, key) -> SharedGetResult<T>` | adapted | `PHASE_DICT_WRITE` | same | Same result type as lpop; pops tail | `tests/ngx_shared_dict_list.rs` |
| `ngx.shared.DICT.lpop` | `ngx::shared::lpop(dict, key) -> SharedGetResult` | adapted | `PHASE_DICT_WRITE` | decode before unlink | Fields `value`, `flags`, `stale`, `error`; empty/missing → `{value: null}`; decode failure → `{error: "bad value type"}` and the item stays queued | `tests/ngx_shared_dict_list.rs` |
| `ngx.shared.DICT.rpop` | `ngx::shared::rpop(dict, key) -> SharedGetResult` | adapted | `PHASE_DICT_WRITE` | same | Same result type as lpop; pops tail | `tests/ngx_shared_dict_list.rs` |
| `ngx.shared.DICT.llen` | `ngx::shared::llen(dict, key) -> SharedNumberResult` | adapted | `PHASE_DICT_WRITE` | — | Missing/expired → `{value: 0}`; scalar → `value not a list` | `tests/ngx_shared_dict_list.rs` |
| `ngx.shared.DICT.flush_all` | `ngx::shared::flush_all(dict) -> null` | adapted | `PHASE_DICT_WRITE` | LRU; ≤1024 visits/call | Each call first frees already-expired keys (visit-capped), then marks live LRU keys with `expires=1` (visit-capped), then `expire(0)`. Repeated calls make monotonic progress across >1024 keys; one call does not wipe the zone. | `tests/ngx_shared_dict_list.rs` |
| `ngx.shared.DICT.flush_expired` | `ngx::shared::flush_expired(dict, max) -> int` | adapted | `PHASE_DICT_WRITE` | `max` 0/null still ≤ 1024 results; visits ≤ 1024 | Freed count. Walk stops at the result cap *or* the hard visit cap, whichever comes first. OpenResty `0` is unlimited. | `tests/ngx_shared_dict_list.rs` |
Expand Down
79 changes: 79 additions & 0 deletions docs/host-descriptors.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# ngx host descriptors

ngx-rustscript host functions are owned by explicit
`HostModuleDescriptor` values returned from `ngx_host_modules()`. Each
production function has one descriptor source for schema, binding,
adapter, guest resource effects, and private state.

## Authoring

1. Implement the host with `#[pd_host_function(name = "...", contract = crate::host::catalog::<name>_contract)]`.
2. Keep the guest schema in `src/host/catalog.rs` as a zero-argument
`*_contract()` wrapper around `host_function_contract`.
3. List the generated `*_descriptor` factory on the family module slice
(`codec_host_module`, `timer_host_module`, …).
4. Install through `HostModuleDescriptor::install_from_catalog`. Do not
add parallel `HostApiBuilder::function` / `register_exact_*` loops.

Timer `at`/`every` use `HostAdapterDescriptor::Owned`. Request, pool, and
timer VMs install through `ngx_host_registry()` (restricted transactional
exact catalog). `bind_ngx_hosts` is a thin wrapper around that registry
and is not a second name/arity binding authority.

## Typed resources

Handle types implement `HostResourceType` plus `HostResource`:

| Type | Key |
| --- | --- |
| `SharedDictHandle` | `ngx.shared.dict` |
| `SemaphoreHandle` | `ngx.semaphore` |
| `TcpCosocket` | `ngx.socket.tcp` |
| `UdpCosocket` | `ngx.socket.udp` |
| `TrustedStore` | `ngx.socket.tls.trustedstore` |
| `ClientCert` | `ngx.socket.tls.clientcert` |

Guest resource effects follow descriptor passing modes
(borrow / borrow-mut / take / create). Request-local `ngx::ctx` is not a
VM resource: payloads stay request-owned maps.

Sync methods that do not need `&mut Vm` take `ResourceRef` (shared-dict
ops, `ngx::semaphore::{post,count}`). Constructors still return a raw
guest token because `push_resource` requires `&mut Vm`. Async methods and
`close` keep a raw guest token: `ResourceRef` cannot cross suspension,
and close cannot combine `&mut Vm` with a borrowed resource. Those
slots still advertise `Resource` + passing mode on the catalog contract.

## Named structs

Fixed envelopes are catalog `HostTypeSchema::Named` and Rust
`HostNamedStruct` types. Runtime values remain maps.

| Struct | Functions |
| --- | --- |
| `DecodedArgsResult` | `ngx::decode_args` |
| `RequestHeadersResult` | `ngx::req::get_headers` |
| `RequestArgsResult` | `ngx::req::get_uri_args`, `ngx::req::get_post_args` |
| `SharedGetResult` | `ngx::shared::{get,get_stale,lpop,rpop}` |
| `SharedStoreResult` | `ngx::shared::{set,add,replace}` |
| `SharedSafeStoreResult` | `ngx::shared::{safe_set,safe_add,expire}` |
| `SharedIncrResult` | `ngx::shared::incr` |
| `SharedNumberResult` | `ngx::shared::{ttl,lpush,rpush,llen}` |

Guest scripts use the host named types. Inner `value` fields on shared
results stay `Unknown` because stored dictionary values are heterogeneous.

## Intentional dynamic slots

Positions that stay `HostTypeSchema::Unknown` or `Map(Unknown)` are
listed exactly in `tests/host_catalog.rs`
(`intentional_dynamic_slots_match_the_allowlist`). Typical reasons:

- optional trailing OpenResty arguments (`null` or several runtime shapes)
- open dictionaries (`ngx::encode_args` `args`, header-set maps, ctx payloads)
- heterogeneous stored shared-dict values and `ngx::header::get` (string,
string array, or null)

Do not add a new `Unknown` or `Map(Unknown)` return for a fixed outer
shape. Add a named struct, bump `NGX_HOST_CATALOG_SCHEMA_VERSION`, and
update the fingerprint goldens.
1 change: 1 addition & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ metadata on each directive. The prose is ngx-rustscript's own.
| [Security and policy](security.md) | Denied-by-default grants, one HTTP allow pair, SQLite root, fail-closed log |
| [Examples](examples/index.md) | Rate limiter, HMAC API-key auth, async pipeline |
| [Compatibility](ngx-api-compatibility.md) | OpenResty v0.10.29 mapping, typed results, out of scope |
| [Host descriptors](host-descriptors.md) | Module descriptors, typed resources, named envelopes, retained Unknown slots |
| [Scheduler design](design/ngx-scheduler.md) | Callback execution contract: blocked thread/`on_abort`/`run_worker_thread` families, plus the landed semaphore and timer families and their lifecycle |

## Runnable phases
Expand Down
4 changes: 2 additions & 2 deletions docs/ngx-api-compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,8 +70,8 @@ struct RequestArgsResult {
truncated: bool
}

struct SharedGetResult<T> {
value: T?,
struct SharedGetResult {
value: unknown?,
flags: int,
stale: bool,
error: string?
Expand Down
5 changes: 0 additions & 5 deletions examples/async-pipeline.rss
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,6 @@ use sqlite;
use json;
use bytes as builtin_bytes;

struct SharedStoreResult {
ok: bool,
error: string?
}

let url_file = ngx::var::get("rss_pipeline_url_file").unwrap_or("");
let database_file = ngx::var::get("rss_pipeline_database").unwrap_or("");

Expand Down
11 changes: 0 additions & 11 deletions examples/rate-limit.rss
Original file line number Diff line number Diff line change
@@ -1,16 +1,5 @@
use ngx;

struct SharedSafeStoreResult {
ok: bool,
error: string?
}

struct SharedIncrResult {
value: number?,
error: string?,
forcible: bool
}

let LIMIT = 5;
let WINDOW_SECONDS = 2;
let RETRY_AFTER = "2";
Expand Down
31 changes: 1 addition & 30 deletions src/async_bridge.rs
Original file line number Diff line number Diff line change
Expand Up @@ -131,36 +131,7 @@ mod tests {
}

fn test_catalog() -> vm::HostApiCatalog {
let mut builder = vm::HostApiBuilder::new();
builder.function(vm::HostFunctionSchema::with_return(
"ngx::sleep",
vec![vm::HostParamSchema::value(
"seconds",
vm::HostTypeSchema::Number,
)],
vm::HostTypeSchema::Null,
));
builder.function(vm::HostFunctionSchema::with_return(
"ngx::sleep_ms",
vec![vm::HostParamSchema::value("ms", vm::HostTypeSchema::Int)],
vm::HostTypeSchema::Null,
));
builder.function(vm::HostFunctionSchema::with_return(
"ngx::probe",
vec![vm::HostParamSchema::value("ms", vm::HostTypeSchema::Int)],
vm::HostTypeSchema::Int,
));
builder.function(vm::HostFunctionSchema::with_return(
"ngx::ready",
vec![],
vm::HostTypeSchema::Int,
));
builder.function(vm::HostFunctionSchema::with_return(
"ngx::fail",
vec![],
vm::HostTypeSchema::Int,
));
builder.build().expect("test catalog")
crate::host::test_bind_catalog()
}

fn bind_vm(program: vm::Program, handle: Handle, test_hosts: bool) -> Vm {
Expand Down
Loading
Loading