diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4888c74ef..652bc3493 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,140 +1,61 @@ -# Contributing Guidelines +# Contributing -### Contents +Trusted Server accepts focused code and documentation contributions. By +submitting a contribution, you agree that it is licensed under the repository's +Apache License 2.0. -- [Submitting Pull Requests](#repeat-submitting-pull-requests) -- [Writing Commit Messages](#memo-writing-commit-messages) -- [Code Review](#white_check_mark-code-review) -- [Coding Style](#nail_care-coding-style) -- [Credits](#pray-credits) +## Before implementation -## :repeat: Submitting Pull Requests +Search the issue tracker before opening a new issue. For a substantial change, +describe the problem, intended behavior, affected adapters, and compatibility +constraints before investing in an implementation. Do not place credentials or +non-public vulnerability details in an issue; contact the maintainers privately +before disclosure. -We **love** pull requests! Before [forking the repo](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) and [creating a pull request](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/proposing-changes-to-your-work-with-pull-requests) for non-trivial changes, it is usually best to first open an issue to discuss the changes, or discuss your intended approach for solving the problem in the comments for an existing issue. +Read [CLAUDE.md](CLAUDE.md) for repository architecture, target constraints, +coding conventions, error handling, documentation rules, and commit policy. +The [project governance document](ProjectGovernance.md) defines decision and +release responsibilities. -For most contributions, after your first pull request is accepted and merged, you will be [invited to the project](https://help.github.com/en/github/setting-up-and-managing-your-github-user-account/inviting-collaborators-to-a-personal-repository) and given **push access**. :tada: +## Make a focused change -_Note: All contributions will be licensed under the project's license._ +- Keep one pull request centered on one coherent outcome. +- Do not refactor or reformat unrelated code. +- Add tests for new behavior and regression tests for defects. +- Keep platform-specific dependencies out of `trusted-server-core`. +- Use `error-stack` reports for production errors; the Spin entry-point FFI is + the sole documented `anyhow` exception. +- Use fictional example data by default. A real public vendor endpoint requires + the exact reviewed exception described in `CLAUDE.md`. +- Update the canonical guide or crate README when a user-visible contract + changes; do not copy volatile matrices into multiple documents. -- **Smaller is better.** Submit **one** pull request per bug fix or feature. A pull request should contain isolated changes pertaining to a single bug fix or feature implementation. **Do not** refactor or reformat code that is unrelated to your change. It is better to **submit many small pull requests** rather than a single large one. Enormous pull requests will take enormous amounts of time to review, or may be rejected altogether. +## Verify -- **Coordinate bigger changes.** For large and non-trivial changes, open an issue to discuss a strategy with the maintainers. Otherwise, you risk doing a lot of work for nothing! +Run the target-matched checks in [AGENTS.md](AGENTS.md#ci-gates), the +canonical command surface for local and CI verification. [TESTING.md](TESTING.md) +covers auction-orchestration testing specifically and repeats the adapter test +aliases relevant to that runbook; it isn't a link index for other runbooks. -- **Prioritize understanding over cleverness.** Write code clearly and concisely. Remember that source code usually gets written once and read often. Ensure the code is clear to the reader. The purpose and logic should be obvious to a reasonably skilled developer, otherwise you should add a comment that explains it. +Keep a pull request in draft while required checks or known changes remain. +Before requesting review, inspect the complete diff, resolve all failures, and +state any platform path that could not be exercised. -- **Follow existing coding style and conventions.** Keep your code consistent with the style, formatting, and conventions in the rest of the code base. When possible, these will be enforced with a linter. Consistency makes it easier to review and modify in the future. +## Commits -- **Include test coverage.** Add unit tests or UI tests when possible. Follow existing patterns for implementing tests. +Write concise, imperative, sentence-case subjects without semantic prefixes or +bracketed tags. Keep the subject at 50 characters when practical and do not end +it with a period. Use a wrapped body when the reason, compatibility effect, or +non-obvious tradeoff is not evident from the diff. -- **Use the repo's default branch.** Branch from and [submit your pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request) to the repo's default branch. Usually this is `main`, but it could be `dev`, `develop`, or `master`. +Examples: -- **[Resolve any merge conflicts](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/resolving-a-merge-conflict-on-github)** that occur. +- `Document Cloudflare startup configuration` +- `Reject ambiguous integration route records` -- **Keep PR in Draft** until ready for review. When it is ready, assign the PR to yourself and request at least one reviewer. +## Review -- **Promptly address any CI failures**. If your pull request fails to build or pass tests, please push another commit to fix it. - -- When writing comments, use properly constructed sentences, including punctuation. - -- Use spaces, not tabs. - -## :memo: Writing Commit Messages - -Please [write a great commit message](https://chris.beams.io/posts/git-commit/). - -1. Separate subject from body with a blank line -1. Limit the subject line to 50 characters -1. Use sentence case (capitalize the first word) -1. Do not end the subject line with a period -1. Use the imperative mood in the subject line (example: "Fix networking issue") -1. Wrap the body at about 72 characters -1. Use the body to explain **why**, _not what and how_ (the code shows that!) -1. Do not use semantic prefixes or tags (examples: `fix:`, `feat:`, `[Docs]`) -1. Keep PR state out of commit messages; use GitHub Draft PRs instead - -``` -Short summary of changes in 50 chars or less - -Add a more detailed explanation here, if necessary. Possibly give -some background about the issue being fixed, etc. The body of the -commit message can be several paragraphs. Further paragraphs come -after blank lines and please do proper word-wrap. - -Wrap it to about 72 characters or so. In some contexts, -the first line is treated as the subject of the commit and the -rest of the text as the body. The blank line separating the summary -from the body is critical (unless you omit the body entirely); -various tools like `log`, `shortlog` and `rebase` can get confused -if you run the two together. - -Explain the problem that this commit is solving. Focus on why you -are making this change as opposed to how or what. The code explains -how or what. Reviewers and your future self can read the patch, -but might not understand why a particular solution was implemented. -Are there side effects or other unintuitive consequences of this -change? Here's the place to explain them. - - - Bullet points are okay, too - - - A hyphen or asterisk should be used for the bullet, preceded - by a single space, with blank lines in between - -Note the fixed or relevant GitHub issues at the end: - -Resolves: #123 -See also: #456, #789 -``` - -## :white_check_mark: Code Review - -- **Review the code, not the author.** Look for and suggest improvements without disparaging or insulting the author. Provide actionable feedback and explain your reasoning. - -- **You are not your code.** When your code is critiqued, questioned, or constructively criticized, remember that you are not your code. Do not take code review personally. - -- **Always do your best.** No one writes bugs on purpose. Do your best, and learn from your mistakes. - -- Kindly note any violations to the guidelines specified in this document. - -- **Use CREG (Code Review Emoji Guide)** to give the reviewee added context and clarity to follow up on code review. - -### Emoji Legend - -| | `:code:` | Meaning | -| :----: | :------------------------------: | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 😃👍💯 | `:smiley:` `:+1:` `:100:` | I like this...

...and I want the author to know it! This is a way to highlight positive parts of a code review. | -| 🔧 | `:wrench:` | I think this needs to be changed.

This is a concern or suggested change/refactor that I feel is worth addressing. | -| ❓ | `:question:` | I have a question.

This should be a fully formed question with sufficient information and context that requires a response. | -| 🤔💭 | `:thinking:` `:thought_balloon:` | Let me think out loud here for a minute.

I might express concern, suggest an alternative solution, or walk through the code in my own words to make sure I understand. | -| 🌱 | `:seedling:` | Planting a seed for future.

An observation or suggestion that is not a change request, but may have larger implications. Generally something to keep in mind for the future. | -| 📝 | `:memo:` | This is an explanatory note, fun fact, or relevant commentary that does not require any action. | -| ⛏ | `:pick:` | This is a nitpick.

This does not require any changes and is often better left unsaid. This may include stylistic, formatting, or organization suggestions and should likely be prevented/enforced by linting if they really matter | -| ♻️ | `:recycle:` | Suggestion for refactoring.

Should include enough context to be actionable and not be considered a nitpick. | -| 🏕 | `:camping:` | Here is an opportunity, not directly related to your changes, for us to leave the campground [code] cleaner than we found it. | -| 📌 | `:pushpin:` | This is a concern that is _out of scope_ and should be staged appropriately for follow up. | - -## :nail_care: Coding Style - -Consistency is the most important. Following the existing Rust style, formatting, and naming conventions of the file you are modifying and of the overall project. Failure to do so will result in a prolonged review process that has to focus on updating the superficial aspects of your code, rather than improving its functionality and performance. - -Style and format will be enforced with a linter when PR is created. - -## :warning: Error Handling - -We use [error-stack](https://docs.rs/error-stack/latest/error_stack/) for error handling to provide rich context and traceability. - -### Guidelines - -1. **Use `Report`**: Public functions should generally return `Result>`. -2. **Context**: Use `.change_context(TrustedServerError::Variant)` to wrap errors and provide semantic meaning. - ```rust - // Good - file.read_to_string(&mut content) - .change_context(TrustedServerError::Configuration { message: "Failed to read config".into() })?; - ``` -3. **Attachments**: Use `.attach_printable("additional info")` to add debugging context without changing the error variant. -4. **Consistency**: Avoid returning bare `TrustedServerError` unless absolutely necessary (e.g. implementing traits). Wrap them in `Report::new()`. - -## :pray: Credits - -- https://github.com/jessesquires/.github/blob/main/CONTRIBUTING.md -- https://github.com/erikthedeveloper/code-review-emoji-guide +Review the change, not the author. Tie blocking feedback to a correctness, +security, compatibility, maintainability, or documented-requirement concern. +Mark optional improvements as non-blocking and move unrelated work to a +separate issue. diff --git a/crates/trusted-server-adapter-axum/README.md b/crates/trusted-server-adapter-axum/README.md new file mode 100644 index 000000000..3e4863e14 --- /dev/null +++ b/crates/trusted-server-adapter-axum/README.md @@ -0,0 +1,25 @@ +# trusted-server-adapter-axum + +Native development adapter for Trusted Server. It runs the shared application +through Axum on the host, without an edge simulator, and is not a production +deployment target. + +The crate owns Axum route registration, middleware, outbound HTTP through +`reqwest`, and environment-backed platform services. Its route surface follows +the shared core contract, but request-time KV operations are unavailable and a +startup error makes every route, including `/health`, return 500. Application +configuration reaches the process through the documented environment bridge; +the local EdgeZero config-store file is not read directly. + +Build and test from the repository root: + +```bash +cargo build-axum +cargo test-axum +``` + +Run the isolated first-success check with `./scripts/smoke-axum.sh`. See the +[Axum development guide](../../docs/guide/axum-dev.md) for its config and +secret handoff, negative cases, success oracle, and cleanup behavior. Shared +runtime behavior belongs in +[`trusted-server-core`](../trusted-server-core/README.md), not this adapter. diff --git a/crates/trusted-server-adapter-cloudflare/README.md b/crates/trusted-server-adapter-cloudflare/README.md new file mode 100644 index 000000000..208827e68 --- /dev/null +++ b/crates/trusted-server-adapter-cloudflare/README.md @@ -0,0 +1,25 @@ +# trusted-server-adapter-cloudflare + +Cloudflare Workers adapter for Trusted Server. Production code targets +`wasm32-unknown-unknown` with the `cloudflare` feature; native builds exist for +adapter tests and deliberately exclude the Workers entry point. + +The crate translates Worker requests and responses, registers the shared +router, and implements Cloudflare-specific HTTP and secret access. The current +runtime does not open the KV config value written by `ts config push`; startup +instead consumes the nested `TRUSTED_SERVER_CONFIG` variable bridge. It also +has no `/health` route or request-time KV registry and permits one enabled +auction provider. + +Build and test from the repository root: + +```bash +cargo build-cloudflare +cargo test-cloudflare +``` + +Run `./scripts/smoke-cloudflare.sh` for the isolated Wrangler handoff check. +The [Cloudflare deployment guide](../../docs/guide/cloudflare.md) documents the +required double encoding, secret bindings, failure cases, and cleanup. Keep +portable request behavior in +[`trusted-server-core`](../trusted-server-core/README.md). diff --git a/crates/trusted-server-adapter-fastly/README.md b/crates/trusted-server-adapter-fastly/README.md new file mode 100644 index 000000000..24558148a --- /dev/null +++ b/crates/trusted-server-adapter-fastly/README.md @@ -0,0 +1,25 @@ +# trusted-server-adapter-fastly + +Production Fastly Compute adapter for Trusted Server, targeting +`wasm32-wasip1`. + +This crate owns the Fastly entry point, dynamic backend construction, edge +cache behavior, config and secret-store access, EC KV operations, rate limits, +streaming template assembly, and Fastly-only Tinybird auction emission. Its +`/health` response is served before application construction, so liveness does +not prove that configuration loaded. Platform-neutral routing, settings, +auctions, integrations, and rewrites remain in `trusted-server-core`. + +Build and test from the repository root: + +```bash +cargo build-fastly +cargo test-fastly +``` + +The test alias uses Viceroy for the WASM target. Run +`./scripts/smoke-fastly.sh` for the isolated config-store, secret-store, and +publisher-response handoff. See the +[Fastly deployment guide](../../docs/guide/fastly.md) for provisioning and +runtime boundaries and [Auction Telemetry](../../docs/guide/telemetry.md) for +the Fastly-only telemetry path. diff --git a/crates/trusted-server-adapter-spin/README.md b/crates/trusted-server-adapter-spin/README.md new file mode 100644 index 000000000..fe33c9055 --- /dev/null +++ b/crates/trusted-server-adapter-spin/README.md @@ -0,0 +1,23 @@ +# trusted-server-adapter-spin + +Experimental Fermyon Spin adapter for Trusted Server. The component targets +`wasm32-wasip1` with the `spin` feature; native builds exercise route and +platform tests. + +The crate maps Spin HTTP, variables, and the `default` key-value store into the +shared runtime. Startup failures install a restricted 503 router that keeps +`/health` live. The adapter permits one enabled auction provider. Request-time +KV services beyond startup config loading remain unavailable. Its entry point +uses `anyhow::Result` only because the EdgeZero Spin FFI requires that type. + +Build and test from the repository root: + +```bash +cargo build --package trusted-server-adapter-spin --target wasm32-wasip1 --features spin --release +cargo test-spin +``` + +Run `./scripts/smoke-spin.sh` for the isolated config, encoded-secret, failure, +and publisher-response checks. See the +[Spin deployment guide](../../docs/guide/spin.md) for the exact local-store +mapping and operational limitations. diff --git a/crates/trusted-server-cli/README.md b/crates/trusted-server-cli/README.md new file mode 100644 index 000000000..443919b71 --- /dev/null +++ b/crates/trusted-server-cli/README.md @@ -0,0 +1,23 @@ +# trusted-server-cli + +Host-target operator CLI for Trusted Server. The installed binary is `ts`. + +The CLI validates and publishes application configuration through EdgeZero, +delegates platform lifecycle commands, audits public pages and ad-template +configuration, and builds external Prebid artifacts. It is not part of any +adapter WASM artifact. Most commands support Linux and macOS; the production- +hostname development proxy and local CA commands are macOS-only. + +Install and test from the repository root: + +```bash +cargo install-cli +./scripts/test-cli.sh +``` + +The test script selects and, when necessary, installs the host Rust target, +runs the CLI suite, and executes the ignored browser-backed audit fixtures +serially. See the [CLI guide](../../docs/guide/cli.md) for the generated +two-platform command inventory and the +[EdgeZero guide](../../docs/guide/edgezero.md) for configuration lifecycle and +store semantics. diff --git a/crates/trusted-server-core/README.md b/crates/trusted-server-core/README.md index 69575b0d2..1d8c54ea9 100644 --- a/crates/trusted-server-core/README.md +++ b/crates/trusted-server-core/README.md @@ -1,59 +1,51 @@ # trusted-server-core -Utilities shared by Trusted Server components. This crate contains HTML/CSS rewriting helpers used to normalize ad creative assets to first‑party proxy endpoints. - -## Creative Rewriting - -The `creative` module rewrites external asset URLs in creative markup to a unified first‑party proxy so the publisher controls egress. - -Key rules: - -- Proxy absolute/protocol‑relative URLs (http/https or `//`) to `/first-party/proxy?tsurl=&&tstoken=` -- Leave relative URLs unchanged (e.g., `/path`, `../path`, `local/file`) -- Ignore non‑network schemes: `data:`, `javascript:`, `mailto:`, `tel:`, `blob:`, `about:` - -Rewritten locations: - -- ``, `data-src`, `[srcset]`, `[imagesrcset]` -- `` - - The bundle guards anchor clicks by restoring the originally rewritten first‑party link at click time. - - Served through the unified endpoint described below. - -Helpers: - -- `rewrite_creative_html(settings, markup) -> String` — rewrite an HTML fragment -- `rewrite_css_body(settings, css) -> Result` — rewrite a CSS body (`url(...)` entries) -- `rewrite_srcset(settings, srcset) -> String` — proxy absolute candidates; preserve descriptors (`1x`, `1.5x`, `100w`) -- `split_srcset_candidates(srcset) -> Vec<&str>` — robust splitting for commas with/without spaces; avoids splitting the first `data:` mediatype comma - -JS bundles (served by publisher module): - -- Dynamic endpoint: `/static/tsjs=tsjs-unified.min.js?v=` - - At build time, embedded integrations are compiled as separate IIFEs (`tsjs-core.js`, `tsjs-creative.js`, etc.); Prebid is generated externally and served through `/integrations/prebid/bundle.js`. - - At runtime, the server concatenates `tsjs-core.js` + enabled integration modules based on `IntegrationRegistry` config - - The URL filename is fixed for backward compatibility; the `?v=` hash changes when modules change - -Behavior is covered by an extensive test suite in `crates/trusted-server-core/src/creative.rs`. - -## Edge Cookie (EC) Identifier Propagation - -- The `ec/` module owns the EC identity subsystem: - - `ec/generation.rs` — creates HMAC-based IDs using the client IP and publisher passphrase (format: `64hex.6alnum`). - - `ec/mod.rs` — `EcContext` struct with two-phase lifecycle (`read_from_request` + `generate_if_needed`), `get_ec_id` helper. - - `ec/consent.rs` — EC-specific consent gating wrapper. - - `ec/cookies.rs` — `Set-Cookie` header creation and expiration helpers. -- `publisher.rs::handle_publisher_request` issues the `ts-ec` cookie when absent so the browser keeps the identifier on subsequent requests. -- `proxy.rs::handle_first_party_proxy` replays the identifier to third-party creative origins by appending `ts-ec=` to the reconstructed target URL, follows redirects (301/302/303/307/308) up to four hops, and keeps downstream fetches linked to the same user scope. -- `proxy.rs::handle_first_party_click` adds `ts-ec=` to outbound click redirect URLs so analytics endpoints can associate clicks with impressions without third-party cookies. +Portable application core shared by every Trusted Server adapter. It targets +both supported WASM environments and native adapter tests, so it must not depend +on an edge SDK, Tokio runtime, filesystem, socket, or host-only process API. + +## Responsibilities + +- `settings` and `settings_data` define typed application configuration, + validation, secret references, and runtime normalization. +- `platform` defines the HTTP, backend, store, geo, client-info, and telemetry + service boundary adapters implement. +- `publisher`, `router`, `handlers`, and `response` dispatch publisher and + administrative requests through platform-neutral request/response types. +- `auction` builds plans, invokes providers and mediators, selects bids, and + emits bounded telemetry events. +- `integrations` registers explicit proxy, rewrite, injection, filter, + post-processing, provider, and browser-module capabilities. +- `ec` owns edge-cookie generation, consent decisions, identity graph access, + and partner synchronization. +- `html_processor`, `host_rewrite`, `streaming_processor`, and `rsc_flight` + transform eligible publisher responses without corrupting non-HTML or RSC + payloads. +- `proxy`, `creative`, `image_optimizer`, and `asset_routes` implement bounded + first-party asset and creative handling. +- `auth`, `request_signing`, `key_manager`, and `jwk` implement authentication + and signing contracts. +- `cache_policy`, `template_cache`, and `template_assembly` define portable + cache and assembly decisions; adapters supply storage and streaming I/O. +- `tsjs` selects embedded browser modules supplied by `trusted-server-js`. +- `openrtb` connects auction logic to the checked OpenRTB data model. + +Adapter crates own runtime startup, SDK conversion, concrete storage, outbound +transport, and target-specific limitations. Adding an integration should use +the narrowest registration hook and the core-neutral `RuntimeServices` +boundary; see the [Integration Guide](../../docs/guide/integration-guide.md). + +## Build and test + +From the repository root: + +```bash +cargo build-fastly +cargo test-fastly +``` + +The Fastly aliases compile and test the core for `wasm32-wasip1` through +Viceroy. Cloudflare and native adapter suites exercise the same core under +their target configurations. Use [TESTING.md](../../TESTING.md) for the complete +target matrix and [Architecture](../../docs/guide/architecture.md) for the +request-level system view. diff --git a/crates/trusted-server-integration-tests/README.md b/crates/trusted-server-integration-tests/README.md index e82cb8837..2d3981721 100644 --- a/crates/trusted-server-integration-tests/README.md +++ b/crates/trusted-server-integration-tests/README.md @@ -1,257 +1,40 @@ -# Integration Tests +# trusted-server-integration-tests -End-to-end tests that verify the trusted server against real frontend -containers using [Testcontainers](https://testcontainers.com/) and -[Playwright](https://playwright.dev/). +Native test package for cross-adapter parity, documentation compilation, and +end-to-end publisher behavior. It is not linked into an adapter artifact. -## Prerequisites +## Test surfaces -- **Docker** — running and accessible -- **Viceroy** — Fastly local simulator (`cargo install viceroy --version 0.17.0 --locked --force`) -- **wasm32-wasip1 target** — `rustup target add wasm32-wasip1` -- **Node.js** — version pinned in `.tool-versions`, for browser tests only +- `tests/parity.rs` calls Axum, Cloudflare, and Spin routers in process and + compares their shared route behavior. +- `tests/documentation_snippets.rs` extracts the checked integration-guide + fixture and compiles it in an isolated offline crate. +- `tests/integration.rs` exercises the Fastly/Viceroy and Axum paths against the + WordPress and Next.js fixture containers. +- `browser/` uses Playwright and Chromium to verify script loading, navigation, + rewriting, APS rendering, and GPT diagnostics in real pages. -## Quick start +The package uses the native host target. Fastly application artifacts are +compiled separately for `wasm32-wasip1`; Docker, Viceroy, Node, and Chromium +are required only by the end-to-end surfaces that invoke them. -### HTTP-level tests +## Run -```bash -./scripts/integration-tests.sh -``` - -This script handles everything: - -1. Builds the WASM binary -2. Generates Viceroy configs from the readable `trusted-server.integration.toml` - fixture -3. Builds the WordPress and Next.js Docker images -4. Runs all integration tests sequentially - -### Browser tests +From the repository root: ```bash +cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity +cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test documentation_snippets +./scripts/integration-tests.sh ./scripts/integration-tests-browser.sh ``` -This script: - -1. Builds the WASM binary and Docker images (same as above) -2. Generates the Viceroy config consumed by Playwright global setup -3. Installs Playwright and Chromium -4. Runs browser tests for Next.js and WordPress sequentially - -### Run a single test - -```bash -# HTTP-level -./scripts/integration-tests.sh test_wordpress_fastly -./scripts/integration-tests.sh test_nextjs_fastly - -# Browser — single framework after building WASM/images and generating configs -cd crates/trusted-server-integration-tests/browser -VICEROY_CONFIG_PATH=../../../target/integration-test-artifacts/configs/viceroy.toml \ -TEST_FRAMEWORK=nextjs npx playwright test -VICEROY_CONFIG_PATH=../../../target/integration-test-artifacts/configs/viceroy.toml \ -TEST_FRAMEWORK=wordpress npx playwright test -``` - -### Verbose output - -```bash -./scripts/integration-tests.sh --nocapture -``` - -## Docker images - -Two test images are built from fixtures in `fixtures/frameworks/`: - -| Image | Dockerfile | Description | -|---|---|---| -| `test-wordpress:latest` | `fixtures/frameworks/wordpress/Dockerfile` | PHP built-in server with a minimal test theme | -| `test-nextjs:latest` | `fixtures/frameworks/nextjs/Dockerfile` | Next.js 14 standalone app with 4 pages, API routes, forms, shared navigation, and deferred scripts | - -Both images include test fixtures with absolute origin URLs (`ORIGIN_HOST` env -var) so the trusted server's URL rewriting can be verified. - -### Build images manually - -```bash -docker build -t test-wordpress:latest \ - crates/trusted-server-integration-tests/fixtures/frameworks/wordpress/ - -docker build \ - --build-arg NODE_VERSION="$(grep '^nodejs ' .tool-versions | awk '{print $2}')" \ - -t test-nextjs:latest \ - crates/trusted-server-integration-tests/fixtures/frameworks/nextjs/ -``` - -## Generated Viceroy configs - -The source-controlled Viceroy template contains only local runtime resources such -as KV stores, secret stores, and JWKS config. The Trusted Server application -config is kept as readable TOML in -`fixtures/configs/trusted-server.integration.toml` and converted into an -EdgeZero `BlobEnvelope` at test setup time. - -Generate the post-cutover Viceroy config manually with: - -```bash -ARTIFACTS_DIR=target/integration-test-artifacts \ -INTEGRATION_ORIGIN_PORT=8888 \ -./scripts/generate-integration-viceroy-configs.sh -``` - -Generated output: - -| File | Purpose | -|---|---| -| `target/integration-test-artifacts/configs/viceroy.toml` | Fastly integration, EC lifecycle, and browser tests | - -Set `VICEROY_CONFIG_PATH` to the generated file when invoking `cargo test` or -Playwright directly. - -## Test scenarios - -### HTTP-level — standard (all frameworks) - -| Scenario | What it tests | -|---|---| -| `HtmlInjection` | Exactly one ` ``` **Timing**: Injected **once per HTML response** before any other scripts. -### Integration Bundles +### Integration bundles -Integrations can request additional bundles: +The integration registry selects TSJS modules by enabled integration ID. A +registered ID with a compiled module is included in the immediate unified +bundle unless its builder calls `.with_deferred_js()`; deferred modules are +served separately as `/static/tsjs=tsjs-.min.js`. Builders can call +`.without_js()` when the Rust integration must not select a TSJS module. -```rust -IntegrationRegistration::builder("my_integration") - .with_asset("my_integration") // Requests tsjs-my_integration.min.js - .build() -``` - -**Result**: - -```html - - - - - -``` +The always-present `creative` module and all immediate integration modules are +served through `/static/tsjs=tsjs-unified.min.js`. Trusted Server does not +accept an arbitrary asset name from an integration registration. ### Bundle Types diff --git a/docs/guide/edgezero.md b/docs/guide/edgezero.md new file mode 100644 index 000000000..30d56856b --- /dev/null +++ b/docs/guide/edgezero.md @@ -0,0 +1,76 @@ +# EdgeZero Lifecycle and Stores + +EdgeZero is the deployment and configuration layer used by the `ts` CLI. +The repository's `edgezero.toml` declares one application, three logical +stores, and the Fastly, Axum, Cloudflare, and Spin adapter commands. + +## Lifecycle + +Use `ts config init` to create a file, edit the generated +`trusted-server.toml`, and run `ts config validate` before any platform +write. `ts config diff --adapter ADAPTER` compares the validated local +configuration with the selected platform value. `ts config push --adapter +ADAPTER` writes it. Build, serve, deploy, health-check, rollback, and active +version operations use the adapter commands declared in `edgezero.toml`. + +Configuration is startup state, not a live control plane. A successful push +does not change a running instance until the target's restart or deployment +path loads that value. Use `--staging` only with the corresponding staging +deployment flow: it writes `LOGICAL_ID_staging` in the same physical store, +while production continues to read the ordinary logical key. + +## Logical and physical stores + +The manifest declares portable logical IDs: + +| Kind | Logical ID | +| ------------- | ------------------------ | +| KV | `trusted_server_kv` | +| Configuration | `trusted_server_config` | +| Secrets | `trusted_server_secrets` | + +An adapter maps those IDs to platform resources. The runtime-facing +`StoreName` used for reads and the management-facing `StoreId` used for +writes are deliberately distinct types. Environment mappings such as +`EDGEZERO__STORES__CONFIG__TRUSTED_SERVER_CONFIG__NAME` may change the +physical store name without changing application configuration. + +Fastly binds its stores in `fastly.toml`. Axum uses isolated local state for +development. Cloudflare maps the configuration value through its Worker +binding. Spin resolves the manifest mapping and writes supported local +key-value backends. Consult the adapter guide before assuming that a declared +logical store is wired for runtime reads on every target. + +## Blob and chunk flow + +`config push` serializes validated settings into one EdgeZero +`BlobEnvelope`. Store-backed settings contain secret key names, not secret +values; the adapter resolves those keys from the logical secret store during +startup. + +When a Fastly envelope exceeds one Config Store entry, the writer stores +bounded chunks and replaces the root value with a versioned +`fastly_config_chunks` pointer. Startup verifies each chunk's declared +length and SHA-256 and then verifies the reconstructed envelope. A missing, +oversized, reordered, or modified chunk fails configuration loading. + +`ts config gc --adapter fastly` previews orphaned chunks by default. An +actual deletion requires `--yes --older-than WINDOW`. The age assertion +applies to the whole physical store, not one root key; verify the reported +store before deletion, especially when using `--store` or `--no-env`. + +## Safe operator sequence + +1. Put required secret values in the physical store mapped from + `trusted_server_secrets`. +2. Run `ts config validate`. +3. Review `ts config diff --adapter ADAPTER`; use `--no-diff` when + deliberately inline configuration must not appear in logs. +4. Run `ts config push --adapter ADAPTER`. +5. Start or deploy the adapter and verify a non-health publisher route. A + healthy endpoint alone may not prove that application configuration loaded. + +See [Configuration](/guide/configuration) for the field contract, the +[CLI guide](/guide/cli) for exact command help, and +[deployment guides](/guide/integrations-overview#adapter-support) for target +behavior. diff --git a/docs/guide/error-reference.md b/docs/guide/error-reference.md index 0f7fd26bc..3b3fe65ed 100644 --- a/docs/guide/error-reference.md +++ b/docs/guide/error-reference.md @@ -8,7 +8,7 @@ Common errors, their causes, and solutions when working with Trusted Server. - [Runtime Errors](#runtime-errors) - [Integration Errors](#integration-errors) - [Request Signing Errors](#request-signing-errors) -- [Build & Deployment Errors](#build--deployment-errors) +- [Build & Deployment Errors](#build-deployment-errors) --- @@ -606,10 +606,10 @@ npm ci npm run build ``` -3. Check for TypeScript errors: +3. Run the TypeScript-aware ESLint checks: ```bash -npm run type-check +npm run lint ``` 4. Skip TSJS build temporarily: diff --git a/docs/guide/fastly.md b/docs/guide/fastly.md index f63b4f627..bc1174033 100644 --- a/docs/guide/fastly.md +++ b/docs/guide/fastly.md @@ -2,6 +2,17 @@ This guide covers setting up your Fastly account and Compute service for Trusted Server. +## Support status + +| Adapter | Release status | Health | Startup status | Startup health | Provider fan-out | Trusted-client-IP handling | Request normalization | +| -------- | -------------- | ---------- | -------------- | -------------- | ---------------- | ------------------------------ | --------------------- | +| `fastly` | production | pre router | `500` | yes | multiple | entry-point resolve + sanitize | none | + +The row above summarizes the +[adapter-support contract](./api-reference#adapter-and-startup-support). A healthy +response does not prove that configuration loaded: Fastly serves `/health` +before it constructs the application. + ## Create a Fastly Account 1. Go to [manage.fastly.com](https://manage.fastly.com) and create an account if you don't have one @@ -345,9 +356,41 @@ fastly resource-link list --service-id --version If EC sync returns `kv_unavailable` or identify responses are degraded, first check that the identity store is present and linked to the active version. Legacy partner/consent KV bindings can be removed once no deployment-specific tooling depends on them. +## Verify the complete local handoff + +Run the repository smoke from a clean shell: + +```bash +./scripts/smoke-fastly.sh +``` + +The script creates an isolated application config, applies its publisher-origin +overrides, and runs strict validation. It then executes `ts config push +--adapter fastly --local`, adds all three required entries to +`[local_server.secret_stores.ts_secrets]`, and starts `fastly compute serve` +through Viceroy. The required keys are `handler_password`, +`publisher_proxy_secret`, and `ec_passphrase`. + +The check deliberately proves both halves of startup. With no config entry, it +requires `/health` to return 200 while the publisher route returns 500 with the +missing `trusted_server_config` diagnostic. It then removes each required +secret independently and requires the corresponding setting path to fail. +Finally, the publisher request must return 200, retain the stub-origin +sentinel, rewrite an origin URL to the Fastly listener, and omit the original +URL. A green health response cannot satisfy that final assertion. + +The script copies `edgezero.toml` and `fastly.toml` into a per-run temporary +project before pushing local configuration or adding synthetic secrets. The +tracked manifests remain untouched, including when smoke runs overlap. Its trap +stops Viceroy and the stub origin and removes the temporary project. For a +deployed service, provision and link the stores described above and write the +three secrets through Fastly's secret-store interface. + ## Next Steps - Return to [Getting Started](/guide/getting-started) to continue setup - See [Configuration](/guide/configuration) for detailed configuration options - See [EC Setup Guide](/guide/ec-setup-guide) for end-to-end EC verification - See [Request Signing](/guide/request-signing) for setting up cryptographic signing +- Compare the [Cloudflare](./cloudflare), [Spin](./spin), and [Axum](./axum-dev) + adapter journeys diff --git a/docs/guide/index.md b/docs/guide/index.md index e69de29bb..8a2c00d04 100644 --- a/docs/guide/index.md +++ b/docs/guide/index.md @@ -0,0 +1,30 @@ +# Guide + +Use these guides to understand Trusted Server, configure a deployment, and +contribute to the project. + +## Start here + +- [What is Trusted Server?](/guide/what-is-trusted-server) +- [Getting Started](/guide/getting-started) + +## Core concepts + +- [Edge Cookies](/guide/edge-cookies) +- [GDPR Compliance](/guide/gdpr-compliance) +- [Ad Serving](/guide/ad-serving) +- [First-Party Proxy](/guide/first-party-proxy) +- [Asset Routes](/guide/asset-routes) + +## Development + +- [Architecture](/guide/architecture) +- [Configuration](/guide/configuration) +- [CLI](/guide/cli) +- [Testing](/guide/testing) +- [Integration Guide](/guide/integration-guide) + +## Reference + +- [API Reference](/guide/api-reference) +- [Error Reference](/guide/error-reference) diff --git a/docs/guide/integration-guide.md b/docs/guide/integration-guide.md index cf7da71f6..f127d4207 100644 --- a/docs/guide/integration-guide.md +++ b/docs/guide/integration-guide.md @@ -1,380 +1,223 @@ -# Integration Guide +# Integration Development -This document explains how to integrate a new integration module with the Trusted Server runtime. The workflow mirrors the built-in `testlight` sample in `crates/trusted-server-core/src/integrations/testlight.rs`. +Trusted Server integrations are platform-neutral registrations assembled by +`IntegrationRegistry`. Adapter crates provide I/O through +`RuntimeServices`; integration code must not import Fastly, Cloudflare, +Axum, or Spin SDK types. -## Architecture Overview +## Choose the narrowest hook -| Component | Purpose | -| ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `crates/trusted-server-core/src/integrations/registry.rs` | Defines the `IntegrationProxy`, `IntegrationAttributeRewriter`, `IntegrationScriptRewriter`, and `IntegrationHeadInjector` traits and hosts the `IntegrationRegistry`, which drives proxy routing, HTML/text rewrites, and head injection. | -| `Settings::integrations` (`crates/trusted-server-core/src/settings.rs`) | Free-form JSON blob keyed by integration ID. Use `IntegrationSettings::insert_config` to seed configs; each module deserializes and validates (`validator::Validate`) its own config and exposes an `enabled` flag so the core settings schema stays stable. | -| Fastly entrypoint (`crates/trusted-server-adapter-fastly/src/main.rs`) | Instantiates the registry once per request, routes `/integrations//…` requests to the appropriate proxy, and passes the registry to the publisher origin proxy so HTML rewriting remains integration-aware. | -| `html_processor.rs` | Applies first-party URL rewrites, injects the Trusted Server JS shim, and lets integrations override attribute values (for example to swap script URLs). | +- `IntegrationProxy` owns explicit method/path endpoints and receives + `Settings`, `RuntimeServices`, and an EdgeZero-neutral request. +- `IntegrationAttributeRewriter` inspects selected HTML attributes. +- `IntegrationScriptRewriter` handles one declared selector. +- `IntegrationHeadInjector` inserts deterministic head markup. +- `IntegrationHtmlPostProcessor` is for bounded whole-document work that + cannot be performed during streaming. +- `IntegrationRequestFilter` makes an early request decision. -## Step-by-Step Integration +Build one `IntegrationRegistration` with only the hooks the feature needs. +Use `with_deferred_js()` only for a separately loaded integration module and +`without_js()` when another asset path owns delivery. Proxy routes should be +namespaced and bounded; do not introduce a general outbound proxy. -### 1. Define Integration Configuration +## Compiling core-neutral fixture -Add a `trusted-server.toml` block and any environment overrides under `TRUSTED_SERVER__INTEGRATIONS____*`. Configuration values are exposed to your module via `Settings::integration_config()`. +The fixture below registers an attribute rewriter and constructs every +required `RuntimeServices` service without an adapter dependency. The +documentation test extracts this exact fence and compiles it as an isolated +crate. -```toml -[integrations.my_integration] -endpoint = "https://example.com/api" -timeout_ms = 1000 -rewrite_scripts = true -``` - -### 2. Create the Integration Module - -Add a module under `crates/trusted-server-core/src/integrations//mod.rs` (see `crates/trusted-server-core/src/integrations/testlight.rs` for reference) and expose it in `crates/trusted-server-core/src/integrations/mod.rs`. - -Key pieces: + ```rust -#[derive(Deserialize, Validate)] -struct MyIntegrationConfig { - #[serde(default = "default_enabled")] - enabled: bool, - // … -} - -impl IntegrationConfig for MyIntegrationConfig { - fn is_enabled(&self) -> bool { self.enabled } -} - -pub struct MyIntegration { - config: MyIntegrationConfig, -} - -pub fn build(settings: &Settings) -> Option> { - let config = settings - .integration_config::("my_integration") - .ok() - .flatten()?; - Some(Arc::new(MyIntegration { config })) -} - -// Tests or scaffolding code can seed configs without hand-writing JSON: -settings - .integrations - .insert_config( - "my_integration", - &serde_json::json!({ - "enabled": true, - "endpoint": "https://example.com/api" - }), - )?; -``` - -`Settings::integration_config::` automatically deserializes the raw JSON blob, runs [`validator`](https://docs.rs/validator/latest/validator/) on the type, and drops configs whose `is_enabled` returns `false`. Always derive/implement `Validate` for schema enforcement and implement `IntegrationConfig` (typically wrapping a `#[serde(default)] enabled` flag) so operators can toggle integrations without code changes. - -### 3. Return an IntegrationRegistration +use std::net::IpAddr; +use std::sync::Arc; + +use error_stack::Report; +use trusted_server_core::integrations::{ + AttributeRewriteAction, IntegrationAttributeContext, + IntegrationAttributeRewriter, IntegrationRegistration, +}; +use trusted_server_core::platform::{ + BackendNamingPolicy, ClientInfo, GeoInfo, PlatformBackend, + PlatformBackendSpec, PlatformConfigStore, PlatformError, PlatformGeo, + PlatformSecretStore, RuntimeServices, StoreId, StoreName, + UnavailableHttpClient, UnavailableKvStore, +}; + +struct ReadOnlyStore; + +impl PlatformConfigStore for ReadOnlyStore { + fn get( + &self, + _store: &StoreName, + _key: &str, + ) -> Result> { + Err(Report::new(PlatformError::Unsupported)) + } -Each integration registers itself via a `register` function that returns an `IntegrationRegistration`. This object describes which HTTP proxies and HTML rewrites the integration exposes: + fn put( + &self, + _store: &StoreId, + _key: &str, + _value: &str, + ) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) + } -```rust -pub fn register(settings: &Settings) -> Option { - let integration = build(settings)?; - Some( - IntegrationRegistration::builder("my_integration") - .with_proxy(integration.clone()) - .with_attribute_rewriter(integration.clone()) - .with_script_rewriter(integration.clone()) - .with_head_injector(integration) - .with_asset("my_integration") - .build(), - ) + fn delete( + &self, + _store: &StoreId, + _key: &str, + ) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) + } } -``` - -Any combination of the vectors may be populated. Modules that only need HTML rewrites can skip the `proxies` field altogether, and vice versa. The registry automatically iterates over the static builder list in `crates/trusted-server-core/src/integrations/mod.rs`, so adding the new `register` function is enough to make the integration discoverable. - -### 4. Implement IntegrationProxy for Endpoints -Implement the trait from `registry.rs` when your integration needs its own HTTP entrypoint: - -```rust -#[async_trait(?Send)] -impl IntegrationProxy for MyIntegration { - fn integration_name(&self) -> &'static str { - "my_integration" +impl PlatformSecretStore for ReadOnlyStore { + fn get_bytes( + &self, + _store: &StoreName, + _key: &str, + ) -> Result, Report> { + Err(Report::new(PlatformError::Unsupported)) } - fn routes(&self) -> Vec { - vec![ - self.post("/auction"), - self.get("/status"), - ] + fn create( + &self, + _store: &StoreId, + _key: &str, + _value: &str, + ) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) } - async fn handle( + fn delete( &self, - settings: &Settings, - req: Request, - ) -> Result> { - // Parse/generate EC IDs, forward upstream, and return the response. + _store: &StoreId, + _key: &str, + ) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) } } -``` - -::: tip Route Helpers -Use the provided helper methods to automatically namespace your routes under `/integrations/{integration_name()}/`. Available helpers: `get()`, `post()`, `put()`, `delete()`, and `patch()`. This lets you define routes with just their relative paths (e.g., `self.post("/auction")` becomes `"/integrations/my_integration/auction"`). -::: - -Routes are matched verbatim in `crates/trusted-server-adapter-fastly/src/main.rs`, so stick to stable paths and register whichever HTTP methods you need. **New integrations should namespace their routes under `/integrations/{INTEGRATION_NAME}/`** using the helper methods for consistency, but you can define routes manually if needed (e.g., for backwards compatibility). - -The shared context already injects Trusted Server logging, headers, and error handling; the handler only needs to deserialize the request, call the upstream endpoint, and stamp integration-specific headers. - -#### Proxying Upstream Requests - -Use the shared helper in `crates/trusted-server-core/src/proxy.rs` to forward requests so you automatically get the same header copying, redirect handling, HTML/CSS rewrite behavior, and EC ID handling the first-party proxy uses: -```rust -use crate::proxy::{proxy_request, ProxyRequestConfig}; -use fastly::http::{header, HeaderValue}; - -let payload = serde_json::to_vec(&my_body)?; -let response = proxy_request( - settings, - req, - ProxyRequestConfig::new(&self.config.endpoint) - .with_body(payload) - .with_header(header::CONTENT_TYPE, HeaderValue::from_static("application/json")) - .with_streaming(), // stream passthrough; disable if you need HTML rewrites -) -.await?; -``` - -Set `forward_ec_id` to `false` if the upstream should not receive the caller's EC ID (`Testlight` does this), and disable `follow_redirects` if you need to surface redirects directly to the caller. - -**Streaming passthrough example:** - -```rust -let response = proxy_request( - settings, - req, - ProxyRequestConfig::new("https://example.com/pixel") - .with_streaming() // no HTML/CSS rewrites; preserves origin compression -); -``` +struct FixtureBackend; -::: info When to Use Streaming -Use streaming when the upstream response is binary or large and you do not need creative rewrites. Keep the default (non-streaming) mode when you want HTML/CSS content rewritten through the existing creative pipeline. -::: - -### 5. Implement HTML Rewrite Hooks (Optional) - -If the integration needs to rewrite script/link tags or inject HTML, implement `IntegrationAttributeRewriter` for attribute mutation and `IntegrationScriptRewriter` for inline `"#, - ctx.request_host - )] +impl PlatformGeo for FixtureGeo { + fn lookup( + &self, + _client_ip: Option, + ) -> Result, Report> { + Ok(None) } } -``` - -`html_processor.rs` calls `head_inserts` once per HTML response when the `` element is first encountered. The returned snippets are concatenated before the unified script tag, so ordering between integrations is not guaranteed — keep snippets self-contained. - -::: tip When to Use Head Injection -Use `IntegrationHeadInjector` when you need to emit configuration, inline scripts, or `` tags that must appear early in ``. For attribute or script content changes on existing elements, prefer `IntegrationAttributeRewriter` or `IntegrationScriptRewriter` instead. -::: - -### 6. Register the Module - -Add the module to `crates/trusted-server-core/src/integrations/mod.rs`'s builder list. The registry will call its `register` function automatically. Once registered: - -- `crates/trusted-server-adapter-fastly/src/main.rs` automatically exposes the declared route(s). -- `handle_publisher_request` receives the same registry so HTML responses get integration shims without further code changes. -- `IntegrationRegistry::registered_integrations()` exposes a machine-readable summary of hooks for tests, tooling, or diagnostics. -- Declared assets are injected automatically into ``; the runtime emits `