Skip to content

Rewrite Sourcepoint responses without Content-Length - #1183

Draft
ChristianPavilonis wants to merge 2 commits into
mainfrom
fix/sourcepoint-unknown-length-1088
Draft

ChristianPavilonis wants to merge 2 commits into
mainfrom
fix/sourcepoint-unknown-length-1088

Conversation

@ChristianPavilonis

Copy link
Copy Markdown
Collaborator

Summary

  • Rewrite eligible Sourcepoint JavaScript and HTML even when upstream omits Content-Length, so embedded URLs and privacy-manager assets still use the first-party proxy.
  • Use Fastly's streaming response path and the existing 5 MiB collector. Bodies that exceed the limit during collection return 502; declared oversized bodies still pass through unchanged.
  • Request uncompressed /mms/v2/get_site_data responses and preserve their upstream and cookie-aware cache policy instead of applying the static JavaScript cache policy.

Changes

File Change
crates/trusted-server-core/src/integrations/sourcepoint.rs Remove the missing-length bypass, request streaming where supported, handle site-data encoding and caching, and add regression tests for rewriting, limits, pass-through, and headers.
docs/guide/integrations/sourcepoint.md Document the rewrite limit, 502 overflow policy, cache behavior, and adapter limitations.

Scope

Limited to the Sourcepoint integration and its documentation, using existing collection and streaming APIs. Most added code is regression coverage. No adapter implementation or browser JavaScript changes are included.

Fastly enforces the limit while reading the upstream stream. Cloudflare and Spin still buffer upstream bodies before the integration checks them; fixing that adapter-level limitation is deferred. This change does not address campaign or consent-state behavior that can suppress the banner.

Closes

Closes #1088

Test plan

  • cargo test-fastly && cargo test-axum
  • cargo clippy-fastly && cargo clippy-axum
  • cargo fmt --all -- --check
  • JS tests: cd crates/trusted-server-js/lib && npx vitest run (893 passed)
  • JS format: cd crates/trusted-server-js/lib && npm run format
  • Docs format: cd docs && npm run format
  • WASM release build: cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1
  • Manual testing via fastly compute serve
  • cargo test-cloudflare && cargo test-spin
  • cargo clippy-cloudflare && cargo clippy-cloudflare-wasm && cargo clippy-spin-native && cargo clippy-spin-wasm
  • cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity (13 passed)
  • cargo test-fastly integrations::sourcepoint (63 passed under Viceroy)

The new missing-length JavaScript and HTML tests failed before the fix and passed afterward. Stream tests cover exactly 5 MiB, overflow, understated lengths, and stopping reads at the limit. Independent review found no introduced correctness issues.

Tests use stub upstream streams. A live Sourcepoint exchange, wire framing, and deployed cache behavior have not been smoke-tested.

Checklist

  • Changes follow CLAUDE.md conventions
  • No unwrap() in production code
  • Uses log macros, not println!, as required by CLAUDE.md
  • New code has tests
  • No secrets or credentials committed

Allow bounded JavaScript and HTML rewriting when upstream responses omit
Content-Length. Request streaming on supported adapters so the 5 MiB
collector can stop before buffering an oversized response.

Return 502 on collection overflow, retain declared-oversize pass-through,
and request identity encoding for site data without overriding its dynamic
cache policy. Cover stream limits, rewriting, pass-through, and headers.

Closes #1088
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Rewrite bounded Sourcepoint responses when upstream omits Content-Length

2 participants