Skip to content

fix(fsregion): refuse cross-origin redirects for the credential-bearing region catalog request - #95

Merged
srstack merged 1 commit into
mainfrom
fix/fsregion-redirect-and-base-url
Nov 18, 2026
Merged

srstack merged 1 commit into
mainfrom
fix/fsregion-redirect-and-base-url

Conversation

@srstack

@srstack srstack commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

The Filesystem region catalog client built its HTTP client as &http.Client{Transport: digestTransport, Timeout: catalogTimeout} with a nil CheckRedirect, so Go's default policy followed up to ten redirects to any host and any scheme.

That request is the one place ti presents the organization API key pair to a non-user-chosen URL, and it does so through internal/api/transport.NewDigest. A Digest transport answers whatever 401 challenge it receives, so a redirect to an attacker's origin makes ti compute and send a Digest response for that origin — the credentials leak exactly as reported. (net/http strips Authorization on cross-host redirects for plain header-based clients, but that protection does not apply to a transport that recomputes the header.)

Fix

  • One shared policy, no drift. The same-origin redirect policy already protecting internal/api moved into internal/api/transport (SameOriginRedirect, SameOrigin, MaxRedirects, ErrTooManyRedirects). Both credential-bearing clients now use it: internal/api keeps mapping a same-origin redirect loop to api.redirect_loop, and internal/fsregion applies the same policy to its Digest client, including the caller-supplied-client seam the resolver uses.
  • Refused redirects are reported, not silently swallowed. A cross-origin catalog redirect now fails with api.fs_regions_redirect_refused naming both origins, instead of collapsing into a generic catalog failure.
  • A caller-supplied redirect policy cannot widen it. transport.SameOriginThen(caller) runs the mandatory same-origin rule first and then the caller's policy, so a caller can narrow redirects (refuse all of them) but never re-enable a cross-origin hop. Both internal/fsregion and internal/api compose their caller-client seam through it; internal/api had the same widening hole.
  • The endpoint itself is validated, and rejected URLs are redacted. Fetch and NormalizeServiceAddress share parseSecureOrigin: absolute, host name required, no embedded credentials, query (including a bare ?), or fragment, and https except for a loopback test server — checked before any round trip, because this request spends the organization API key pair. describeRejectedOrigin strips userinfo/query/fragment and parse errors never echo the raw string, so a password or token in the URL cannot reach stderr.
  • Redirect classification is shared. transport.CrossOriginRedirect reduces both origins to scheme://host and reports false for a same-origin redirect, a Location-less 3xx, and a non-redirect, so a same-origin redirect refused by a stricter caller policy falls through to normal status handling (instead of a self-contradictory "from X to X"). The SQL operation log records db.sql_http_redirect_loop rather than contradicting the command error with db.sql_http_network.
  • The rule covers every credential-bearing client. internal/db/sqlhttp (Basic-authenticated SQL HTTPS requests) now composes transport.SameOriginThen too and reports db.sql_http_redirect_refused / db.sql_http_redirect_loop; net/http still copies Authorization across same-host port changes, subdomain redirects, and https→http downgrades, so it needed the same treatment.

Verification

  • internal/api/transport: TestSameOriginRedirectKeepsCredentialsOnTheSameOrigin — same-origin path and case-folded host followed; other host and other scheme refused with http.ErrUseLastResponse; the redirect chain stops at MaxRedirects.
  • internal/fsregion: TestFetchRefusesCrossOriginRedirect (error code plus zero requests reaching the second origin), TestFetchAppliesRedirectPolicyToCallerClient, TestFetchEnforcesSameOriginUnderCallerRedirectPolicy (a permissive caller policy cannot widen it), TestFetchHonorsStricterCallerRedirectPolicy (a caller that refuses all redirects still narrows), TestFetchFollowsSameOriginRedirect (ordinary same-origin redirects still work), TestFetchRejectsInsecureCatalogEndpoint (http:// non-loopback, ftp://, userinfo, query, fragment, and scheme-less values, with the expected message per case; loopback http still reaches the server).
  • internal/api pins the no-widening rule on its own caller-client seam.
  • internal/api/transport: TestSameOriginThenKeepsCallerRestrictionsWithoutWidening.
  • internal/fsregion: TestRejectedOriginsNeverEchoSecrets (no password or query token in either error path, including opaque URLs and a pasted token as the path).
  • internal/api/transport: TestCrossOriginRedirectClassification and TestSameOriginRedirectFailsClosedOnMissingInput (also covers the same-host port change).
  • internal/db/sqlhttp: TestExecuteHTTPRefusesCrossOriginRedirect (attacker origin receives zero requests), the permissive-caller-policy variant, TestExecuteHTTPFollowsSameOriginRedirect (307 keeps the authenticated POST and body), TestExecuteHTTPSameOriginRedirectRefusedByCallerFallsThrough, TestExecuteHTTPRecordsRedirectLoopEvent, and TestExecuteHTTPReportsRedirectLoop (mutation-checked: without the composition the attacker receives the Basic-auth request).
  • Mutation-checked: with CheckRedirect removed, the cross-origin test fails because the attacker's 401 challenge is answered (auth.invalid_credentials from the attacker's realm). With the policy in place the attacker receives no request.
  • gofmt clean; go vet ./... clean on linux, GOOS=windows, GOOS=darwin; go test -count=1 ./... passes; make e2e passes.

README.md and AGENTS.md document the rule: a credential-bearing client must never be built with a nil CheckRedirect.

@srstack
srstack requested review from mornyx and qiffang and a balanced review from Copilot November 18, 2026 08:05
@ti-chi-bot ti-chi-bot Bot added the size/L label Nov 18, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A critical security issue remains because the Digest-authenticated fsregion client can inherit a caller-provided CheckRedirect policy instead of enforcing SameOriginRedirect, risking cross-origin redirect following and credential leakage (with an additional minor endpoint-validation error-message clarity nit).

2 open issues
What changed in this PR

Hardens the filesystem region-catalog fetch path to prevent credential leakage via cross-origin redirects when using Digest authentication by enforcing a same-origin redirect policy and validating the catalog endpoint.

Changes:

  • Centralize a shared same-origin-only redirect policy in internal/api/transport and reuse it across credential-bearing clients.
  • Update internal/fsregion to apply the redirect policy, surface refused redirects with a dedicated error code, and validate catalog endpoints before any request.
  • Add unit tests for redirect and endpoint-validation behavior, and document the credential-safe redirect rule.
File Description
README.md Documents the credential-safe redirect rule and security constraints around the filesystem region catalog endpoint and redirects.
internal/​fsregion/​fsregion_test.go Adds tests covering cross-origin redirect refusal, same-origin redirect following, caller-supplied client behavior, and insecure endpoint rejection.
internal/​fsregion/​client.go Implements same-origin redirect handling for the Digest-authenticated region-catalog request and validates the catalog endpoint (scheme/host and disallowed components).
internal/​api/​transport/​transport.go Introduces shared helpers/constants for credential-safe redirect handling (same-origin checks, redirect cap, and redirect errors).
internal/​api/​transport/​transport_test.go Adds unit tests validating the shared redirect policy behavior (same-origin allowed, cross-origin refused, and redirect-limit handling).
internal/​api/​client.go Switches the internal API client to use the shared redirect policy and origin-check helpers from internal/api/transport.
AGENTS.md Codifies the repository rule that credential-bearing HTTP clients must not use a nil CheckRedirect and must refuse cross-origin redirects.

Review effort: Balanced
Copilot was unable to complete this review at the selected tier due to model policy or plan restrictions.

Comment thread internal/fsregion/client.go Resolved
Comment thread internal/fsregion/client.go Resolved
@srstack
srstack force-pushed the fix/fsregion-redirect-and-base-url branch from e049c6d to 07bc328 Compare November 18, 2026 08:14
@ti-chi-bot ti-chi-bot Bot added size/XL and removed size/L labels Nov 18, 2026

@mornyx mornyx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

This PR closes the cross-origin credential-leakage vector on the Digest-authenticated Filesystem region catalog client by extracting the same-origin redirect policy into internal/api/transport, composing caller-supplied policies through SameOriginThen across both internal/fsregion and internal/api, and validating the catalog origin before any round trip. Both items raised earlier on the thread—caller CheckRedirect widening and folded endpoint-validation messages—are cleanly resolved and verified on 07bc328. Under a strict maintainability and security bar, there are no merge blockers, though two non-blocking follow-ups around shared origin validation (including secret redaction on rejected URLs) and broadening the repo-wide credential-redirect invariant to cover Basic-auth SQL HTTP requests are worth addressing.

What looks good

  • Extracting SameOriginRedirect, SameOriginThen, SameOrigin, MaxRedirects, and ErrTooManyRedirects into internal/api/transport picks the clean package boundary shared by internal/api and internal/fsregion without creating import cycles.
  • SameOriginThen enforces the origin check and redirect cap before delegating to any caller-supplied CheckRedirect, closing the caller-client widening seam in both internal/fsregion and internal/api without mutating caller-owned *http.Client values.
  • Cross-origin redirect refusals surface a dedicated api.fs_regions_redirect_refused code naming only scheme://host from the target Location, and tests verify zero outbound requests hit the second origin under a Digest 401 challenge.

Should fix (non-blocking, strong)

1. Deduplicate fsregion origin validation and avoid echoing embedded secrets in rejected endpoint errors
Client.Fetch (internal/fsregion/client.go:77-90) duplicates the secure-origin validation already in NormalizeServiceAddress (internal/fsregion/fsregion.go:156-166) inside the same package with inverted check ordering (User/RawQuery/Fragment before Scheme in client.go, vice versa in fsregion.go). More importantly, right after detecting parsedBase.User != nil || parsedBase.RawQuery != "" || parsedBase.Fragment != "" (internal/fsregion/client.go:81-83), %q formats the raw baseURL into the returned apperr.Error, which prints embedded userinfo passwords or query tokens in cleartext to stderr (for example, "https://user:pass@fs.tidbapi.com" in TestFetchRejectsInsecureCatalogEndpoint includes "pass" in the error message). Both copies also check parsed.Host == "" rather than parsed.Hostname() == "" (allowing port-only inputs like "https://:8443") and miss parsed.ForceQuery (trailing ?).

  • Fix: Extract one unexported origin-validation helper in internal/fsregion shared by Client.Fetch and NormalizeServiceAddress that enforces a single check order, requires parsed.Hostname() != "" and !parsed.ForceQuery, and strips User/RawQuery/Fragment (plus applies apitransport.Redactor on unparseable inputs) before formatting the rejected endpoint into an error message.

2. Broaden the CheckRedirect invariant beyond Digest/Bearer and align internal/db/sqlhttp
AGENTS.md:1071 states that every credential-bearing client must use transport.SameOriginRedirect, whereas AGENTS.md:1080-1081 narrows the prohibition to clients carrying "Digest or Bearer credentials" (mirroring the assumption that net/http strips header-based Authorization on cross-origin redirects). In Go's net/http (shouldCopyHeaderOnRedirect), plain Authorization headers such as req.SetBasicAuth are still copied across https -> http scheme downgrades on the same host, port changes on the same hostname, and redirects to subdomains, while custom credential headers (such as X-TiDBCloud-Private-Key) are copied across all hosts. Currently, internal/db/sqlhttp/sqlhttp.go:41-44 still constructs &http.Client{Timeout: 30 * time.Second} with a nil CheckRedirect (and leaves opts.HTTPClient.CheckRedirect unwrapped) when sending Basic-authenticated SQL requests.

  • Fix: Broaden the rule in AGENTS.md to cover all credential-bearing HTTP clients (Digest, Bearer, Basic, and custom credential headers), and wire apitransport.SameOriginThen into internal/db/sqlhttp here or in an immediate follow-up PR.

Nits

  • internal/api/client.go:29,280 & internal/fsregion/client.go:152-161: var sameOriginRedirect = apitransport.SameOriginRedirect in internal/api/client.go is now dead production code (only referenced by TestSameOriginRedirectRules in internal/api/redirect_test.go:169, which duplicates transport_test.go), and fsregion.Client.httpClient() allocates a default &http.Client that is immediately discarded when c.HTTPClient != nil. Since apitransport.SameOriginThen(nil) returns SameOriginRedirect, httpClient() can copy-and-configure a single http.Client value just like internal/api.New.
  • internal/api/transport/transport.go:27,56: Now that SameOriginRedirect and SameOrigin are exported functions, guarding req == nil || req.URL == nil || len(via) == 0 || via[0] == nil (returning http.ErrUseLastResponse) and a != nil && b != nil && a.Scheme != "" && a.Host != "" && strings.EqualFold(a.Scheme, b.Scheme) avoids nil/empty-slice panics on direct callers (note internal/fsregion/client.go:178 checks response.Request != nil without checking response.Request.URL != nil).
  • README.md:229: The ti fs list-regions user documentation mentions internal Go CheckRedirect caller-policy composition and loopback http test servers; keeping user-visible redirect refusal (api.fs_regions_redirect_refused) in README.md while leaving Go test-seam mechanics in AGENTS.md would keep the CLI README focused on user-facing behavior.

Recommendation

Comment. No merge-blocking defects remain on current HEAD, with two non-blocking maintainability and security-hardening improvements recommended above.

@srstack
srstack force-pushed the fix/fsregion-redirect-and-base-url branch from 07bc328 to de48d2d Compare November 18, 2026 09:16
@srstack

srstack commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

Addressed mornyx's review on head de48d2d.

Should-fix 1 — shared origin validation, and no secrets in rejected-endpoint errors. Client.Fetch and NormalizeServiceAddress now share one unexported validator, parseSecureOrigin, with a single check order: parse → Hostname() != "" (so port-only inputs like https://:8443 are rejected) → no userinfo/query/ForceQuery/fragment → https or loopback http. Rejected URLs are formatted through describeRejectedOrigin, which strips userinfo, query, and fragment before quoting, and parse failures never echo the raw string, so https://user:pass@… can no longer print pass to stderr. TestRejectedOriginsNeverEchoSecrets pins that for both the catalog endpoint and the service address, and TestFetchRejectsInsecureCatalogEndpoint/TestNormalizeServiceAddress gained the https://:8443 and trailing-? cases.

Should-fix 2 — the invariant now covers every credential-bearing client, and sqlhttp is wired. AGENTS.md now says Digest, Bearer, Basic, and custom credential headers alike, naming the net/http behaviours you cited (same-host port changes, subdomain redirects, https→http downgrades) and the db.sql_http_* codes. internal/db/sqlhttp composes transport.SameOriginThen under its caller client, reports a refused redirect as db.sql_http_redirect_refused (naming both origins), and maps the same-origin loop cap to db.sql_http_redirect_loop. New tests: TestExecuteHTTPRefusesCrossOriginRedirect (attacker origin receives zero requests), the permissive-caller-policy variant, TestExecuteHTTPFollowsSameOriginRedirect, and TestExecuteHTTPReportsRedirectLoop. Mutation-checked: removing the composition makes the attacker receive the Basic-auth request, and the loop cap disappears.

Nits.

  • The dead sameOriginRedirect var (and the duplicated TestSameOriginRedirectRules) is gone from internal/api; transport coverage owns it.
  • fsregion.Client.httpClient() now copy-configures a single *http.Client instead of allocating a default it may discard.
  • SameOriginRedirect/SameOrigin are nil-safe for direct callers (nil request/URL, empty via), and the catalog redirect diagnostic guards response.Request.URL.
  • README.md keeps only the user-visible behaviour (refusal code, https endpoint, no credentials/query/fragment); the Go composition mechanics live in AGENTS.md.

Verification at de48d2d: gofmt clean, go vet ./... on linux/GOOS=windows/GOOS=darwin, go test -count=1 ./..., make e2e. CI is running on the new head.

@ti-chi-bot ti-chi-bot Bot added size/XXL and removed size/XL labels Nov 18, 2026

@mornyx mornyx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

Re-reviewing de48d2d confirms that all items from the previous round—shared parseSecureOrigin validation, credential-bearing coverage for internal/db/sqlhttp and AGENTS.md, SameOrigin/SameOriginRedirect nil safety, dead-code cleanup in internal/api, and README.md trimming—landed cleanly without regressions or caller http.Client mutation. Under a strict maintainability and security bar, there are no merge blockers; two non-blocking edge cases in the newly added sqlhttp redirect classification/logging and describeRejectedOrigin's field-denylist sanitization are worth tightening.

What looks good

  • Extending transport.SameOriginThen to internal/db/sqlhttp closes net/http's Authorization forwarding across https -> http downgrades, port changes, and subdomain hops for Basic-authenticated SQL requests, backed by zero-attacker-hit tests.
  • parseSecureOrigin unifies Client.Fetch and NormalizeServiceAddress under a single validation path covering port-only hosts (:8443), ForceQuery (trailing ?), and standard scheme://user:pass@host?query#frag redaction.
  • Nil and empty-via guards on transport.SameOrigin and transport.SameOriginRedirect fail closed cleanly for direct callers.

Should fix (non-blocking, strong)

1. Guard sqlhttp 3xx refusal on cross-origin Location (shared helper) and record db.sql_http_redirect_loop in oplog
Unlike internal/api/client.go and internal/fsregion/client.go, which only classify a 3xx as redirect_refused when res.Location() resolves to a different origin, sqlhttp.Execute (internal/db/sqlhttp/sqlhttp.go:92-96) maps every 300 <= res.StatusCode < 400 response to db.sql_http_redirect_refused. If a caller supplies a stricter CheckRedirect (such as refusing all redirects, explicitly supported by SameOriginThen) and receives a same-origin 302, refusedSQLRedirectError emits the self-contradictory message "refused a SQL HTTPS API redirect from <origin> to <origin>: ti follows redirects only within the same scheme and host", and Location-less 3xx responses (e.g. 304) are misclassified instead of falling through to statusError(res). In addition, recordSQLHTTPEvent (internal/db/sqlhttp/sqlhttp.go:84,141-144) always logs ErrorCode: "db.sql_http_network" to oplog even when Execute returns db.sql_http_redirect_loop.

  • Fix: Extract a small helper in internal/api/transport (e.g. CrossOriginRedirect(res *http.Response, fallbackReq *http.Request) (fromOrigin, toOrigin string, ok bool)) shared across internal/api, internal/fsregion, and internal/db/sqlhttp so same-origin caller refusals and Location-less 3xx responses fall through to normal status handling without three-way duplication, and set event.ErrorCode = "db.sql_http_redirect_loop" in recordSQLHTTPEvent when errors.Is(requestErr, transport.ErrTooManyRedirects).

2. Use a scheme://host allowlist in describeRejectedOrigin so Opaque / Path inputs cannot echo secrets
describeRejectedOrigin (internal/fsregion/fsregion.go:199-213) clears a field denylist (User, RawQuery, ForceQuery, Fragment) before calling copied.String(). For malformed inputs without //—such as "https:user:secret@fs.tidbapi.com" (Opaque = "user:secret@..."), "user:secret@fs.tidbapi.com", or a raw token accidentally pasted in place of a URL (Path = "ti_fs_...")—url.Parse leaves User == nil and Hostname() == "", so parseSecureOrigin returns (parsed, errOriginNeedsHost) and copied.String() still echoes Opaque or Path verbatim to stderr. Relatedly, parseSecureOrigin (internal/fsregion/fsregion.go:177) checks parsed.Hostname() == "" without parsed.Scheme == "", causing scheme-relative "//fs.tidbapi.com" to report errOriginInsecureScheme instead of errOriginNeedsHost.

  • Fix: Check if parsed.Scheme == "" || parsed.Hostname() == "" for errOriginNeedsHost in parseSecureOrigin, and replace the four field resets in describeRejectedOrigin with an explicit scheme://host allowlist: format (&url.URL{Scheme: parsed.Scheme, Host: parsed.Host}).String() only when parsed != nil && parsed.Opaque == "" && parsed.Scheme != "" && parsed.Hostname() != "" (leaving display = "" otherwise), and add an opaque-URL case to TestRejectedOriginsNeverEchoSecrets.

Nits

  • internal/api/transport/transport_test.go:132 & internal/db/sqlhttp/sqlhttp_test.go:219: When TestSameOriginRedirectRules was consolidated from internal/api/redirect_test.go, the same-host port-mismatch case (:8443) and direct nil/empty-via assertions were not carried over to transport_test.go; and TestExecuteHTTPFollowsSameOriginRedirect uses 302 Found (which converts POST to a bodyless GET) rather than 307 Temporary Redirect asserting POST, JSON body, and BasicAuth preservation on /moved.

Recommendation

Comment. All prior findings and cross-origin credential-redirect protections are verified on de48d2d with no merge blockers; two non-blocking edge-case cleanups in the newly added helpers are recommended above.

@srstack
srstack force-pushed the fix/fsregion-redirect-and-base-url branch from de48d2d to 4968670 Compare November 18, 2026 09:48
@srstack

srstack commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

Second round addressed on head 4968670.

Should-fix 1 — shared cross-origin classifier, and the oplog no longer contradicts the error. Added transport.CrossOriginRedirect(res, fallbackReq) which reduces both sides to scheme://host (via transport.Origin) and returns false for a non-3xx, a Location-less response, and a same-origin redirect. All three clients now classify through it:

  • internal/db/sqlhttp only reports db.sql_http_redirect_refused for an actual cross-origin Location; a same-origin 302 refused by a stricter caller policy (or a 304 with no Location) falls through to the normal status handling instead of the self-contradictory "from X to X" message.
  • internal/api (api.redirect_refused) and internal/fsregion (api.fs_regions_redirect_refused) use the same helper and the same origin formatting.
  • recordSQLHTTPEvent now records db.sql_http_redirect_loop when errors.Is(requestErr, transport.ErrTooManyRedirects), so the operation log matches the command error.

Tests: TestCrossOriginRedirectClassification (cross-origin, same-origin, 304/no-Location/non-3xx, origins reduced to scheme://host), TestExecuteHTTPSameOriginRedirectRefusedByCallerFallsThrough (db.sql_execution_failed, not a redirect refusal), TestExecuteHTTPRecordsRedirectLoopEvent. Mutation-checked: classifying every 3xx and dropping the oplog branch both reproduce exactly the two defects you described.

Should-fix 2 — scheme://host allowlist for rejected origins. describeRejectedOrigin now formats only (&url.URL{Scheme: parsed.Scheme, Host: parsed.Host}), and only when parsed.Opaque == "" && Scheme != "" && Hostname() != ""; every other input is described without the value. parseSecureOrigin now requires Scheme != "" || Hostname() != "" for errOriginNeedsHost, so //fs.tidbapi.com reports a missing scheme/host rather than a scheme problem. TestRejectedOriginsNeverEchoSecrets gained the opaque (https:user:<password>@host), bare user:<password>@host, and pasted-token-as-path cases; TestFetchRejectsInsecureCatalogEndpoint covers //fs.tidbapi.com.

Nits. TestSameOriginRedirectKeepsCredentialsOnTheSameOrigin regained the same-host port-mismatch case, and TestSameOriginRedirectFailsClosedOnMissingInput pins the nil request / empty via / no-origin behaviour. TestExecuteHTTPFollowsSameOriginRedirect now uses 307 Temporary Redirect and asserts the redirected request is still the authenticated POST with the SQL body.

Verification at 4968670: gofmt clean, go vet ./... on linux/GOOS=windows/GOOS=darwin, go test -count=1 ./..., make e2e. CI is running on the new head.

@mornyx mornyx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

Re-reviewing 4968670 confirms that all items from the previous round—shared transport.CrossOriginRedirect wiring across all three clients, db.sql_http_redirect_loop operation-log consistency, the scheme://host allowlist in describeRejectedOrigin, //host handling in parseSecureOrigin, and the expanded 307/port-mismatch/opaque-URL test cases—landed cleanly with no merge blockers. Under a strict correctness and maintainability bar, one non-blocking logic bug remains in the newly exported transport.CrossOriginRedirect helper around relative Location resolution on its fallbackReq path and originless redirect targets.

What looks good

  • Consolidating redirect classification behind transport.CrossOriginRedirect removes the three-way duplication across internal/api, internal/fsregion, and internal/db/sqlhttp while keeping caller-refused same-origin redirects on their normal status path.
  • describeRejectedOrigin now uses a strict scheme://host allowlist (Opaque == "" && Scheme != "" && Hostname() != ""), verified against opaque URLs and pasted-token inputs in TestRejectedOriginsNeverEchoSecrets.
  • TestExecuteHTTPFollowsSameOriginRedirect (307 preserving POST, body, and BasicAuth) and TestExecuteHTTPRecordsRedirectLoopEvent pin both the wire behavior and oplog error-code contract.

Should fix (non-blocking, strong)

1. Resolve Location against req.URL and reject empty origins in transport.CrossOriginRedirect
CrossOriginRedirect (internal/api/transport/transport.go:63-79) accepts fallbackReq for responses without res.Request, sets req = fallbackReq when res.Request == nil, and then calls res.Location() (internal/api/transport/transport.go:74). In Go's net/http, (*http.Response).Location() resolves relative Location headers only against res.Request.URL and falls back to bare url.Parse when res.Request == nil. As a result, whenever fallbackReq is exercised (custom RoundTrippers or direct callers), a relative same-origin redirect such as Location: /other parses without scheme or host, fails SameOrigin, and is misclassified as a cross-origin redirect with toOrigin == "" (ok == true), while Location: //evil.example/other also yields toOrigin == "". In addition, even when res.Request != nil, an originless Location such as about:blank, https:///moved, or https://:8443/moved causes CrossOriginRedirect to return ok == true with an empty (or port-only) toOrigin, producing "refused a ... redirect from https://... to : ...".

  • Fix: Fall back to fallbackReq whenever res.Request == nil || res.Request.URL == nil, resolve res.Header.Get("Location") directly via req.URL.Parse(location), require u.Opaque == "" && u.Scheme != "" && u.Hostname() != "" in Origin (and SameOrigin), and return ("", "", false) if fromOrigin == "" || toOrigin == "". Add relative Location: "/other" / "//evil.example/other" (with res.Request == nil and non-nil fallbackReq) and originless Location cases to TestCrossOriginRedirectClassification.

Nits

  • internal/fsregion/client.go:174 & internal/fsregion/fsregion.go:200: Passing request into readCatalogPage so CrossOriginRedirect(response, request) receives the fallback request matches internal/api and internal/db/sqlhttp; and once transport.Origin checks Opaque == "" && Hostname() != "", describeRejectedOrigin can call apitransport.Origin(parsed) directly instead of repeating the allowlist condition.

Recommendation

Comment. All credential-redirect protections and prior round fixes hold on 4968670 with no merge blockers; tightening transport.CrossOriginRedirect for relative fallbackReq and originless Location headers is recommended above.

…TTP clients

`fsregion.Client.httpClient()` built `&http.Client{Transport, Timeout}` with a nil
`CheckRedirect`, so net/http followed up to ten redirects to any scheme and any
host. That request is Digest-authenticated with the organization API key pair, and
the digest transport answers whatever 401 challenge it receives, so a redirect to
an attacker-chosen origin made ti compute and send a Digest response for that
host - the exact credential exposure this hardening closes. The mutation check in
the new test shows it: with the policy removed, the second origin's 401 challenge
is answered (the fake attacker returns `auth.invalid_credentials`), and with the
policy in place the second origin receives no request at all.

- Move the existing same-origin redirect policy out of `internal/api` into
  `internal/api/transport` (`SameOriginRedirect`, `SameOrigin`, `SameOriginThen`,
  `MaxRedirects`, `ErrTooManyRedirects`, nil-safe for direct callers) so the
  credential-bearing clients cannot drift.
- A caller-supplied `CheckRedirect` is composed through `SameOriginThen`: it runs
  after the mandatory same-origin rule, so a caller can narrow redirects (refuse
  all of them) but can never re-enable a cross-origin hop. `internal/api` and
  `internal/fsregion` both use it, closing the seam on each client.
- Extend the rule to the Basic-authenticated SQL HTTPS transport
  (`internal/db/sqlhttp`): net/http still copies `Authorization` across same-host
  port changes, subdomain redirects, and https->http downgrades.
- Classify redirects through one shared helper (`transport.CrossOriginRedirect`,
  which resolves the Location header against the request URL - including when the
  response carries no request - reduces both sides to `scheme://host`, and refuses
  to report an originless target): a same-origin redirect refused by a stricter
  caller policy falls through to normal status handling instead of being reported
  as an origin change that never happened. A refused cross-origin
  redirect reports `db.sql_http_redirect_refused`; a same-origin loop reports
  `db.sql_http_redirect_loop`, and the operation log records that same code
  instead of contradicting the command error with `db.sql_http_network`.
- A refused catalog redirect reports `api.fs_regions_redirect_refused` naming both
  origins instead of a generic catalog failure.
- Share one origin validator (`parseSecureOrigin`) between the catalog endpoint and
  published service addresses: absolute, host name required, no embedded
  credentials, query (including a bare "?"), or fragment, and https except for a
  loopback test server. Rejected origins are described through
  `describeRejectedOrigin`, which formats an explicit `scheme://host` allowlist, so
  neither a password in userinfo/query nor a secret carried by an Opaque or Path
  value can reach stderr. A scheme-relative `//host` input is reported as missing
  a scheme and host.

Tests: transport `TestSameOriginRedirectKeepsCredentialsOnTheSameOrigin` and
`TestSameOriginThenKeepsCallerRestrictionsWithoutWidening`; fsregion
`TestFetchRefusesCrossOriginRedirect` (zero requests at the second origin),
`TestFetchAppliesRedirectPolicyToCallerClient`,
`TestFetchEnforcesSameOriginUnderCallerRedirectPolicy`,
`TestFetchHonorsStricterCallerRedirectPolicy`,
`TestFetchFollowsSameOriginRedirect`, `TestFetchRejectsInsecureCatalogEndpoint`
(per-case messages), and `TestRejectedOriginsNeverEchoSecrets`; sqlhttp
`TestExecuteHTTPRefusesCrossOriginRedirect` (plus the caller-policy variant),
`TestExecuteHTTPFollowsSameOriginRedirect`, and
`TestExecuteHTTPReportsRedirectLoop`. internal/api pins the no-widening rule on
its caller-client seam.

Verification: gofmt, `go vet ./...` on linux/darwin/windows, `go test -count=1
./...`, `make e2e`.
@srstack
srstack force-pushed the fix/fsregion-redirect-and-base-url branch from 4968670 to 3e8a42f Compare November 18, 2026 10:19
@srstack

srstack commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

Third round addressed on head 3e8a42f.

transport.CrossOriginRedirect now resolves Location correctly and never reports an originless target.

  • The request fallback is used whenever res.Request == nil || res.Request.URL == nil, and the header is resolved with req.URL.Parse(location) instead of res.Location(). http.Response.Location only resolves relative values against res.Request, so the fallback path used to turn Location: /other into a bogus cross-origin redirect and drop the scheme from //evil.example/other.
  • Origin now requires Opaque == "" && Scheme != "" && Hostname() != "" (port-only hosts are rejected), SameOrigin applies the same origin requirement, and CrossOriginRedirect returns false when either side reduces to an empty origin, so about:blank, https:///moved, and https://:8443/moved can no longer produce "... to : ...".
  • Tests: TestCrossOriginRedirectClassification gained the no-res.Request matrix — relative same-origin (/other → not cross-origin), scheme-relative other host (//evil.example/other → https://evil.example), opaque target, empty-host target, port-only target, missing Location, and a response with neither request nor fallback. Mutation-checked: restoring res.Location() reproduces the exact misclassification (from="https://fs.tidbapi.com" to="").

Nits. fsregion now passes the request into readCatalogPage so CrossOriginRedirect(response, request) gets the fallback (matching internal/api and sqlhttp), and describeRejectedOrigin calls apitransport.Origin(parsed) instead of repeating the allowlist condition.

Verification at 3e8a42f: gofmt clean, go vet ./... on linux/GOOS=windows/GOOS=darwin, go test -count=1 ./..., make e2e. CI is running on the new head.

@mornyx mornyx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

Re-reviewing 3e8a42f confirms that every item from the previous rounds is cleanly resolved: transport.CrossOriginRedirect now falls back whenever res.Request == nil || res.Request.URL == nil, resolves Location directly against req.URL, and rejects originless targets via Origin and SameOrigin, while fsregion passes the request fallback into readCatalogPage and reuses transport.Origin directly in describeRejectedOrigin. Across internal/api/transport, internal/api, internal/fsregion, and internal/db/sqlhttp, credential-bearing redirect enforcement, cross-origin classification, and redacted endpoint validation sit cleanly in their canonical layers with no remaining correctness, security, or maintainability issues.

What looks good

  • internal/api/transport provides one cohesive, non-widening boundary (SameOriginRedirect, SameOriginThen, CrossOriginRedirect, Origin, SameOrigin) shared across Digest, Bearer, and Basic-auth clients without mutating caller *http.Client instances.
  • parseSecureOrigin and describeRejectedOrigin (backed by transport.Origin) enforce identical HTTPS/loopback rules across catalog endpoints and published service addresses while ensuring malformed, opaque, or credential-bearing inputs never echo secrets to stderr.
  • Behavioral and negative tests cover zero-attacker-hit guarantees across all three clients, 307 POST body and BasicAuth preservation, stricter caller-policy fallthrough, oplog loop error-code alignment, and relative/originless Location matrices.

Recommendation

Approve. All credential-redirect protections, error-redaction guarantees, and structural cleanups are verified on 3e8a42f with no remaining issues.

@srstack
srstack merged commit 98aca0a into main Nov 18, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants