From 3e8a42ffe8d1b20ea129f9365cfb45f1f696bcbf Mon Sep 17 00:00:00 2001 From: srstack Date: Wed, 18 Nov 2026 16:05:37 +0800 Subject: [PATCH] fix(security): refuse cross-origin redirects for credential-bearing HTTP clients `fsregion.Client.httpClient()` built `&http.Client{Transport, Timeout}` with a nil `CheckRedirect`, so net/http followed up to ten redirects to any scheme and any host. That request is Digest-authenticated with the organization API key pair, and the digest transport answers whatever 401 challenge it receives, so a redirect to an attacker-chosen origin made ti compute and send a Digest response for that host - the exact credential exposure this hardening closes. The mutation check in the new test shows it: with the policy removed, the second origin's 401 challenge is answered (the fake attacker returns `auth.invalid_credentials`), and with the policy in place the second origin receives no request at all. - Move the existing same-origin redirect policy out of `internal/api` into `internal/api/transport` (`SameOriginRedirect`, `SameOrigin`, `SameOriginThen`, `MaxRedirects`, `ErrTooManyRedirects`, nil-safe for direct callers) so the credential-bearing clients cannot drift. - A caller-supplied `CheckRedirect` is composed through `SameOriginThen`: it runs after the mandatory same-origin rule, so a caller can narrow redirects (refuse all of them) but can never re-enable a cross-origin hop. `internal/api` and `internal/fsregion` both use it, closing the seam on each client. - Extend the rule to the Basic-authenticated SQL HTTPS transport (`internal/db/sqlhttp`): net/http still copies `Authorization` across same-host port changes, subdomain redirects, and https->http downgrades. - Classify redirects through one shared helper (`transport.CrossOriginRedirect`, which resolves the Location header against the request URL - including when the response carries no request - reduces both sides to `scheme://host`, and refuses to report an originless target): a same-origin redirect refused by a stricter caller policy falls through to normal status handling instead of being reported as an origin change that never happened. A refused cross-origin redirect reports `db.sql_http_redirect_refused`; a same-origin loop reports `db.sql_http_redirect_loop`, and the operation log records that same code instead of contradicting the command error with `db.sql_http_network`. - A refused catalog redirect reports `api.fs_regions_redirect_refused` naming both origins instead of a generic catalog failure. - Share one origin validator (`parseSecureOrigin`) between the catalog endpoint and published service addresses: absolute, host name required, no embedded credentials, query (including a bare "?"), or fragment, and https except for a loopback test server. Rejected origins are described through `describeRejectedOrigin`, which formats an explicit `scheme://host` allowlist, so neither a password in userinfo/query nor a secret carried by an Opaque or Path value can reach stderr. A scheme-relative `//host` input is reported as missing a scheme and host. Tests: transport `TestSameOriginRedirectKeepsCredentialsOnTheSameOrigin` and `TestSameOriginThenKeepsCallerRestrictionsWithoutWidening`; fsregion `TestFetchRefusesCrossOriginRedirect` (zero requests at the second origin), `TestFetchAppliesRedirectPolicyToCallerClient`, `TestFetchEnforcesSameOriginUnderCallerRedirectPolicy`, `TestFetchHonorsStricterCallerRedirectPolicy`, `TestFetchFollowsSameOriginRedirect`, `TestFetchRejectsInsecureCatalogEndpoint` (per-case messages), and `TestRejectedOriginsNeverEchoSecrets`; sqlhttp `TestExecuteHTTPRefusesCrossOriginRedirect` (plus the caller-policy variant), `TestExecuteHTTPFollowsSameOriginRedirect`, and `TestExecuteHTTPReportsRedirectLoop`. internal/api pins the no-widening rule on its caller-client seam. Verification: gofmt, `go vet ./...` on linux/darwin/windows, `go test -count=1 ./...`, `make e2e`. --- AGENTS.md | 19 +++ README.md | 2 +- internal/api/client.go | 39 ++--- internal/api/redirect_test.go | 33 ++-- internal/api/transport/transport.go | 103 ++++++++++++ internal/api/transport/transport_test.go | 153 +++++++++++++++++ internal/db/sqlhttp/sqlhttp.go | 23 +++ internal/db/sqlhttp/sqlhttp_test.go | 160 ++++++++++++++++++ internal/fsregion/client.go | 47 ++++-- internal/fsregion/fsregion.go | 56 +++++- internal/fsregion/fsregion_test.go | 206 +++++++++++++++++++++++ 11 files changed, 770 insertions(+), 71 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index b4211e2f..9f5d70be 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1066,6 +1066,25 @@ credential when ti learns about the File System (create/import) and refreshed by `ti fs list-regions`. Region-to-address resolution for data-plane traffic must never consult the network. +Every HTTP client that carries a credential - Digest, Bearer, Basic auth, or a +custom credential header - must use the shared `transport.SameOriginRedirect` +policy, and a caller-supplied `CheckRedirect` is composed through +`transport.SameOriginThen`, so a caller may narrow redirects (refuse all of them, +for example) but can never re-enable a cross-origin hop. The Filesystem region +catalog spends the organization API key pair through +`internal/api/transport.NewDigest`, and that transport answers any 401 challenge +it receives; the SQL HTTPS API spends the cluster's SQL user password through +Basic auth, and net/http still copies `Authorization` across same-host port +changes, subdomain redirects, and https->http downgrades. A refused redirect is +reported instead of followed: `api.fs_regions_redirect_refused` (fsregion), +`api.redirect_refused` / `api.redirect_loop` (internal/api), and +`db.sql_http_redirect_refused` / `db.sql_http_redirect_loop` (sqlhttp). The +catalog endpoint itself must be `https`, or plain `http` only on a loopback host, +without embedded credentials, a query, or a fragment, and a rejected endpoint is +described through `describeRejectedOrigin` so a password or token in the URL +never reaches stderr. Never build a credential-bearing client with a nil +`CheckRedirect`. + Do not store secrets in logs, telemetry, generated docs examples, or test fixtures. diff --git a/README.md b/README.md index aa7262ad..63fd9a53 100644 --- a/README.md +++ b/README.md @@ -226,7 +226,7 @@ ti fs list-layer-chain --layer-ref experiment-a --output text Use `ti fs delete-layer --layer-ref experiment-a` to abandon a rejected leaf. Deleting a layer with live descendants fails unless `--cascade` is supplied explicitly. Layer names are not durable unique identifiers because abandoned layers remain visible; automation should capture returned layer IDs and use run-unique checkpoint IDs. -`ti fs list-regions` lists the Filesystem regions published by TiDB Cloud Filesystem, with their cloud provider, display name, state, and regional service address. It uses TiDB Cloud credentials, always asks the API, and refreshes the recorded `service_address` of locally stored File System credentials: +`ti fs list-regions` lists the Filesystem regions published by TiDB Cloud Filesystem, with their cloud provider, display name, state, and regional service address. It uses TiDB Cloud credentials, always asks the API, and refreshes the recorded `service_address` of locally stored File System credentials. The catalog request is Digest-authenticated with the organization API key pair, so `ti` follows a redirect only when it keeps the same scheme and host and otherwise fails with `api.fs_regions_redirect_refused`; the catalog endpoint itself must be `https` (plain `http` is accepted only for a loopback test server) and must not embed credentials, a query, or a fragment: ```shell ti fs list-regions diff --git a/internal/api/client.go b/internal/api/client.go index b9bd7be0..dc407a04 100644 --- a/internal/api/client.go +++ b/internal/api/client.go @@ -26,7 +26,7 @@ const defaultMaxRetries = 2 // maxRedirects matches net/http's default limit, which a custom CheckRedirect // replaces. -const maxRedirects = 10 +const maxRedirects = apitransport.MaxRedirects type Client struct { BaseURL *url.URL @@ -84,11 +84,12 @@ func New(opts Options) (*Client, error) { Timeout: opts.Timeout, } } - if httpClient.CheckRedirect == nil { - copied := *httpClient - copied.CheckRedirect = sameOriginRedirect - httpClient = &copied - } + // The credential-bearing transports must never follow a cross-origin redirect, + // so the mandatory rule runs under any caller-supplied policy: a caller can + // narrow redirects, never widen them. + copiedClient := *httpClient + copiedClient.CheckRedirect = apitransport.SameOriginThen(httpClient.CheckRedirect) + httpClient = &copiedClient maxRetries := opts.MaxRetries if maxRetries == 0 { @@ -275,26 +276,10 @@ func (c *Client) do(req *http.Request) (*http.Response, error) { return nil, lastErr } -// sameOriginRedirect follows a redirect only when it keeps the scheme and host -// of the original request. The digest and bearer transports add credentials on -// every round trip, so following a redirect to another origin would send them -// there. A refused redirect comes back as the 3xx response. -func sameOriginRedirect(req *http.Request, via []*http.Request) error { - if len(via) >= maxRedirects { - return errTooManyRedirects - } - if !sameOrigin(req.URL, via[0].URL) { - return http.ErrUseLastResponse - } - return nil -} - -var errTooManyRedirects = fmt.Errorf("stopped after %d redirects", maxRedirects) - // requestError maps a failed round trip to an API error. A same-origin // redirect loop is reported as such rather than as a network problem. func requestError(req *http.Request, err error) error { - if errors.Is(err, errTooManyRedirects) { + if errors.Is(err, apitransport.ErrTooManyRedirects) { return &Error{ Code: "api.redirect_loop", Category: "api", @@ -312,10 +297,6 @@ func requestError(req *http.Request, err error) error { } } -func sameOrigin(a, b *url.URL) bool { - return a.Scheme == b.Scheme && strings.EqualFold(a.Host, b.Host) -} - func retryable(statusCode int) bool { return statusCode == http.StatusTooManyRequests || statusCode >= 500 } @@ -339,14 +320,14 @@ func (c *Client) statusError(req *http.Request, res *http.Response) error { apiMessage := remoteError.Message requestID := responseRequestID(res.Header) if res.StatusCode >= 300 && res.StatusCode < 400 { - if target, err := res.Location(); err == nil && !sameOrigin(target, req.URL) { + if fromOrigin, toOrigin, crossOrigin := apitransport.CrossOriginRedirect(res, req); crossOrigin { return &Error{ Code: "api.redirect_refused", Category: "api", ExitCode: 1, StatusCode: res.StatusCode, RequestID: requestID, - Message: fmt.Sprintf("refused a redirect from %s://%s to %s://%s: ti follows redirects only within the same scheme and host", req.URL.Scheme, req.URL.Host, target.Scheme, target.Host), + Message: fmt.Sprintf("refused a redirect from %s to %s: ti follows redirects only within the same scheme and host", fromOrigin, toOrigin), Body: string(body), } } diff --git a/internal/api/redirect_test.go b/internal/api/redirect_test.go index b6130f42..48642802 100644 --- a/internal/api/redirect_test.go +++ b/internal/api/redirect_test.go @@ -154,28 +154,6 @@ func TestRefusedDowngradeRedirectNamesBothSchemes(t *testing.T) { } } -func TestSameOriginRedirectRules(t *testing.T) { - via := []*http.Request{{URL: mustParseURL(t, "https://api.example.com/v1/a")}} - for _, tc := range []struct { - target string - allow bool - }{ - {"https://api.example.com/v1/b", true}, - {"https://API.example.com/v1/b", true}, - {"http://api.example.com/v1/b", false}, - {"https://api.example.com:8443/v1/b", false}, - {"https://evil.example/v1/b", false}, - } { - err := sameOriginRedirect(&http.Request{URL: mustParseURL(t, tc.target)}, via) - if tc.allow && err != nil { - t.Fatalf("%s: expected redirect to be followed, got %v", tc.target, err) - } - if !tc.allow && !errors.Is(err, http.ErrUseLastResponse) { - t.Fatalf("%s: expected redirect to be refused, got %v", tc.target, err) - } - } -} - func TestNewKeepsCallerRedirectPolicyAndDoesNotMutateClient(t *testing.T) { shared := &http.Client{} client, err := New(Options{Endpoint: endpoints.Endpoint{BaseURL: "https://api.example.com"}, HTTPClient: shared}) @@ -197,6 +175,17 @@ func TestNewKeepsCallerRedirectPolicyAndDoesNotMutateClient(t *testing.T) { if err := client.HTTPClient.CheckRedirect(&http.Request{URL: mustParseURL(t, "https://api.example.com/x")}, []*http.Request{{URL: mustParseURL(t, "https://api.example.com/")}}); !errors.Is(err, http.ErrUseLastResponse) { t.Fatalf("expected the caller's CheckRedirect to be kept, got %v", err) } + + // A caller policy that allows every redirect must not widen the client past + // the credential-bearing same-origin rule. + permissive := func(*http.Request, []*http.Request) error { return nil } + client, err = New(Options{Endpoint: endpoints.Endpoint{BaseURL: "https://api.example.com"}, HTTPClient: &http.Client{CheckRedirect: permissive}}) + if err != nil { + t.Fatalf("New failed: %v", err) + } + if err := client.HTTPClient.CheckRedirect(&http.Request{URL: mustParseURL(t, "https://evil.example/x")}, []*http.Request{{URL: mustParseURL(t, "https://api.example.com/")}}); !errors.Is(err, http.ErrUseLastResponse) { + t.Fatalf("a permissive caller re-enabled a cross-origin redirect: %v", err) + } } func testBearerClient(t *testing.T, baseURL string) *Client { diff --git a/internal/api/transport/transport.go b/internal/api/transport/transport.go index 6ea5760c..dfd27aee 100644 --- a/internal/api/transport/transport.go +++ b/internal/api/transport/transport.go @@ -4,11 +4,114 @@ import ( "fmt" "io" "net/http" + "net/url" "strings" "github.com/icholy/digest" ) +// MaxRedirects matches net/http's default limit, which a custom CheckRedirect +// replaces. +const MaxRedirects = 10 + +// ErrTooManyRedirects reports that a same-origin redirect chain hit MaxRedirects, +// so a caller can tell a loop from a refused cross-origin redirect. +var ErrTooManyRedirects = fmt.Errorf("stopped after %d redirects", MaxRedirects) + +// SameOriginRedirect is the CheckRedirect policy for every credential-bearing +// client: it follows a redirect only when it keeps the scheme and host of the +// original request, because the digest and bearer transports attach credentials +// on every round trip (the digest transport answers whatever 401 challenge it +// receives, including one from a host the user never chose). A refused redirect +// comes back to the caller as the 3xx response. +func SameOriginRedirect(req *http.Request, via []*http.Request) error { + if len(via) >= MaxRedirects { + return ErrTooManyRedirects + } + if req == nil || req.URL == nil || len(via) == 0 || via[0] == nil || via[0].URL == nil { + return http.ErrUseLastResponse + } + if !SameOrigin(req.URL, via[0].URL) { + return http.ErrUseLastResponse + } + return nil +} + +// SameOriginThen returns a CheckRedirect policy that always applies the same-origin +// rule and then defers to caller. A caller-supplied policy can therefore narrow +// redirects further - for example refuse all of them - but can never re-enable a +// cross-origin hop for a credential-bearing client. A nil caller gets only the +// same-origin rule. +func SameOriginThen(caller func(req *http.Request, via []*http.Request) error) func(req *http.Request, via []*http.Request) error { + if caller == nil { + return SameOriginRedirect + } + return func(req *http.Request, via []*http.Request) error { + if err := SameOriginRedirect(req, via); err != nil { + return err + } + return caller(req, via) + } +} + +// CrossOriginRedirect classifies a 3xx response as a cross-origin redirect and +// returns the two origins as scheme://host, without userinfo, query, or fragment. +// It reports false for a non-redirect, a missing or unparsable Location, and a +// same-origin redirect - a caller that refused a same-origin redirect (for +// example through a stricter caller policy) must fall through to its normal +// status handling instead of reporting an origin change that did not happen. +func CrossOriginRedirect(res *http.Response, fallbackReq *http.Request) (fromOrigin, toOrigin string, ok bool) { + if res == nil || res.StatusCode < 300 || res.StatusCode >= 400 { + return "", "", false + } + req := res.Request + if req == nil || req.URL == nil { + req = fallbackReq + } + if req == nil || req.URL == nil { + return "", "", false + } + // Resolve the header against the request URL directly: http.Response.Location + // only resolves relative values when the response carries its own request, so a + // fallback request would otherwise make "/other" look like another origin and + // "//evil.example/other" lose its scheme. + location := strings.TrimSpace(res.Header.Get("Location")) + if location == "" { + return "", "", false + } + target, err := req.URL.Parse(location) + if err != nil || target == nil || SameOrigin(target, req.URL) { + return "", "", false + } + fromOrigin, toOrigin = Origin(req.URL), Origin(target) + if fromOrigin == "" || toOrigin == "" { + // An originless target cannot name a different origin; never report one. + return "", "", false + } + return fromOrigin, toOrigin, true +} + +// Origin renders scheme://host for a URL, dropping any userinfo, path, query, or +// fragment. It returns an empty string when the URL has no origin. +func Origin(u *url.URL) string { + if u == nil || u.Opaque != "" || u.Scheme == "" || u.Hostname() == "" { + return "" + } + return (&url.URL{Scheme: u.Scheme, Host: u.Host}).String() +} + +// SameOrigin reports whether two URLs share a scheme and host. Host comparison +// folds case because DNS names are case-insensitive. +func SameOrigin(a, b *url.URL) bool { + if a == nil || b == nil || a.Opaque != "" || b.Opaque != "" { + return false + } + if a.Scheme == "" || a.Hostname() == "" || b.Scheme == "" || b.Hostname() == "" { + return false + } + return strings.EqualFold(a.Scheme, b.Scheme) && strings.EqualFold(a.Host, b.Host) +} + func NewDigest(username, password string, base http.RoundTripper) http.RoundTripper { return &digest.Transport{ Username: username, diff --git a/internal/api/transport/transport_test.go b/internal/api/transport/transport_test.go index c8aeb646..7d94d5df 100644 --- a/internal/api/transport/transport_test.go +++ b/internal/api/transport/transport_test.go @@ -2,9 +2,11 @@ package transport import ( "bytes" + "errors" "fmt" "net/http" "net/http/httptest" + "net/url" "strings" "testing" ) @@ -98,3 +100,154 @@ func TestDebugRoundTripperRedactsSecrets(t *testing.T) { t.Fatalf("debug output did not redact secret:\n%s", log.String()) } } + +func TestSameOriginThenKeepsCallerRestrictionsWithoutWidening(t *testing.T) { + mustURL := func(value string) *url.URL { + parsed, err := url.Parse(value) + if err != nil { + t.Fatalf("parse %q: %v", value, err) + } + return parsed + } + same := &http.Request{URL: mustURL("https://fs.tidbapi.com/v1beta2/regions")} + cross := &http.Request{URL: mustURL("https://evil.example/v1beta2/regions")} + via := []*http.Request{{URL: mustURL("https://fs.tidbapi.com/v1beta2/regions")}} + + if SameOriginThen(nil) == nil { + t.Fatal("nil caller returned no policy") + } + permissive := SameOriginThen(func(*http.Request, []*http.Request) error { return nil }) + if err := permissive(cross, via); !errors.Is(err, http.ErrUseLastResponse) { + t.Fatalf("a permissive caller re-enabled a cross-origin redirect: %v", err) + } + if err := permissive(same, via); err != nil { + t.Fatalf("a permissive caller should not be checked twice for a same-origin hop: %v", err) + } + strict := SameOriginThen(func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }) + if err := strict(same, via); !errors.Is(err, http.ErrUseLastResponse) { + t.Fatalf("a stricter caller must still narrow same-origin redirects: %v", err) + } +} + +func TestSameOriginRedirectKeepsCredentialsOnTheSameOrigin(t *testing.T) { + mustURL := func(value string) *url.URL { + parsed, err := url.Parse(value) + if err != nil { + t.Fatalf("parse %q: %v", value, err) + } + return parsed + } + via := []*http.Request{{URL: mustURL("https://fs.tidbapi.com/v1beta2/regions")}} + for _, tc := range []struct { + name string + target string + wantErr error + }{ + {"same-origin path", "https://fs.tidbapi.com/v1beta2/regions?pageToken=next", nil}, + {"case-folded host", "https://FS.TiDBAPI.com/other", nil}, + {"other host", "https://evil.example/v1beta2/regions", http.ErrUseLastResponse}, + {"other scheme", "http://fs.tidbapi.com/v1beta2/regions", http.ErrUseLastResponse}, + {"other port", "https://fs.tidbapi.com:8443/v1beta2/regions", http.ErrUseLastResponse}, + } { + t.Run(tc.name, func(t *testing.T) { + err := SameOriginRedirect(&http.Request{URL: mustURL(tc.target)}, via) + if !errors.Is(err, tc.wantErr) && err != tc.wantErr { + t.Fatalf("SameOriginRedirect(%q) = %v, want %v", tc.target, err, tc.wantErr) + } + }) + } + + chain := make([]*http.Request, MaxRedirects) + for i := range chain { + chain[i] = &http.Request{URL: mustURL("https://fs.tidbapi.com/v1beta2/regions")} + } + if err := SameOriginRedirect(chain[0], chain); !errors.Is(err, ErrTooManyRedirects) { + t.Fatalf("redirect limit error = %v, want ErrTooManyRedirects", err) + } +} + +func TestSameOriginRedirectFailsClosedOnMissingInput(t *testing.T) { + if err := SameOriginRedirect(nil, nil); !errors.Is(err, http.ErrUseLastResponse) { + t.Fatalf("nil request error = %v, want http.ErrUseLastResponse", err) + } + if err := SameOriginRedirect(&http.Request{}, nil); !errors.Is(err, http.ErrUseLastResponse) { + t.Fatalf("URL-less request error = %v, want http.ErrUseLastResponse", err) + } + if SameOrigin(nil, nil) || SameOrigin(&url.URL{Scheme: "https"}, &url.URL{Scheme: "https"}) { + t.Fatal("SameOrigin must fail closed for URLs without an origin") + } +} + +func TestCrossOriginRedirectClassification(t *testing.T) { + mustURL := func(value string) *url.URL { + parsed, err := url.Parse(value) + if err != nil { + t.Fatalf("parse %q: %v", value, err) + } + return parsed + } + origin := mustURL("https://fs.tidbapi.com/v1beta2/regions") + redirect := func(status int, location string, req *http.Request) *http.Response { + res := &http.Response{StatusCode: status, Header: http.Header{}, Request: req} + if location != "" { + res.Header.Set("Location", location) + } + return res + } + + if _, _, ok := CrossOriginRedirect(redirect(http.StatusFound, "https://evil.example/v1beta2/regions", nil), &http.Request{URL: origin}); !ok { + t.Fatal("a cross-origin redirect was not classified") + } + from, to, ok := CrossOriginRedirect(redirect(http.StatusFound, "https://fs.tidbapi.com/other", nil), &http.Request{URL: origin}) + if ok { + t.Fatalf("a same-origin redirect was classified as cross-origin: %s -> %s", from, to) + } + from, to, ok = CrossOriginRedirect(redirect(http.StatusFound, "https://user:pass@evil.example/?token=x#frag", nil), &http.Request{URL: origin}) + if !ok || from != "https://fs.tidbapi.com" || to != "https://evil.example" { + t.Fatalf("origins were not reduced to scheme://host: from=%q to=%q ok=%v", from, to, ok) + } + for _, res := range []*http.Response{ + nil, + redirect(http.StatusOK, "", nil), + redirect(http.StatusNotModified, "", nil), + redirect(http.StatusFound, "", nil), + } { + if _, _, ok := CrossOriginRedirect(res, &http.Request{URL: origin}); ok { + t.Fatalf("non-cross-origin response classified as cross-origin: %#v", res) + } + } + if Origin(nil) != "" || Origin(mustURL("/relative")) != "" { + t.Fatal("Origin must be empty without a scheme and host") + } + if Origin(mustURL("about:blank")) != "" || Origin(mustURL("https://:8443/x")) != "" { + t.Fatal("Origin must be empty for opaque and port-only URLs") + } + + // Responses without their own request (custom RoundTrippers, direct callers) must + // resolve a relative or scheme-relative Location against the fallback request. + for _, tc := range []struct { + name string + location string + wantOK bool + wantTo string + }{ + {"relative same-origin", "/other", false, ""}, + {"scheme-relative other host", "//evil.example/other", true, "https://evil.example"}, + {"opaque target", "about:blank", false, ""}, + {"empty host target", "https:///moved", false, ""}, + {"port-only target", "https://:8443/moved", false, ""}, + {"no location", "", false, ""}, + } { + res := redirect(http.StatusFound, tc.location, nil) + from, to, ok := CrossOriginRedirect(res, &http.Request{URL: origin}) + if ok != tc.wantOK { + t.Fatalf("%s: ok = %v, want %v (from=%q to=%q)", tc.name, ok, tc.wantOK, from, to) + } + if tc.wantOK && to != tc.wantTo { + t.Fatalf("%s: toOrigin = %q, want %q", tc.name, to, tc.wantTo) + } + } + if _, _, ok := CrossOriginRedirect(redirect(http.StatusFound, "https://evil.example/x", nil), nil); ok { + t.Fatal("a response without any request must not be classified") + } +} diff --git a/internal/db/sqlhttp/sqlhttp.go b/internal/db/sqlhttp/sqlhttp.go index 910a87fe..004f4453 100644 --- a/internal/db/sqlhttp/sqlhttp.go +++ b/internal/db/sqlhttp/sqlhttp.go @@ -6,6 +6,7 @@ import ( "crypto/rand" "encoding/hex" "encoding/json" + "errors" "fmt" "io" "net/http" @@ -13,6 +14,7 @@ import ( "strings" "time" + "github.com/tidbcloud/ti-cli/internal/api/transport" "github.com/tidbcloud/ti-cli/internal/apperr" "github.com/tidbcloud/ti-cli/internal/db/sqlcred" "github.com/tidbcloud/ti-cli/internal/db/sqlresult" @@ -42,6 +44,13 @@ func Execute(ctx context.Context, opts Options) (sqlresult.Result, error) { if client == nil { client = &http.Client{Timeout: 30 * time.Second} } + // SQL HTTPS API requests carry the cluster's SQL user credentials in Basic auth. + // net/http still copies Authorization across same-host port changes, subdomain + // redirects, and https->http downgrades, so the credential-bearing same-origin + // rule is enforced here too, composed under any caller-supplied policy. + copied := *client + copied.CheckRedirect = transport.SameOriginThen(client.CheckRedirect) + client = &copied endpoint, err := endpointURL(opts) if err != nil { return sqlresult.Result{}, err @@ -74,9 +83,18 @@ func Execute(ctx context.Context, opts Options) (sqlresult.Result, error) { res, err := client.Do(req) recordSQLHTTPEvent(ctx, traceID, res, err, time.Since(start)) if err != nil { + if errors.Is(err, transport.ErrTooManyRedirects) { + return sqlresult.Result{}, apperr.Wrap("db.sql_http_redirect_loop", "database", 1, "SQL HTTPS API redirect loop: ti follows redirects only within the same scheme and host", err) + } return sqlresult.Result{}, apperr.Wrap("db.sql_http_network", "database", 1, "SQL HTTPS API request failed: check network connectivity and cluster endpoint", err) } defer res.Body.Close() + if fromOrigin, toOrigin, crossOrigin := transport.CrossOriginRedirect(res, req); crossOrigin { + // The cross-origin redirect was refused instead of followed: report the + // origin change rather than the bare status code. A same-origin redirect that + // a caller policy refused keeps its normal status handling. + return sqlresult.Result{}, apperr.New("db.sql_http_redirect_refused", "database", 1, fmt.Sprintf("refused a SQL HTTPS API redirect from %s to %s: ti follows redirects only within the same scheme and host", fromOrigin, toOrigin)) + } if res.StatusCode < 200 || res.StatusCode >= 300 { return sqlresult.Result{}, statusError(res) } @@ -113,6 +131,11 @@ func recordSQLHTTPEvent(ctx context.Context, traceID string, res *http.Response, } if requestErr != nil { event.ErrorCode = "db.sql_http_network" + if errors.Is(requestErr, transport.ErrTooManyRedirects) { + // Execute reports this as db.sql_http_redirect_loop; the operation log must + // not contradict the command error. + event.ErrorCode = "db.sql_http_redirect_loop" + } event.ErrorCategory = "database" } oplog.FromContext(ctx).Record(ctx, event) diff --git a/internal/db/sqlhttp/sqlhttp_test.go b/internal/db/sqlhttp/sqlhttp_test.go index b2307a23..0fa857cf 100644 --- a/internal/db/sqlhttp/sqlhttp_test.go +++ b/internal/db/sqlhttp/sqlhttp_test.go @@ -6,6 +6,7 @@ import ( "net/http" "net/http/httptest" "strings" + "sync/atomic" "testing" "github.com/tidbcloud/ti-cli/internal/apperr" @@ -147,3 +148,162 @@ func TestEndpointURLFromClusterHost(t *testing.T) { t.Fatalf("endpointURL = %q", got) } } + +const sqlHTTPJSONResponse = `{"types":[{"name":"n","type":"INT","nullable":false}],"rows":[["1"]],"rowsAffected":0}` + +func executeAgainst(t *testing.T, baseURL string, client *http.Client) error { + t.Helper() + _, err := Execute(context.Background(), Options{ + ClusterID: "cluster-1", + AccessMode: sqlcred.ReadWrite, + Username: "user", + Password: "pass", + Database: "test", + SQL: "select 1", + BaseURL: baseURL, + HTTPClient: client, + }) + return err +} + +// TestExecuteHTTPRefusesCrossOriginRedirect covers the credential-redirect rule +// for the Basic-authenticated SQL transport: the second origin must never receive +// the request, and the failure names the origin change. +func TestExecuteHTTPRefusesCrossOriginRedirect(t *testing.T) { + var attackerHits atomic.Int32 + attacker := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + attackerHits.Add(1) + _, _ = w.Write([]byte(sqlHTTPJSONResponse)) + })) + t.Cleanup(attacker.Close) + redirecting := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, attacker.URL+"/v1beta/sql", http.StatusTemporaryRedirect) + })) + t.Cleanup(redirecting.Close) + + err := executeAgainst(t, redirecting.URL, nil) + if apperr.CodeFor(err) != "db.sql_http_redirect_refused" { + t.Fatalf("error code = %q, want db.sql_http_redirect_refused: %v", apperr.CodeFor(err), err) + } + if message := apperr.MessageFor(err); !strings.Contains(message, "refused a SQL HTTPS API redirect") || !strings.Contains(message, "to "+attacker.URL) { + t.Fatalf("refusal message is not actionable: %q", message) + } + if got := attackerHits.Load(); got != 0 { + t.Fatalf("the redirect target received %d request(s); SQL credentials could have leaked", got) + } +} + +// TestExecuteHTTPRefusesCrossOriginRedirectUnderCallerPolicy pins that a caller +// policy can only narrow the rule, never widen it. +func TestExecuteHTTPRefusesCrossOriginRedirectUnderCallerPolicy(t *testing.T) { + var attackerHits atomic.Int32 + attacker := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + attackerHits.Add(1) + })) + t.Cleanup(attacker.Close) + redirecting := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, attacker.URL+"/v1beta/sql", http.StatusFound) + })) + t.Cleanup(redirecting.Close) + + permissive := &http.Client{CheckRedirect: func(*http.Request, []*http.Request) error { return nil }} + if err := executeAgainst(t, redirecting.URL, permissive); apperr.CodeFor(err) != "db.sql_http_redirect_refused" { + t.Fatalf("permissive caller policy error = %v, want db.sql_http_redirect_refused", err) + } + if got := attackerHits.Load(); got != 0 { + t.Fatalf("the redirect target received %d request(s)", got) + } +} + +// TestExecuteHTTPFollowsSameOriginRedirect keeps ordinary same-origin redirects +// working. 307 preserves the method, so the redirected request must still be the +// authenticated POST with the SQL body. +func TestExecuteHTTPFollowsSameOriginRedirect(t *testing.T) { + var requests atomic.Int32 + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + requests.Add(1) + if r.URL.Path != "/moved" { + http.Redirect(w, r, "/moved", http.StatusTemporaryRedirect) + return + } + if r.Method != http.MethodPost { + t.Errorf("redirected method = %s, want POST", r.Method) + } + if user, password, ok := r.BasicAuth(); !ok || user != "user" || password != "pass" { + t.Errorf("redirected request lost its Basic auth: %q %q", user, password) + } + var body map[string]string + if err := json.NewDecoder(r.Body).Decode(&body); err != nil || body["query"] != "select 1" { + t.Errorf("redirected request lost the SQL body: %#v (%v)", body, err) + } + _, _ = w.Write([]byte(sqlHTTPJSONResponse)) + })) + t.Cleanup(server.Close) + + if err := executeAgainst(t, server.URL, nil); err != nil { + t.Fatalf("same-origin redirect failed: %v", err) + } + if got := requests.Load(); got != 2 { + t.Fatalf("requests = %d, want 2", got) + } +} + +// TestExecuteHTTPSameOriginRedirectRefusedByCallerFallsThrough pins the +// classification rule: when a stricter caller policy refuses a same-origin +// redirect, Execute must not claim a cross-origin refusal. +func TestExecuteHTTPSameOriginRedirectRefusedByCallerFallsThrough(t *testing.T) { + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, "/moved", http.StatusFound) + })) + t.Cleanup(server.Close) + + noRedirects := &http.Client{CheckRedirect: func(*http.Request, []*http.Request) error { + return http.ErrUseLastResponse + }} + err := executeAgainst(t, server.URL, noRedirects) + if code := apperr.CodeFor(err); code != "db.sql_execution_failed" { + t.Fatalf("error code = %q, want db.sql_execution_failed: %v", code, err) + } + if message := apperr.MessageFor(err); strings.Contains(message, "refused a SQL HTTPS API redirect") { + t.Fatalf("a same-origin refusal was misclassified as a cross-origin redirect: %q", message) + } +} + +// TestExecuteHTTPRecordsRedirectLoopEvent pins that the operation log does not +// contradict the command error for a same-origin redirect loop. +func TestExecuteHTTPRecordsRedirectLoopEvent(t *testing.T) { + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, r.URL.Path, http.StatusFound) + })) + t.Cleanup(server.Close) + recorder := &memoryRecorder{} + + _, err := Execute(oplog.WithRecorder(context.Background(), recorder), Options{ + ClusterID: "cluster-1", + Username: "user", + Password: "pass", + SQL: "select 1", + BaseURL: server.URL, + }) + if apperr.CodeFor(err) != "db.sql_http_redirect_loop" { + t.Fatalf("error code = %q: %v", apperr.CodeFor(err), err) + } + if len(recorder.events) != 1 { + t.Fatalf("expected one event, got %#v", recorder.events) + } + if code := recorder.events[0].ErrorCode; code != "db.sql_http_redirect_loop" { + t.Fatalf("operation log error code = %q, want db.sql_http_redirect_loop", code) + } +} + +// TestExecuteHTTPReportsRedirectLoop pins the redirect cap for the SQL client. +func TestExecuteHTTPReportsRedirectLoop(t *testing.T) { + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, r.URL.Path, http.StatusFound) + })) + t.Cleanup(server.Close) + + if err := executeAgainst(t, server.URL, nil); apperr.CodeFor(err) != "db.sql_http_redirect_loop" { + t.Fatalf("redirect loop error = %v, want db.sql_http_redirect_loop", err) + } +} diff --git a/internal/fsregion/client.go b/internal/fsregion/client.go index 5cef52e8..45805b7a 100644 --- a/internal/fsregion/client.go +++ b/internal/fsregion/client.go @@ -74,9 +74,12 @@ func (c Client) Fetch(ctx context.Context) (Catalog, error) { if baseURL == "" { baseURL = DefaultBaseURL } - parsedBase, err := url.Parse(baseURL) - if err != nil || parsedBase.Scheme == "" || parsedBase.Host == "" { - return Catalog{}, apperr.New("api.fs_regions_request", "api", 1, fmt.Sprintf("invalid Filesystem region catalog endpoint %q", baseURL)) + // This request carries the organization API key pair, so the endpoint itself + // must be a secure origin, and a rejected endpoint is described without the + // parts that could carry secrets. + parsedBase, err := parseSecureOrigin(baseURL) + if err != nil { + return Catalog{}, apperr.New("api.fs_regions_request", "config", 2, describeRejectedOrigin("Filesystem region catalog endpoint", parsedBase, err)) } endpoint := strings.TrimRight(parsedBase.String(), "/") + RegionsPath @@ -101,7 +104,7 @@ func (c Client) Fetch(ctx context.Context) (Catalog, error) { if err != nil { return Catalog{}, apperr.Wrap("api.fs_regions_unavailable", "api", 1, "the Filesystem region catalog is unavailable", err) } - pageData, readErr := c.readCatalogPage(response) + pageData, readErr := c.readCatalogPage(request, response) response.Body.Close() if readErr != nil { return Catalog{}, readErr @@ -128,21 +131,30 @@ func (c Client) Fetch(ctx context.Context) (Catalog, error) { func (c Client) httpClient() *http.Client { var base http.RoundTripper = http.DefaultTransport - if c.HTTPClient != nil && c.HTTPClient.Transport != nil { - base = c.HTTPClient.Transport + if c.HTTPClient != nil { + if c.HTTPClient.Transport != nil { + base = c.HTTPClient.Transport + } } redactor := apitransport.Redactor{Secrets: []string{c.Credentials.PublicKey, c.Credentials.PrivateKey}} transport := apitransport.NewDebugRoundTripper(base, c.Debug, redactor) transport = apitransport.NewDigest(c.Credentials.PublicKey, c.Credentials.PrivateKey, transport) + + // The catalog request is Digest-authenticated, and the digest transport answers + // any 401 challenge it receives, so a redirect to another origin would hand the + // organization API key pair to that host. The same-origin rule is composed under + // any caller-supplied policy: a caller can narrow redirects further, never widen + // them past this origin check. client := &http.Client{Transport: transport, Timeout: catalogTimeout} if c.HTTPClient != nil { - copied := *c.HTTPClient - copied.Transport = transport - if copied.Timeout == 0 { - copied.Timeout = catalogTimeout + caller := *c.HTTPClient + if caller.Timeout == 0 { + caller.Timeout = catalogTimeout } - client = &copied + client = &caller + client.Transport = transport } + client.CheckRedirect = apitransport.SameOriginThen(client.CheckRedirect) return client } @@ -155,7 +167,18 @@ func catalogQuery(pageToken string) url.Values { return query } -func (c Client) readCatalogPage(response *http.Response) (catalogPage, error) { +func (c Client) readCatalogPage(request *http.Request, response *http.Response) (catalogPage, error) { + // A refused cross-origin redirect was never followed: report the origin change + // instead of a generic catalog failure so the hardening is visible. A same-origin + // redirect that a caller policy refused keeps its normal status handling. + if fromOrigin, toOrigin, crossOrigin := apitransport.CrossOriginRedirect(response, request); crossOrigin { + return catalogPage{}, apperr.New( + "api.fs_regions_redirect_refused", + "api", + 1, + fmt.Sprintf("refused a Filesystem region catalog redirect from %s to %s: ti follows redirects only within the same scheme and host", fromOrigin, toOrigin), + ) + } if response.StatusCode < 200 || response.StatusCode >= 300 { // The catalog is a Digest-authenticated TiDB Cloud control-plane // endpoint, so credential, authorization, and not-found failures keep the diff --git a/internal/fsregion/fsregion.go b/internal/fsregion/fsregion.go index b183bbd9..90bada68 100644 --- a/internal/fsregion/fsregion.go +++ b/internal/fsregion/fsregion.go @@ -9,12 +9,14 @@ package fsregion import ( + "errors" "fmt" "net" "net/url" "sort" "strings" + apitransport "github.com/tidbcloud/ti-cli/internal/api/transport" "github.com/tidbcloud/ti-cli/internal/apperr" "github.com/tidbcloud/ti-cli/internal/config/region" ) @@ -153,17 +155,57 @@ func NormalizeServiceAddress(value string) (string, error) { if trimmed == "" { return "", invalid("the Filesystem region catalog entry is missing service_address") } - parsed, err := url.Parse(trimmed) - if err != nil || parsed.Scheme == "" || parsed.Host == "" { - return "", invalid(fmt.Sprintf("invalid Filesystem service address %q", trimmed)) + parsed, err := parseSecureOrigin(trimmed) + if err != nil { + return "", invalid(describeRejectedOrigin("Filesystem service address", parsed, err)) + } + return strings.TrimRight(parsed.String(), "/"), nil +} + +// parseSecureOrigin parses a URL that ti will spend credentials on and enforces +// the origin rules shared by the catalog endpoint and published service +// addresses: an absolute https URL (plain http only for a loopback test server) +// with a host name and without embedded credentials, a query - including a bare +// trailing "?" - or a fragment. +// +// The returned URL is safe to describe in an error only through +// describeRejectedOrigin; the raw input may embed a password or token. +func parseSecureOrigin(value string) (*url.URL, error) { + parsed, err := url.Parse(value) + if err != nil { + return nil, errOriginUnparsable + } + if parsed.Scheme == "" || parsed.Hostname() == "" { + return parsed, errOriginNeedsHost + } + if parsed.User != nil || parsed.RawQuery != "" || parsed.ForceQuery || parsed.Fragment != "" { + return parsed, errOriginCarriesSecrets } if parsed.Scheme != "https" && !(parsed.Scheme == "http" && isLoopbackHost(parsed.Hostname())) { - return "", invalid(fmt.Sprintf("invalid Filesystem service address %q: scheme must be https", trimmed)) + return parsed, errOriginInsecureScheme } - if parsed.User != nil || parsed.RawQuery != "" || parsed.Fragment != "" { - return "", invalid(fmt.Sprintf("invalid Filesystem service address %q: credentials, query, and fragment are not allowed", trimmed)) + return parsed, nil +} + +var ( + errOriginUnparsable = errors.New("it cannot be parsed as a URL") + errOriginNeedsHost = errors.New("it needs a scheme and a host name") + errOriginCarriesSecrets = errors.New("credentials, query, and fragment are not allowed") + errOriginInsecureScheme = errors.New("scheme must be https, and plain http is allowed only for localhost") +) + +// describeRejectedOrigin phrases a rejected URL without echoing the parts that +// can carry secrets, so a rejected endpoint never prints a password or query +// token to stderr. +func describeRejectedOrigin(kind string, parsed *url.URL, reason error) string { + // Only the scheme and host are echoed. Formatting the parsed URL, or any other + // field of it, would echo secrets for inputs that do not parse as a hierarchical + // URL: "https:user:secret@host" becomes Opaque, and a pasted token becomes Path. + display := apitransport.Origin(parsed) + if display == "" { + return fmt.Sprintf("invalid %s: %v", kind, reason) } - return strings.TrimRight(parsed.String(), "/"), nil + return fmt.Sprintf("invalid %s %q: %v", kind, display, reason) } // isLoopbackHost reports whether host is localhost or a loopback IP address. diff --git a/internal/fsregion/fsregion_test.go b/internal/fsregion/fsregion_test.go index 98723c56..27fee436 100644 --- a/internal/fsregion/fsregion_test.go +++ b/internal/fsregion/fsregion_test.go @@ -214,6 +214,8 @@ func TestNormalizeServiceAddress(t *testing.T) { {value: "http://drive9.example", ok: false}, {value: "https://user:pass@drive9.example", ok: false}, {value: "https://drive9.example/#fragment", ok: false}, + {value: "https://drive9.example?", ok: false}, + {value: "https://:8443", ok: false}, {value: "https://", ok: false}, {value: "", ok: false}, } { @@ -274,3 +276,207 @@ func TestDeriveServiceAddress(t *testing.T) { } } } + +// TestFetchRefusesCrossOriginRedirect covers the reported leak: the catalog +// request is Digest-authenticated with the organization API key pair, so a +// redirect to another origin must never be followed, and the second origin must +// not even receive the request. +func TestFetchRefusesCrossOriginRedirect(t *testing.T) { + var attackerHits atomic.Int32 + attacker := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + attackerHits.Add(1) + w.Header().Set("WWW-Authenticate", `Digest realm="attacker", nonce="nonce", qop="auth"`) + w.WriteHeader(http.StatusUnauthorized) + })) + t.Cleanup(attacker.Close) + + catalog := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, attacker.URL+RegionsPath, http.StatusFound) + })) + t.Cleanup(catalog.Close) + + _, err := Client{BaseURL: catalog.URL, Credentials: testCredentials()}.Fetch(context.Background()) + if apperr.CodeFor(err) != "api.fs_regions_redirect_refused" { + t.Fatalf("error code = %q, want api.fs_regions_redirect_refused: %v", apperr.CodeFor(err), err) + } + message := apperr.MessageFor(err) + if !strings.Contains(message, "refused a Filesystem region catalog redirect") || + !strings.Contains(message, "to "+attacker.URL) { + t.Fatalf("refusal message is not actionable: %q", message) + } + if got := attackerHits.Load(); got != 0 { + t.Fatalf("the redirect target received %d request(s); credentials could have leaked", got) + } +} + +// TestFetchAppliesRedirectPolicyToCallerClient pins that the policy survives the +// caller-supplied client seam the resolver uses for tests. +func TestFetchAppliesRedirectPolicyToCallerClient(t *testing.T) { + var attackerHits atomic.Int32 + attacker := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + attackerHits.Add(1) + })) + t.Cleanup(attacker.Close) + catalog := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, attacker.URL+RegionsPath, http.StatusTemporaryRedirect) + })) + t.Cleanup(catalog.Close) + + _, err := Client{ + BaseURL: catalog.URL, + HTTPClient: &http.Client{}, + Credentials: testCredentials(), + }.Fetch(context.Background()) + if apperr.CodeFor(err) != "api.fs_regions_redirect_refused" { + t.Fatalf("error code = %q: %v", apperr.CodeFor(err), err) + } + if got := attackerHits.Load(); got != 0 { + t.Fatalf("the redirect target received %d request(s)", got) + } +} + +// TestFetchFollowsSameOriginRedirect keeps ordinary same-origin redirects working. +func TestFetchFollowsSameOriginRedirect(t *testing.T) { + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/moved"+RegionsPath { + http.Redirect(w, r, "/moved"+RegionsPath, http.StatusFound) + return + } + _, _ = w.Write([]byte(`{"regions":[` + regionJSON("aws-us-east-1", "aws", "https://aws-us-east-1.drive9.ai", "ACTIVE") + `],"totalSize":1,"nextPageToken":""}`)) + })) + t.Cleanup(server.Close) + + catalog, err := Client{BaseURL: server.URL, Credentials: testCredentials()}.Fetch(context.Background()) + if err != nil { + t.Fatalf("same-origin redirect failed: %v", err) + } + if len(catalog.Entries) != 1 || catalog.Entries[0].RegionCode != "aws-us-east-1" { + t.Fatalf("catalog = %#v", catalog.Entries) + } +} + +// TestFetchEnforcesSameOriginUnderCallerRedirectPolicy pins that the mandatory +// same-origin check cannot be relaxed by a caller-supplied client: it runs before +// the caller's policy, so a permissive caller cannot re-enable cross-origin +// redirects while a stricter caller can still narrow them further. +func TestFetchEnforcesSameOriginUnderCallerRedirectPolicy(t *testing.T) { + var attackerHits atomic.Int32 + attacker := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + attackerHits.Add(1) + w.Header().Set("WWW-Authenticate", `Digest realm="attacker", nonce="nonce", qop="auth"`) + w.WriteHeader(http.StatusUnauthorized) + })) + t.Cleanup(attacker.Close) + catalog := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, attacker.URL+RegionsPath, http.StatusFound) + })) + t.Cleanup(catalog.Close) + + permissive := &http.Client{CheckRedirect: func(*http.Request, []*http.Request) error { return nil }} + _, err := Client{ + BaseURL: catalog.URL, + HTTPClient: permissive, + Credentials: testCredentials(), + }.Fetch(context.Background()) + if apperr.CodeFor(err) != "api.fs_regions_redirect_refused" { + t.Fatalf("permissive caller policy error = %v, want api.fs_regions_redirect_refused", err) + } + if got := attackerHits.Load(); got != 0 { + t.Fatalf("the redirect target received %d request(s)", got) + } +} + +// TestFetchHonorsStricterCallerRedirectPolicy keeps the composition honest: a +// caller that refuses all redirects still wins over the same-origin policy. +func TestFetchHonorsStricterCallerRedirectPolicy(t *testing.T) { + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, "/moved"+RegionsPath, http.StatusFound) + })) + t.Cleanup(server.Close) + + noRedirects := &http.Client{CheckRedirect: func(*http.Request, []*http.Request) error { + return http.ErrUseLastResponse + }} + _, err := Client{ + BaseURL: server.URL, + HTTPClient: noRedirects, + Credentials: testCredentials(), + }.Fetch(context.Background()) + if apperr.CodeFor(err) != "api.fs_regions_unavailable" { + t.Fatalf("stricter caller policy error = %v, want a 3xx catalog failure", err) + } +} + +// TestFetchRejectsInsecureCatalogEndpoint pins the base-URL rule: the endpoint +// spends the organization API key pair, so it must be https (or a loopback test +// server) before any request is made. Each rejection names the actual problem +// instead of blaming the wrong part of the URL. +func TestFetchRejectsInsecureCatalogEndpoint(t *testing.T) { + for _, tc := range []struct { + baseURL string + want string + }{ + {"http://fs.tidbapi.com", "scheme must be https"}, + {"ftp://fs.tidbapi.com", "scheme must be https"}, + {"https://user:pass@fs.tidbapi.com", "credentials, query, and fragment are not allowed"}, + {"https://fs.tidbapi.com?region=aws-us-east-1", "credentials, query, and fragment are not allowed"}, + {"https://fs.tidbapi.com?", "credentials, query, and fragment are not allowed"}, + {"https://fs.tidbapi.com#frag", "credentials, query, and fragment are not allowed"}, + {"https://:8443", "it needs a scheme and a host name"}, + {"//fs.tidbapi.com", "it needs a scheme and a host name"}, + {"fs.tidbapi.com", "it needs a scheme and a host name"}, + } { + _, err := Client{BaseURL: tc.baseURL, Credentials: testCredentials()}.Fetch(context.Background()) + if apperr.CodeFor(err) != "api.fs_regions_request" { + t.Fatalf("endpoint %q error = %v, want api.fs_regions_request", tc.baseURL, err) + } + if exit := apperr.ExitCodeFor(err); exit != 2 { + t.Fatalf("endpoint %q exit code = %d, want 2", tc.baseURL, exit) + } + message := apperr.MessageFor(err) + if !strings.Contains(message, tc.want) { + t.Fatalf("endpoint %q message = %q, want it to contain %q", tc.baseURL, message, tc.want) + } + } + if message := apperr.MessageFor(func() error { + _, err := Client{BaseURL: "fs.tidbapi.com", Credentials: testCredentials()}.Fetch(context.Background()) + return err + }()); strings.Contains(message, "credentials, query") { + t.Fatalf("a missing scheme must not be reported as a credentials/query problem: %q", message) + } +} + +// TestRejectedOriginsNeverEchoSecrets pins the redaction rule: a rejected +// endpoint or service address must not print the embedded password or query +// token to stderr. +func TestRejectedOriginsNeverEchoSecrets(t *testing.T) { + const password = "sup3r-s3cret" + const queryToken = "token-value" + for _, raw := range []string{ + "https://user:" + password + "@fs.tidbapi.com", + "https://fs.tidbapi.com?pageToken=" + queryToken, + "https://user:" + password + "@fs.tidbapi.com?pageToken=" + queryToken + "#frag", + // Inputs that do not parse as a hierarchical URL must not be echoed either: + // Opaque carries "user:secret@host" and Path can carry a pasted token. + "https:user:" + password + "@fs.tidbapi.com", + password + "@fs.tidbapi.com", + "ti_fs_" + queryToken, + } { + _, catalogErr := Client{BaseURL: raw, Credentials: testCredentials()}.Fetch(context.Background()) + if catalogErr == nil { + t.Fatalf("endpoint %q was accepted", raw) + } + if message := apperr.MessageFor(catalogErr); strings.Contains(message, password) || strings.Contains(message, queryToken) { + t.Fatalf("catalog endpoint error echoed a secret: %q", message) + } + if _, serviceErr := NormalizeServiceAddress(raw); serviceErr == nil { + t.Fatalf("service address %q was accepted", raw) + } else if message := apperr.MessageFor(serviceErr); strings.Contains(message, password) || strings.Contains(message, queryToken) { + t.Fatalf("service address error echoed a secret: %q", message) + } + } + + if _, err := (Client{BaseURL: "http://127.0.0.1:1", Credentials: testCredentials()}).Fetch(context.Background()); apperr.CodeFor(err) != "api.fs_regions_unavailable" { + t.Fatalf("loopback http endpoint error = %v, want a transport failure", err) + } +}