Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -1066,6 +1066,25 @@ credential when ti learns about the File System (create/import) and refreshed by
`ti fs list-regions`. Region-to-address resolution for data-plane traffic must
never consult the network.

Every HTTP client that carries a credential - Digest, Bearer, Basic auth, or a
custom credential header - must use the shared `transport.SameOriginRedirect`
policy, and a caller-supplied `CheckRedirect` is composed through
`transport.SameOriginThen`, so a caller may narrow redirects (refuse all of them,
for example) but can never re-enable a cross-origin hop. The Filesystem region
catalog spends the organization API key pair through
`internal/api/transport.NewDigest`, and that transport answers any 401 challenge
it receives; the SQL HTTPS API spends the cluster's SQL user password through
Basic auth, and net/http still copies `Authorization` across same-host port
changes, subdomain redirects, and https->http downgrades. A refused redirect is
reported instead of followed: `api.fs_regions_redirect_refused` (fsregion),
`api.redirect_refused` / `api.redirect_loop` (internal/api), and
`db.sql_http_redirect_refused` / `db.sql_http_redirect_loop` (sqlhttp). The
catalog endpoint itself must be `https`, or plain `http` only on a loopback host,
without embedded credentials, a query, or a fragment, and a rejected endpoint is
described through `describeRejectedOrigin` so a password or token in the URL
never reaches stderr. Never build a credential-bearing client with a nil
`CheckRedirect`.

Do not store secrets in logs, telemetry, generated docs examples, or test
fixtures.

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,7 @@ ti fs list-layer-chain --layer-ref experiment-a --output text

Use `ti fs delete-layer --layer-ref experiment-a` to abandon a rejected leaf. Deleting a layer with live descendants fails unless `--cascade` is supplied explicitly. Layer names are not durable unique identifiers because abandoned layers remain visible; automation should capture returned layer IDs and use run-unique checkpoint IDs.

`ti fs list-regions` lists the Filesystem regions published by TiDB Cloud Filesystem, with their cloud provider, display name, state, and regional service address. It uses TiDB Cloud credentials, always asks the API, and refreshes the recorded `service_address` of locally stored File System credentials:
`ti fs list-regions` lists the Filesystem regions published by TiDB Cloud Filesystem, with their cloud provider, display name, state, and regional service address. It uses TiDB Cloud credentials, always asks the API, and refreshes the recorded `service_address` of locally stored File System credentials. The catalog request is Digest-authenticated with the organization API key pair, so `ti` follows a redirect only when it keeps the same scheme and host and otherwise fails with `api.fs_regions_redirect_refused`; the catalog endpoint itself must be `https` (plain `http` is accepted only for a loopback test server) and must not embed credentials, a query, or a fragment:

```shell
ti fs list-regions
Expand Down
39 changes: 10 additions & 29 deletions internal/api/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ const defaultMaxRetries = 2

// maxRedirects matches net/http's default limit, which a custom CheckRedirect
// replaces.
const maxRedirects = 10
const maxRedirects = apitransport.MaxRedirects

type Client struct {
BaseURL *url.URL
Expand Down Expand Up @@ -84,11 +84,12 @@ func New(opts Options) (*Client, error) {
Timeout: opts.Timeout,
}
}
if httpClient.CheckRedirect == nil {
copied := *httpClient
copied.CheckRedirect = sameOriginRedirect
httpClient = &copied
}
// The credential-bearing transports must never follow a cross-origin redirect,
// so the mandatory rule runs under any caller-supplied policy: a caller can
// narrow redirects, never widen them.
copiedClient := *httpClient
copiedClient.CheckRedirect = apitransport.SameOriginThen(httpClient.CheckRedirect)
httpClient = &copiedClient

maxRetries := opts.MaxRetries
if maxRetries == 0 {
Expand Down Expand Up @@ -275,26 +276,10 @@ func (c *Client) do(req *http.Request) (*http.Response, error) {
return nil, lastErr
}

// sameOriginRedirect follows a redirect only when it keeps the scheme and host
// of the original request. The digest and bearer transports add credentials on
// every round trip, so following a redirect to another origin would send them
// there. A refused redirect comes back as the 3xx response.
func sameOriginRedirect(req *http.Request, via []*http.Request) error {
if len(via) >= maxRedirects {
return errTooManyRedirects
}
if !sameOrigin(req.URL, via[0].URL) {
return http.ErrUseLastResponse
}
return nil
}

var errTooManyRedirects = fmt.Errorf("stopped after %d redirects", maxRedirects)

// requestError maps a failed round trip to an API error. A same-origin
// redirect loop is reported as such rather than as a network problem.
func requestError(req *http.Request, err error) error {
if errors.Is(err, errTooManyRedirects) {
if errors.Is(err, apitransport.ErrTooManyRedirects) {
return &Error{
Code: "api.redirect_loop",
Category: "api",
Expand All @@ -312,10 +297,6 @@ func requestError(req *http.Request, err error) error {
}
}

func sameOrigin(a, b *url.URL) bool {
return a.Scheme == b.Scheme && strings.EqualFold(a.Host, b.Host)
}

func retryable(statusCode int) bool {
return statusCode == http.StatusTooManyRequests || statusCode >= 500
}
Expand All @@ -339,14 +320,14 @@ func (c *Client) statusError(req *http.Request, res *http.Response) error {
apiMessage := remoteError.Message
requestID := responseRequestID(res.Header)
if res.StatusCode >= 300 && res.StatusCode < 400 {
if target, err := res.Location(); err == nil && !sameOrigin(target, req.URL) {
if fromOrigin, toOrigin, crossOrigin := apitransport.CrossOriginRedirect(res, req); crossOrigin {
return &Error{
Code: "api.redirect_refused",
Category: "api",
ExitCode: 1,
StatusCode: res.StatusCode,
RequestID: requestID,
Message: fmt.Sprintf("refused a redirect from %s://%s to %s://%s: ti follows redirects only within the same scheme and host", req.URL.Scheme, req.URL.Host, target.Scheme, target.Host),
Message: fmt.Sprintf("refused a redirect from %s to %s: ti follows redirects only within the same scheme and host", fromOrigin, toOrigin),
Body: string(body),
}
}
Expand Down
33 changes: 11 additions & 22 deletions internal/api/redirect_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -154,28 +154,6 @@ func TestRefusedDowngradeRedirectNamesBothSchemes(t *testing.T) {
}
}

func TestSameOriginRedirectRules(t *testing.T) {
via := []*http.Request{{URL: mustParseURL(t, "https://api.example.com/v1/a")}}
for _, tc := range []struct {
target string
allow bool
}{
{"https://api.example.com/v1/b", true},
{"https://API.example.com/v1/b", true},
{"http://api.example.com/v1/b", false},
{"https://api.example.com:8443/v1/b", false},
{"https://evil.example/v1/b", false},
} {
err := sameOriginRedirect(&http.Request{URL: mustParseURL(t, tc.target)}, via)
if tc.allow && err != nil {
t.Fatalf("%s: expected redirect to be followed, got %v", tc.target, err)
}
if !tc.allow && !errors.Is(err, http.ErrUseLastResponse) {
t.Fatalf("%s: expected redirect to be refused, got %v", tc.target, err)
}
}
}

func TestNewKeepsCallerRedirectPolicyAndDoesNotMutateClient(t *testing.T) {
shared := &http.Client{}
client, err := New(Options{Endpoint: endpoints.Endpoint{BaseURL: "https://api.example.com"}, HTTPClient: shared})
Expand All @@ -197,6 +175,17 @@ func TestNewKeepsCallerRedirectPolicyAndDoesNotMutateClient(t *testing.T) {
if err := client.HTTPClient.CheckRedirect(&http.Request{URL: mustParseURL(t, "https://api.example.com/x")}, []*http.Request{{URL: mustParseURL(t, "https://api.example.com/")}}); !errors.Is(err, http.ErrUseLastResponse) {
t.Fatalf("expected the caller's CheckRedirect to be kept, got %v", err)
}

// A caller policy that allows every redirect must not widen the client past
// the credential-bearing same-origin rule.
permissive := func(*http.Request, []*http.Request) error { return nil }
client, err = New(Options{Endpoint: endpoints.Endpoint{BaseURL: "https://api.example.com"}, HTTPClient: &http.Client{CheckRedirect: permissive}})
if err != nil {
t.Fatalf("New failed: %v", err)
}
if err := client.HTTPClient.CheckRedirect(&http.Request{URL: mustParseURL(t, "https://evil.example/x")}, []*http.Request{{URL: mustParseURL(t, "https://api.example.com/")}}); !errors.Is(err, http.ErrUseLastResponse) {
t.Fatalf("a permissive caller re-enabled a cross-origin redirect: %v", err)
}
}

func testBearerClient(t *testing.T, baseURL string) *Client {
Expand Down
103 changes: 103 additions & 0 deletions internal/api/transport/transport.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,114 @@ import (
"fmt"
"io"
"net/http"
"net/url"
"strings"

"github.com/icholy/digest"
)

// MaxRedirects matches net/http's default limit, which a custom CheckRedirect
// replaces.
const MaxRedirects = 10

// ErrTooManyRedirects reports that a same-origin redirect chain hit MaxRedirects,
// so a caller can tell a loop from a refused cross-origin redirect.
var ErrTooManyRedirects = fmt.Errorf("stopped after %d redirects", MaxRedirects)

// SameOriginRedirect is the CheckRedirect policy for every credential-bearing
// client: it follows a redirect only when it keeps the scheme and host of the
// original request, because the digest and bearer transports attach credentials
// on every round trip (the digest transport answers whatever 401 challenge it
// receives, including one from a host the user never chose). A refused redirect
// comes back to the caller as the 3xx response.
func SameOriginRedirect(req *http.Request, via []*http.Request) error {
if len(via) >= MaxRedirects {
return ErrTooManyRedirects
}
if req == nil || req.URL == nil || len(via) == 0 || via[0] == nil || via[0].URL == nil {
return http.ErrUseLastResponse
}
if !SameOrigin(req.URL, via[0].URL) {
return http.ErrUseLastResponse
}
return nil
}

// SameOriginThen returns a CheckRedirect policy that always applies the same-origin
// rule and then defers to caller. A caller-supplied policy can therefore narrow
// redirects further - for example refuse all of them - but can never re-enable a
// cross-origin hop for a credential-bearing client. A nil caller gets only the
// same-origin rule.
func SameOriginThen(caller func(req *http.Request, via []*http.Request) error) func(req *http.Request, via []*http.Request) error {
if caller == nil {
return SameOriginRedirect
}
return func(req *http.Request, via []*http.Request) error {
if err := SameOriginRedirect(req, via); err != nil {
return err
}
return caller(req, via)
}
}

// CrossOriginRedirect classifies a 3xx response as a cross-origin redirect and
// returns the two origins as scheme://host, without userinfo, query, or fragment.
// It reports false for a non-redirect, a missing or unparsable Location, and a
// same-origin redirect - a caller that refused a same-origin redirect (for
// example through a stricter caller policy) must fall through to its normal
// status handling instead of reporting an origin change that did not happen.
func CrossOriginRedirect(res *http.Response, fallbackReq *http.Request) (fromOrigin, toOrigin string, ok bool) {
if res == nil || res.StatusCode < 300 || res.StatusCode >= 400 {
return "", "", false
}
req := res.Request
if req == nil || req.URL == nil {
req = fallbackReq
}
if req == nil || req.URL == nil {
return "", "", false
}
// Resolve the header against the request URL directly: http.Response.Location
// only resolves relative values when the response carries its own request, so a
// fallback request would otherwise make "/other" look like another origin and
// "//evil.example/other" lose its scheme.
location := strings.TrimSpace(res.Header.Get("Location"))
if location == "" {
return "", "", false
}
target, err := req.URL.Parse(location)
if err != nil || target == nil || SameOrigin(target, req.URL) {
return "", "", false
}
fromOrigin, toOrigin = Origin(req.URL), Origin(target)
if fromOrigin == "" || toOrigin == "" {
// An originless target cannot name a different origin; never report one.
return "", "", false
}
return fromOrigin, toOrigin, true
}

// Origin renders scheme://host for a URL, dropping any userinfo, path, query, or
// fragment. It returns an empty string when the URL has no origin.
func Origin(u *url.URL) string {
if u == nil || u.Opaque != "" || u.Scheme == "" || u.Hostname() == "" {
return ""
}
return (&url.URL{Scheme: u.Scheme, Host: u.Host}).String()
}

// SameOrigin reports whether two URLs share a scheme and host. Host comparison
// folds case because DNS names are case-insensitive.
func SameOrigin(a, b *url.URL) bool {
if a == nil || b == nil || a.Opaque != "" || b.Opaque != "" {
return false
}
if a.Scheme == "" || a.Hostname() == "" || b.Scheme == "" || b.Hostname() == "" {
return false
}
return strings.EqualFold(a.Scheme, b.Scheme) && strings.EqualFold(a.Host, b.Host)
}

func NewDigest(username, password string, base http.RoundTripper) http.RoundTripper {
return &digest.Transport{
Username: username,
Expand Down
Loading
Loading