Support OAuth M2M (client credentials) on the default backend; reject unknown auth_type

[aminghadersohi]

What type of PR is this?

• Feature
• Bug Fix

Description

Changes:

• oauth_client_id + oauth_client_secret now authenticate a Databricks service principal on the default (Thrift) backend with OAuth M2M (client credentials against https://<host>/oidc/v1/token, scope all-apis). Tokens are refreshed when they expire. This is the credential shape the kernel backend (use_kernel=True) already accepts.
• An auth_type the connector does not implement now raises ValueError. Before, it fell through to the interactive browser login.
• Ambiguous combinations are rejected, as the kernel auth bridge does: oauth_client_secret with auth_type="databricks-oauth"/"azure-oauth", and oauth_client_secret with a custom credentials_provider. auth_type="azure-sp-m2m" keeps ignoring oauth_* values.
• ClientCredentialsTokenSource ignores unknown fields in the token response. The workspace endpoint returns scope, and OAuthResponse(**payload) raised TypeError on it.
• Docs: the connect() docstring and CONNECTION_PARAMETERS.md describe oauth_client_secret on Thrift, and oauth_redirect_port is documented as U2M-only. Changelog entries under # Unreleased.

Why: on the Thrift backend oauth_client_secret was silently ignored. oauth_client_id was then used as the client of an interactive U2M login: the connector opened a listener on 127.0.0.1:8020 and blocked until a browser redirect arrived. On a server that redirect never comes, so the process hangs. A mistyped auth_type hung the same way. For Databricks-managed service principals, the only M2M option was a credentials_provider callable, which applications that configure the connector from JSON or connection strings cannot pass.

How is this tested?

• Unit tests

• E2E Tests

• Manually

• N/A

• Unit: tests/unit/test_auth.py covers the M2M request/header, the M2M argument errors (including credentials_provider + oauth_client_secret), and an unknown auth_type with no interactive fallback. The first three fail on main. At the first commit tests/unit passed (530 passed, 237 skipped; test_parameters.py excluded because it fails to collect in the local environment with and without this change). The follow-up commit (ambiguity check, docs) was not run locally; CI covers it.

• Live, against a Databricks SQL warehouse on AWS with a service principal (scope all-apis, CAN USE on the warehouse):

  • connect(oauth_client_id=..., oauth_client_secret=...) runs queries, and current_user() is the service principal.
  • A wrong secret fails right away with RequestError ... 401 invalid_client.
  • auth_type="oauth-u2m" raises ValueError and opens no listener.
  • Before the OAuthResponse fix, the live connect failed with OAuthResponse.__init__() got an unexpected keyword argument 'scope'.

Related Tickets & Documents

None.

[Copilot]

Copilot review overview

🟡 Changes recommended

Reject the ambiguous combination of credentials_provider and oauth_client_secret, and update the OAuth parameter documentation.

Review effort: Lite
Findings: 1

Open (1)

• Scope oauth_redirect_port requirement to U2M authentication · New

What changed in this PR

Adds OAuth M2M client-credentials authentication for the default Thrift backend and rejects invalid or ambiguous authentication settings.

Changes:

• Adds service-principal token acquisition and refresh.
• Supports oauth_client_id and oauth_client_secret.
• Validates unsupported authentication types and OAuth response fields.
• Adds authentication tests.

File	Description
tests/​unit/​test_auth.py	Tests M2M authentication and validation.
src/​databricks/​sql/​client.py	Documents OAuth configuration.
src/​databricks/​sql/​auth/​oauth.py	Handles extra token-response fields.
src/​databricks/​sql/​auth/​common.py	Stores the OAuth client secret.
src/​databricks/​sql/​auth/​authenticators.py	Implements service-principal authentication.
src/​databricks/​sql/​auth/​auth.py	Routes M2M credentials and validates auth types.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.