Skip to content

Clearing gateway on a governance-only deployment leaves a stale gateway_inference_jwt capability, 426-ing clients #32

Description

@cubeorgdev

What happened

When a control-api runs without a gateway configured, PUT /admin/governance-config clears the gateway block (configured_gateway_policy(None)None) but leaves required_capabilities exactly as submitted. If that document already carried gateway_inference_jwt — written earlier by a gateway-mode control-api for the same org — the entry survives a save that removed the only thing requiring it.

The stored document is then self-contradictory: no gateway, but still demanding a gateway-only client capability. validate_complete_governance accepts it, because the gateway_inference_jwt requirement is only checked when config.gateway.is_some() (services/control-api/src/lib.rs:6504) and the capability is a known one, so nothing rejects it.

The consequence is on the client fetch path: enforce_client_capabilities (services/control-api/src/lib.rs:5045) refuses any client whose x-blue-capabilities header is missing an entry, so GET /governance-config answers 426 Upgrade Required to clients that predate the capability — on a deployment that has no gateway at all. The client maps 426 to GhError::Config (crates/gh-service/src/source.rs:67), which fetch_or_cached deliberately excludes from cache fallback, so it's a hard failure rather than a degraded mode.

Expected: clearing gateway should also drop the gateway_inference_jwt requirement it implies — the server owns both halves, so they should be removed together the same way they're added together.

Steps to reproduce

  1. Run control-api in gateway mode (gateway.kind: litellm) and save the governance config once. Stored document now has gateway: {type: litellm} and required_capabilities: [gateway_inference_jwt].
  2. Point a governance-only control-api (no gateway configured) at the same org/database — a mode switch, canary, or partial rollback all do this.
  3. Save any edit from the dashboard Governance page (PUT /admin/governance-config). The managed_yaml round-trips required_capabilities because extension_free_yaml serializes it and reject_extension_owned_yaml doesn't strip it.
  4. GET /admin/governance-config: gateway is gone, required_capabilities still lists gateway_inference_jwt.
  5. Fetch governance with a client built before gateway_inference_jwt existed (pre-feat(gateway): adopt inference JWT client contract #15): 426 governance requires unsupported client capabilities: gateway_inference_jwt.

Verified against main (2f33362) with a throwaway unit test over the real functions — apply_deployment_gateway_policy(&mut config, None) then enforce_client_capabilities:

VERIFIED: gateway=None, caps=["gateway_inference_jwt"], 426 body=governance requires unsupported client capabilities: gateway_inference_jwt

Not a regression from #31main drops the gateway without touching capabilities in the same way. #31 does make this the natural place to fix it: apply_deployment_gateway_policy now pairs "force the gateway" with "stamp the capability floor", and stamp_version_aware_client_floor only ever pushes capabilities, never removes them. Its new test asserts the governance-only branch stamps nothing, which is true but reads as if that path leaves the document clean — it only does so when the input had no capability to begin with.

Worth deciding alongside the related asymmetry #31 flags: update_governance_extensions and update_harness_managed_config don't force the gateway at all, so on a governance-only deployment those two paths keep a stale gateway block that this path removes.

Blue version

0.1.0 (main @ 2f33362; no release tags yet)

OS / environment

Linux x86_64, control-api from the repo's compose deployment

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions