Skip to content

fix(model): refuse to remove a model/version still referenced by an endpoint (STO-360) - #334

Merged
jebenexer merged 1 commit into
mainfrom
benjaminbrannaka/sto-360-warn-about-dependent-endpoints-before-removing-model-repo
Sep 2, 2026
Merged

fix(model): refuse to remove a model/version still referenced by an endpoint (STO-360)#334
jebenexer merged 1 commit into
mainfrom
benjaminbrannaka/sto-360-warn-about-dependent-endpoints-before-removing-model-repo

Conversation

@jebenexer

Copy link
Copy Markdown
Collaborator

STO-360: https://linear.app/runpod/issue/STO-360/warn-about-dependent-endpoints-before-removing-model-repo-versions

  • model remove now checks whether any serverless endpoint still references the model (or, with --hash/--version, that exact version) before removing it.
  • If a dependent endpoint is found, the removal is blocked with a dependent_endpoints coded error naming the endpoint(s) — there is no override flag. A model/version in active use cannot be removed from the CLI.
  • The fix path is spelled out in the error: detach it first with runpodctl serverless update <endpoint-id> --clear-models (or point it at a replacement with --model-reference <new-url>), then retry.
  • The check queries GraphQL (myself { endpoints { modelReferences } }) rather than the REST endpoints list, since REST never returns modelReferences (verified live — it's simply absent from the wire shape).
  • Matching is case-insensitive on the https://local/ scheme+host, since the backend persists it as https://LOCAL/... regardless of the case runpodctl emits it in (verified on both prod and RIAB).
  • If the dependent-endpoint check itself fails (e.g. can't reach the API), the removal is blocked too — "can't verify it's safe" is treated the same as "not safe."

Also fixes a real, unrelated bug found while e2e testing this against a live endpoint: UpdateEndpointModels (backing serverless update --model-reference/--clear-models — the exact command this PR's error message tells users to run) always sent flashBootType: "" because REST GET /v1/endpoints/{id} returns a flashboot bool, never the flashBootType enum saveEndpoint expects, so every call failed enum validation with Value "" does not exist in "FlashBootType" enum.. Now derives the enum from the REST bool when the string is empty, same as serverless create already does.

Known, separate issue not fixed here: UpdateEndpointModels also drops GPU config — REST returns gpuTypeIds (array), but the struct only reads gpuIds (singular string), so saveEndpoint then fails with "gpuId(s) is required for a gpu endpoint" on any real GPU endpoint. This means serverless update --model-reference/--clear-models is still broken today for GPU endpoints created via serverless create --gpu-id. Worth its own follow-up ticket.

Test plan:

  • go test ./...
  • gofmt/go vet clean
  • Live e2e verified against prod (api.runpod.io) and RIAB: blocked while a dependent endpoint exists (model/version untouched), proceeds cleanly once no endpoint references it, version-specific removal only blocks on the exact referenced hash (a different, unreferenced version removes cleanly), --force correctly rejected as unknown flag (no override exists).
  • docs/runpodctl_model_remove.md regenerated for the new behavior/examples.

…ndpoint

STO-360: model remove now checks whether any serverless endpoint
still references the model (or, with --hash/--version, that exact
version) before removing it. If so, the removal is blocked with a
dependent_endpoints error naming the endpoint(s) -- there is no
override; a model/version in active use cannot be removed. Detach it
first with 'serverless update --clear-models' (or point it at a
replacement with --model-reference), then retry.

The check queries GraphQL (myself { endpoints { modelReferences } })
rather than the REST endpoints list, since REST never returns
modelReferences. Matching is case-insensitive on the
https://local/ scheme+host, since the backend persists it as
https://LOCAL/... regardless of the case runpodctl emits it in.

Also fixes a bug found while e2e testing this against a real
endpoint: UpdateEndpointModels (serverless update --model-reference /
--clear-models) always sent flashBootType: "" because REST GET
/v1/endpoints/{id} returns a flashboot bool, never the flashBootType
enum saveEndpoint expects, so every call failed enum validation.
Derives the enum from the REST bool now.

Verified live against prod and RIAB: blocked with dependents present,
proceeds cleanly with none, and version-specific removal only blocks
on the exact referenced hash.
@jebenexer
jebenexer merged commit b6f4932 into main Sep 2, 2026
1 check passed
@jebenexer
jebenexer deleted the benjaminbrannaka/sto-360-warn-about-dependent-endpoints-before-removing-model-repo branch September 2, 2026 18:44
@promptless

promptless Bot commented Sep 2, 2026

Copy link
Copy Markdown

Promptless documentation updates

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants