fix(model): refuse to remove a model/version still referenced by an endpoint (STO-360) - #334
Merged
jebenexer merged 1 commit intoSep 2, 2026
Conversation
…ndpoint
STO-360: model remove now checks whether any serverless endpoint
still references the model (or, with --hash/--version, that exact
version) before removing it. If so, the removal is blocked with a
dependent_endpoints error naming the endpoint(s) -- there is no
override; a model/version in active use cannot be removed. Detach it
first with 'serverless update --clear-models' (or point it at a
replacement with --model-reference), then retry.
The check queries GraphQL (myself { endpoints { modelReferences } })
rather than the REST endpoints list, since REST never returns
modelReferences. Matching is case-insensitive on the
https://local/ scheme+host, since the backend persists it as
https://LOCAL/... regardless of the case runpodctl emits it in.
Also fixes a bug found while e2e testing this against a real
endpoint: UpdateEndpointModels (serverless update --model-reference /
--clear-models) always sent flashBootType: "" because REST GET
/v1/endpoints/{id} returns a flashboot bool, never the flashBootType
enum saveEndpoint expects, so every call failed enum validation.
Derives the enum from the REST bool now.
Verified live against prod and RIAB: blocked with dependents present,
proceeds cleanly with none, and version-specific removal only blocks
on the exact referenced hash.
rutvik-runpod
approved these changes
Sep 2, 2026
jebenexer
deleted the
benjaminbrannaka/sto-360-warn-about-dependent-endpoints-before-removing-model-repo
branch
September 2, 2026 18:44
Promptless documentation updates
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
STO-360: https://linear.app/runpod/issue/STO-360/warn-about-dependent-endpoints-before-removing-model-repo-versions
model removenow checks whether any serverless endpoint still references the model (or, with--hash/--version, that exact version) before removing it.dependent_endpointscoded error naming the endpoint(s) — there is no override flag. A model/version in active use cannot be removed from the CLI.runpodctl serverless update <endpoint-id> --clear-models(or point it at a replacement with--model-reference <new-url>), then retry.myself { endpoints { modelReferences } }) rather than the REST endpoints list, since REST never returnsmodelReferences(verified live — it's simply absent from the wire shape).https://local/scheme+host, since the backend persists it ashttps://LOCAL/...regardless of the case runpodctl emits it in (verified on both prod and RIAB).Also fixes a real, unrelated bug found while e2e testing this against a live endpoint:
UpdateEndpointModels(backingserverless update --model-reference/--clear-models— the exact command this PR's error message tells users to run) always sentflashBootType: ""because RESTGET /v1/endpoints/{id}returns aflashbootbool, never theflashBootTypeenumsaveEndpointexpects, so every call failed enum validation withValue "" does not exist in "FlashBootType" enum.. Now derives the enum from the REST bool when the string is empty, same asserverless createalready does.Known, separate issue not fixed here:
UpdateEndpointModelsalso drops GPU config — REST returnsgpuTypeIds(array), but the struct only readsgpuIds(singular string), sosaveEndpointthen fails with"gpuId(s) is required for a gpu endpoint"on any real GPU endpoint. This meansserverless update --model-reference/--clear-modelsis still broken today for GPU endpoints created viaserverless create --gpu-id. Worth its own follow-up ticket.Test plan:
go test ./...gofmt/go vetcleanapi.runpod.io) and RIAB: blocked while a dependent endpoint exists (model/version untouched), proceeds cleanly once no endpoint references it, version-specific removal only blocks on the exact referenced hash (a different, unreferenced version removes cleanly),--forcecorrectly rejected asunknown flag(no override exists).docs/runpodctl_model_remove.mdregenerated for the new behavior/examples.