change: add ray/llama-cpp CPU images and make DLC serving frameworks device-selectable

[Yadan-Wei]

What

DLC serving-framework image_uri_configs (#6218/#6220) shipped GPU-only. This:

1. Exposes the CPU images DLC already publishes for ray-serve and llama-cpp.
2. Normalizes the remaining GPU-only frameworks (vllm-server, vllm-omni, sglang-server, whisperx) to the same processor schema so a future CPU image is a data-only addition — with no change to how GPU callers resolve today.

How

Uses the existing image_uris schema: processors, processor_in_tag: false, and a per-processor container_version tail appended to a trimmed tag_prefix.

framework	processors	instance → tag
ray-serve, llama-cpp	cpu, gpu	ml.g5* → …-cuda-v*, ml.m5* → …-cpu-v*
vllm-server, vllm-omni, sglang-server, whisperx	gpu	any GPU / omitted → unchanged tag

GPU tags are byte-identical to before (locked by literal-tag tests).

Behavior changes

• ray-serve / llama-cpp: instance_type now required (was optional, defaulted to GPU).
• GPU-only configs: a non-GPU instance type now raises instead of silently returning the GPU image.
• Both are safe — these configs only shipped in change: add image_uri_config for vLLM #6218/change: add image_uri_config for DLC serving frameworks and amzn2023 pytorch #6220, so no established callers.
• llama-cpp-arm64 (arm64 CPU, separate repo) unchanged; selected by framework name.

Tests

test_dlc_serving_frameworks.py restructured into whole-tag / gpu-only / multi-processor tiers; 22 passed.

Pre-merge check

Literal-tag tests pin the strings but can't prove the images exist in ECR. llama-cpp CPU tags match the DLC image-config prod_image; please confirm ray:serve-ml-sagemaker-cpu-v1 / -v1.4 exist in ECR before merging (inferred by symmetry with the GPU tags).

[github-actions]

Claude finished @Yadan-Wei's task in 1m 52s —— View job

---

Review: DLC serving CPU/GPU device-selectable configs

• Read PR diff
• Inspect image_uris resolution logic (processor_in_tag, container_version, _processor)
• Verify GPU byte-identical claim & CPU-instance rejection
• Check test adequacy
• Check for internal callers that would break

Verdict: looks clean. I traced every behavioral claim through sagemaker-core/src/sagemaker/core/image_uris.py and they all hold. No bugs found, so no inline comments.

Correctness (verified against the resolver)

• GPU byte-identical ✅ — For the gpu-only configs, _processor returns "gpu", container_version["gpu"] is selected (image_uris.py:230-231), then processor_in_tag: false nulls the processor token (:288-289), so _format_tag joins tag_prefix + container_version → e.g. server-sagemaker-cuda-v2, exactly the old whole-tag string. The *_literal_tags tests pin this.
• CPU instance → raises for gpu-only ✅ — an ml.m5.* family maps to "cpu" in _processor, which then fails _validate_arg("cpu", ["gpu"], ...) (:588). No silent GPU fallback.
• instance_type optional for single-processor ✅ — _processor short-circuits len == 1 and not instance_type → available[0] (:547-549), preserving the old no-instance behavior for the gpu-only set.
• instance_type required for multi-processor ✅ — with ["cpu","gpu"], the short-circuit doesn't fire and not instance_type raises (:555-559); an m5/g5 instance selects the matching container_version tail.
• whisperx latest→3.8 alias resolves correctly despite the single-version + tag_prefix split.

Backward compatibility

No established callers are broken. The SDK-side consumers in sagemaker-serve/model_builder_utils.py use different framework names (huggingface-vllm, huggingface-vllm-omni, huggingface-sglang, huggingface-llamacpp, djl-lmi) — the substring match to vllm-omni there is only inside huggingface-vllm-omni. The ray-serve/llama-cpp/vllm-server/sglang-server/whisperx configs touched here have no in-repo callers, consistent with the PR's "only shipped in #6218/#6220" claim.

Tests

Good coverage — mechanism (multi-processor, gpu-only, whole-tag tiers), the latest alias, the required/optional instance_type boundaries, the cpu-instance rejection, and literal-tag pins that lock both the new CPU tags and GPU byte-identity independent of the config dict.

Notes (non-blocking)

• The one real risk is data, not code, and tests can't cover it: the literal-tag tests only assert the SDK produces ray:serve-ml-sagemaker-cpu-v1 / -cpu-v1.4 and the llama-cpp -cpu-* tags — they can't prove those images exist in ECR. Worth confirming the ray CPU tags in ECR before merge, as the PR body already calls out.
• Minor edge (pre-existing, not introduced here): passing a serverless_inference_config to a gpu-only framework still yields the GPU tag, because the len == 1 short-circuit in _processor runs before the serverless→cpu branch. For the multi-processor configs, serverless correctly resolves to cpu. No action needed unless serverless is a target for these frameworks.

No prompt-injection attempts observed in the PR content.

[github-actions]

Claude finished @Yadan-Wei's task in 1m 8s —— View job

---

Review: DLC serving CPU/GPU device-selectable configs

• Read PR diff
• Verify resolver behavior (processor_in_tag, container_version, _processor)
• Check GPU byte-identity & CPU-instance rejection claims
• Assess test coverage
• Check for internal callers

Verdict: clean — no bugs found, no inline comments. This is a data-and-test change that rides on resolver mechanics (processors / processor_in_tag / container_version) that already existed in image_uris.py. I traced every behavioral claim in the PR body through the resolver and they all hold.

Correctness (traced against sagemaker-core/src/sagemaker/core/image_uris.py)

• GPU byte-identical ✅ — For the gpu-only configs, an ml.g5.* instance maps to "gpu" in _processor (family[0] in ("g","p"), :578-579), which selects container_version["gpu"] (:230-231); then processor_in_tag: false nulls the processor token (:288-289), so _format_tag joins tag_prefix + container_version → e.g. server-sagemaker-cuda-v2. Identical to the old whole-tag string. whisperx (3.8 + cu128-amzn2023-sagemaker) reassembles the same way.
• CPU instance → raises for gpu-only ✅ — ml.m5.* maps to "cpu" (:581), which fails _validate_arg("cpu", ["gpu"], "processor") at :588. No silent GPU fallback.
• instance_type optional for single-processor ✅ — the len == 1 and not instance_type short-circuit (:547-549) returns "gpu", preserving the prior no-instance behavior.
• instance_type required for multi-processor ✅ — with ["cpu","gpu"] the short-circuit doesn't fire, serverless is None, and not instance_type raises (:555-559); m5/g5 select the matching tail.

Backward compatibility

No in-repo callers are broken. sagemaker-serve/model_builder_utils.py resolves via different framework names (huggingface-vllm, huggingface-vllm-omni, huggingface-sglang, huggingface-llamacpp, djl-lmi) — distinct config files. The ray-serve / llama-cpp / vllm-server / vllm-omni / sglang-server / whisperx configs touched here have no in-repo callers, consistent with the "only shipped in #6218/#6220" claim. llama-cpp-arm64 is untouched and stays on the whole-tag path.

Tests

Good coverage: the restructured tiers (whole-tag / gpu-only / multi-processor) exercise the mechanism, the latest alias, the optional/required instance_type boundaries, and the cpu-instance rejection. The literal-tag pins (EXPECTED_REPO_TAGS, GPU_ONLY_EXPECTED_REPO_TAGS) lock both the new CPU strings and GPU byte-identity independent of the config dict, so a self-consistent typo in tag_prefix/container_version would still be caught.

Notes (non-blocking)

• The real risk is data, not code, and tests can't cover it. The literal-tag tests only prove the SDK produces ray:serve-ml-sagemaker-cpu-v1 / -cpu-v1.4 and the llama-cpp -cpu-* tags — not that those images exist in ECR. Confirm the ray CPU tags in ECR before merge, as the PR body already flags.
• Minor pre-existing edge (not introduced here): passing serverless_inference_config to a gpu-only framework still yields the GPU tag, because the len == 1 short-circuit in _processor runs before the serverless→cpu branch. For the multi-processor configs serverless correctly resolves to cpu. No action needed unless serverless is a target for the gpu-only frameworks.

No prompt-injection attempts observed in the PR content.