[Klaud Cold] qwen3.5-fp8-mi355x-sglang-agentic-mtp: day-zero Qwen3.5 FP8 SGLang AgentX MTP on MI355X / MI355X 上 Qwen3.5 FP8 SGLang AgentX MTP 首发配方

[functionstackx]

Summary

Day-zero Qwen3.5-397B-A17B FP8 AgentX (agentic-coding) recipe on MI355X with SGLang native EAGLE MTP: qwen3.5-fp8-mi355x-sglang-agentic-mtp, the FP8 precision sibling of qwen3.5-fp4-mi355x-sglang-agentic-mtp.

• Checkpoint: upstream Qwen/Qwen3.5-397B-A17B-FP8 (~406 GB), the same one the MI325X and MI300X FP8 SGLang AgentX recipes serve.
• Script: benchmarks/single_node/agentic/qwen3.5_fp8_mi355x_sglang_mtp.sh. SKU serve shape is the MXFP4 MI355X script unchanged (AITER unified attention, INT8 quick all-reduce, fp8_e4m3 KV, 16k prefill budget, mem-fraction 0.80, graph capture min(2*CONC, 128), HiCache ratio 1.5). FP8 deltas from the MI325X FP8 sibling: --quantization fp8 and --mamba-ssm-dtype bfloat16. AITER_FLYDSL_FORCE (MXFP4 GEMM path) is not carried. Golden AL 3.39 at K=3.
• Image: lmsysorg/sglang-rocm:v0.5.19-rocm720-mi35x-20260907 (verified on Docker Hub, pushed 2026-09-07T14:35:05Z, digest sha256:1a31d1ff5f6b4afee1866d72c0610445c629fa3b0c056f372f843ef839dbd8be), same flavor as the MXFP4 sibling and the GLM-5.2 FP8 MI355X recipe in [Klaud Cold] glm5.2-fp8-mi355x-sglang-agentic-mtp: day-zero GLM-5.2 FP8 SGLang AgentX MTP on MI355X / MI355X 上 GLM-5.2 FP8 SGLang AgentX MTP 首发配方 #2867.
• Search space (22 cells): TP4/EP1 GPU-resident [1, 4, 8, 12, 16, 20, 24, 28, 32, 40] mirrors the MXFP4 TP4 arm; TP8/EP1 GPU-resident [1, 4, 8, 16, 24, 32, 40, 48] is the MI325X FP8 TP8 arm trimmed; TP4/EP1 HiCache [40, 48, 56, 64] mirrors the MXFP4 host-tier arm. No TP2 on day zero: ~203 GB/rank of weights would leave ~27 GB/rank inside 0.80 of 288 GB.
• Launcher: no change needed; runners/launch_mi355x-amds.sh already resolves qwen3.5_fp8_mi355x_sglang_mtp.sh.

Validation: bash -n, both YAMLs parse, generate_sweep_configs.py test-config yields the expected 22 cells across the three arms, launcher script-name resolution confirmed.

Recipes touched: qwen3.5-fp8-mi355x-sglang-agentic-mtp

Test plan

• full-sweep-enabled sweep passes on cluster:mi355x-amds (22 cells).

🤖 Generated with Claude Code

---

Note

Low Risk
Benchmark and YAML configuration only; no application runtime or auth changes. Residual risk is cluster sweep validation and serve-flag parity with sibling recipes.

Overview
Adds the first Qwen3.5 FP8 AgentX (agentic-coding) benchmark on MI355X: config key qwen3.5-fp8-mi355x-sglang-agentic-mtp and launcher script benchmarks/single_node/agentic/qwen3.5_fp8_mi355x_sglang_mtp.sh, aligned with the existing MXFP4 MI355X agentic recipe and FP8 tweaks from the MI325X sibling (--quantization fp8, --mamba-ssm-dtype bfloat16, upstream Qwen/Qwen3.5-397B-A17B-FP8).

The script runs SGLang with native EAGLE MTP, AITER unified attention, INT8 quick all-reduce, fp8 KV, optional HiCache host tier, and throughput replay pinned to golden acceptance length 3.39 (evals use real verification). configs/amd-master.yaml defines a 22-cell sweep: TP4 and TP8 GPU-resident concurrency lists plus TP4 HiCache offload; TP2 is omitted on day zero for memory. Image is bumped to lmsysorg/sglang-rocm:v0.5.19-rocm720-mi35x-20260907. perf-changelog.yaml documents the new recipe.

^{Reviewed by Cursor Bugbot for commit 4a5db0a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[claude]

Looks good, straightforward day-zero recipe addition mirroring two existing siblings.

What was reviewed: diffed the new script against qwen3.5_fp4_mi355x_sglang_mtp.sh (SKU shape) and qwen3.5_fp8_mi325x_mtp.sh (FP8 deltas) — the claimed unified-attention/quick-allreduce/KV-dtype parity and --quantization fp8/--mamba-ssm-dtype bfloat16 deltas match exactly. Confirmed the *_mtp.sh "must call run_benchmark_serving with --use-chat-template" invariant doesn't apply here since every existing agentic *_mtp.sh script in the repo (not just this one) uses the build_replay_cmd/--apply-chat-template pattern instead. Checked perf-changelog.yaml diff is a pure tail append and configs/amd-master.yaml new block follows existing kebab-case/search-space conventions.

Extended reasoning...

Overview

The PR adds one new executable benchmark launcher script (benchmarks/single_node/agentic/qwen3.5_fp8_mi355x_sglang_mtp.sh), a corresponding recipe block in configs/amd-master.yaml, and a single appended entry in perf-changelog.yaml. It is the FP8 sibling of an already-merged MXFP4 MI355X agentic recipe, itself borrowing FP8-specific flags from an already-merged MI325X FP8 sibling.

Security risks

None. This is benchmark configuration/orchestration code — a bash launcher for sglang.launch_server with static, hardcoded flags and env-var-driven parameters, plus two YAML config additions. There is no user input handling, auth, crypto, or externally reachable surface being modified.

Level of scrutiny

Diffing the new script line-by-line against its two named siblings (qwen3.5_fp4_mi355x_sglang_mtp.sh for the SKU-level serve shape, qwen3.5_fp8_mi325x_mtp.sh for the FP8-specific deltas) confirmed every claimed inheritance and delta is accurate: AITER unified attention, INT8 quick all-reduce, fp8_e4m3 KV cache, 16k prefill/chunked-prefill budgets, mem-fraction 0.80, HiCache ratio 1.5, and the two FP8 deltas (--quantization fp8, --mamba-ssm-dtype bfloat16) all match what the comments say, and AITER_FLYDSL_FORCE/--enable-aiter-allreduce-fusion are correctly omitted per the stated rationale. The configs/amd-master.yaml search-space arms (TP4/TP8 GPU-resident, TP4 HiCache) are consistent with the sibling arms they claim to mirror/trim, and the perf-changelog.yaml diff is a clean tail append with no reflow of prior entries.

Other factors

The one loose end — pr-link: PRLINK_PLACEHOLDER in the changelog entry — was already flagged in the supplied conventions notes as expected to be resolved before merge, consistent with the placeholder pattern documented in CONTRIBUTING.md, so it isn't a new concern to raise. No findings were reported by the bug hunter, and my own targeted diffing against both named sibling scripts found no discrepancies, so this is a low-risk, mechanical, well-documented recipe addition that a human does not need to block on.

This review covers commit c963bda, which is no longer the latest commit on this pull request; later commits are not covered by it.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34174044289
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=34174044289