Skip to content

Optimize GLM-5.2-MXFP4 SGLang Agentic Performance on MI355X - #2887

Closed
limou102 wants to merge 2 commits into
SemiAnalysisAI:mainfrom
limou102:dev/limou/glm5.2-sglang-opt-0908
Closed

Optimize GLM-5.2-MXFP4 SGLang Agentic Performance on MI355X#2887
limou102 wants to merge 2 commits into
SemiAnalysisAI:mainfrom
limou102:dev/limou/glm5.2-sglang-opt-0908

Conversation

@limou102

@limou102 limou102 commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Update the GLM-5.2-MXFP4 MI355X AgentX SGLang image from v0.5.16 to v0.5.19.
  • Pick up recent SGLang main-branch optimizations.
  • Restore write_through as the default HiCache write policy.

Benchmark Results

Test configuration: MI355X, TP4/EP4, MTP, HiCache DRAM offload, 3600-second AgentX profiling.

Concurrency P90 Interactivity Throughput per Chip
10 66.92 tokens/s/user 13,002.49 tokens/s/GPU
12 57.87 tokens/s/user 12,340.00 tokens/s/GPU

Concurrency 10 provides the best balance of output interactivity and per-GPU throughput.

Validation

  • Both benchmark runs completed successfully with no profiling request errors.
  • Bash syntax, YAML parsing, and targeted matrix generation passed.

Note

Low Risk
Benchmark and config-only changes (container pin and HiCache default); no auth or core application logic.

Overview
Updates the glm5.2-fp4-mi355x-sglang-agentic-mtp recipe to SGLang ROCm v0.5.19 (20260907) from v0.5.16, picking up recent main-branch GLM-5.2-MXFP4 serving work.

In glm5.2_fp4_mi355x_sglang_mtp.sh, the default HiCache write policy switches from write_through_selective back to write_through as the validated baseline for AgentX interactivity and per-GPU throughput; HICACHE_WRITE_POLICY can still be set explicitly for selective-write experiments.

perf-changelog.yaml documents the image bump and write-policy change for this config key.

Reviewed by Cursor Bugbot for commit 5361b95. Bugbot is set up for automated code reviews on this repo. Configure here.

@limou102
limou102 marked this pull request as ready for review September 8, 2026 11:14
@limou102
limou102 requested a review from a team September 8, 2026 11:14

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Dispatched trusted sweep run for approved external revision 5361b955f8451e45f23b23a9fdc4533c35898628.
New commits are not trusted automatically; remove and re-add the primary sweep label to approve a new SHA.

已为获批的外部提交 5361b955f8451e45f23b23a9fdc4533c35898628 调度trusted sweep run
后续新提交不会自动获得信任;如需批准新的 SHA,请移除并重新添加主扫描标签。

@limou102
limou102 force-pushed the dev/limou/glm5.2-sglang-opt-0908 branch from 5361b95 to c43c954 Compare September 9, 2026 02:17
@limou102 limou102 closed this Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

4 participants