Skip to content

[Example] Fix TileKernels Benchmarking - #162

Open
WilliamZhang20 wants to merge 4 commits into
NVIDIA:mainfrom
WilliamZhang20:tilekernels-fix
Open

[Example] Fix TileKernels Benchmarking#162
WilliamZhang20 wants to merge 4 commits into
NVIDIA:mainfrom
WilliamZhang20:tilekernels-fix

Conversation

@WilliamZhang20

Copy link
Copy Markdown
Contributor

List of new kernels:

  • Per-channel cast
  • Top-k
  • mHC merge

New benchmarks:

Example Shape Tilus (ms) TileKernels (ms)
Top-k 128 tokens, 72 experts, k=6 0.0065 0.0065
Top-k 1024 tokens, 256 experts, k=8 0.0084 0.0079
Top-k 8192 tokens, 256 experts, k=8 0.0178 0.0154
Per-token cast 128 × 1024 0.005824 0.006304
Per-token cast 256 × 2048 0.006208 0.006656
Per-token cast 257 × 4096 0.007008 0.007328
Per-token cast 8192 × 8192 0.073440 0.074688
SwiGLU + cast 128 × 1024 0.006080 0.030112
SwiGLU + cast 256 × 2048 0.007168 0.007552
SwiGLU + cast 257 × 4096 0.008288 0.009088
SwiGLU + cast 1024 × 4096 0.014304 0.015040
SwiGLU + cast 4096 × 8192 0.063392 0.064128
Per-channel cast 8192 × 8192 0.0797 0.0760
mHC merge 24 × 4096 0.0052 0.0052

Signed-off-by: William Zhang <wzhang20@yahoo.com>
Signed-off-by: William Zhang <wzhang20@yahoo.com>
Signed-off-by: William Zhang <wzhang20@yahoo.com>
Signed-off-by: William Zhang <wzhang20@yahoo.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 13, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant