Conversation
|
@Nulliik is attempting to deploy a commit to the Community Labs Team on Vercel. A member of the Team first needs to authorize it. |
- results/nulliik/qwen3-8-27b-llamacpp-sycl-q4_k_l-2xb60.json: imported (result 85) Merge ingestion completed. Rerunning this workflow will not duplicate these results. |
|
Hey, thinking through some stuff for this one - notably whether we want to add Swift-Qwen3.8-27B as a separate model, or submit this as is and just keep it as a note. Will get back to you soon :) |
|
Hello, this model is similar to Qwen3.8-27B actually, shouldn't be different in perfomance. I can rerun same test with default model. Also I tested bigger quant Q4_K_L which is missing on website currently. Can lower mine and rerun it or either it has to be added to website. |
|
Okay, no problem, this is fine as it is. I'll merge it over the weekend. |
|
Also going to add the quant to the catalog specified by that model ( |
2 participants
Benchmark submission for Rig 4: Maxsun ARC Pro B60 Dual 48G Turbo (SYCL F16 JIT + Native MTP)
This PR submits measured results for Qwen3.8-27B (
qwen3-8-27b) on dual Intel Arc Pro B60 using a custom JIT build of llama.cpp with the Intel SYCL backend, native FP16 compute kernels, and native embedded Q8_0 Multi-Token Prediction (MTP) draft head:4(Maxsun ARC Pro B60 Dual 48G Turbo)qwen3-8-27b)q4_k_mboard (Swift-Qwen3.8-27B-Q4_K_L.gguf, 17.72 GiB)llamacppupstream master buildb11100(7ab4ee7b)icx/icpx) + MSVC v143 + Ninja-DGGML_SYCL=ON -DGGML_SYCL_F16=ON -DGGML_SYCL_DNN=ON -DGGML_SYCL_TARGET=INTEL -DCMAKE_BUILD_TYPE=Release)SYCL_DEVICE_FILTER=level_zero,SYCL_CACHE_PERSISTENT=0Primary Leaderboard Submission:
results/nulliik/qwen3-8-27b-llamacpp-sycl-q4_k_m-2xb60.jsondecodeTps): 24.00 tok/s (Native embedded Q8_0 MTP draft speculative decoding,--spec-type draft-mtp --spec-draft-device SYCL0 --spec-draft-n-max 3)promptTps): 609.47 tok/s (pp512 on Q8_0 KV cache, scaling up to 925.39 tok/s)ttftMs): 840.1 ms (cold-input evaluation at p512)contextLength): 131072 (full 128k context window supported in VRAM)Comparative Benchmark & Analysis: Dual Intel Arc Pro B60
All tests were performed on Windows 11 Pro with dual Intel Arc Pro B60 24GB (48GB total VRAM).
-sm layer -dev SYCL0/SYCL1 -ts 1.25/0.75(balanced across cards: ~20.8 GB GPU0 / ~21.4 GB GPU1)-fa on), batch size 4096, ubatch size 4096 (-b 4096 -ub 4096)1. Token Generation / Decode Scaling (
decodeTps):llama-serverChat with Q8_0 MTPllama-bench, no MTP)2. Q8_0 KV Cache Performance (Full 128k Context Sweep):
Configuration:
-ctk q8_0 -ctv q8_0 -b 4096 -ub 4096 -ts 1.25/0.75(matches production server profile)3. FP16 KV Cache Performance (Full Precision KV Sweep):
Configuration:
-ctk f16 -ctv f16 -b 4096 -ub 4096 -ts 1.25/0.75(maximum context: 65,536 tokens within 48GB VRAM)4. Side-by-Side: FP16 KV vs Q8_0 KV vs Vulkan:
Key Engineering Takeaways:
draft-mtp) runs fully in VRAM with zero external model overhead, boosting token generation from 16.13 tok/s to 24.00 tok/s (+48.9%), with high acceptance rates (85%–100%) in real conversations.-DGGML_SYCL_F16=ONwith oneDNN eliminates the FP32 execution bottleneck, delivering over 3x prefill speedup over older FP32 SYCL builds and outpacing Vulkan by up to 4.86x on long contexts.-DGGML_SYCL_DEVICE_ARCH=bmg-g21) under Windows causes access violation crashes (0xC0000005), whereas runtime JIT with Level-Zero driver is completely stable.-ts 1.25/0.75distributes memory evenly between primary and secondary cards, preventing GPU1 from exhausting memory while preserving full interconnect throughput.Acknowledgments
Special thanks to @SergiioB for the Intel Arc Pro B70 inference cookbook for key JIT compilation and runtime insights.
Validated locally with:
npm --prefix frontend run results:validate -- --author Nulliik ../results/nulliik/qwen3-8-27b-llamacpp-sycl-q4_k_m-2xb60.json