Skip to content

moe_simd_silu_gate_up_q4_sbf16 kernel missing from Metal metallib (Qwen3-Coder MoE on Apple Silicon) #55

Description

@sandeepraju

Environment:

  • BaseRT: 0.2.3
  • Hardware: Apple M4 Pro, 48 GB
  • Model: Qwen/Qwen3-Coder-30B-A3B-Instruct (converted via basert pull --target default-q4)

Error (repeated per request):
get_pipeline: no function named 'moe_simd_silu_gate_up_q4_sbf16' in metallib

Steps to reproduce:

  1. basert pull Qwen/Qwen3-Coder-30B-A3B-Instruct
  2. basert serve Qwen/Qwen3-Coder-30B-A3B-Instruct --port 8081
  3. Send any chat/completions request

Notes:
The 0.2.0 changelog mentions "fused SiLU gate/up for 4-bit experts" was added for CUDA.
It appears the Metal metallib doesn't include the equivalent moe_simd_silu_gate_up_q4_sbf16 kernel.
The model converts successfully but fails at inference time.


Additional context: The model was converted via basert pull Qwen/Qwen3-Coder-30B-A3B-Instruct (convert-on-pull from raw HuggingFace safetensors, not a pre-built catalog model). Conversion completed without errors and chose the default-q4 profile. The kernel name moe_simd_silu_gate_up_q4_sbf16 suggests it selected a sbf16-scale variant that only exists in the CUDA metallib. It may be worth checking whether the convert-on-pull profile selector is Metal-aware when picking MoE quant variants.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions