fix(qwen3_5_moe): accept a per-channel fp8 weight_scale in the dense loader

[salekseev]

Hit this while trying to load a compressed-tensors FP8 checkpoint that uses per-channel weight scales. _per_row_scale assumes the scale is always a single scalar:

return scalar.reshape(1).to(torch.float32).expand(rows)

That's right for strategy: "tensor", but llm-compressor also emits strategy: "channel" -- one scalar per output row, stored as [rows, 1]. On that shape the reshape blows up:

RuntimeError: shape '[1]' is invalid for input of size 248320

Since Fp8PerTensorLinear.weight_scale is already declared per output row, there's nothing to change downstream -- a genuine per-tensor weight just stores the same scalar in every row. So this reshapes to [-1] and branches on the element count: 1 broadcasts exactly as before, rows passes straight through.

The part I'd push back on if I were reviewing: the third branch raises instead of doing something lenient. That's deliberate. If a mis-shaped scale broadcast row 0's factor across all 248320 output rows, the model would load happily and generate fluent nonsense. I'd much rather it fail at load with a message naming both counts.

Reachability, since it affects how you want to review this

You can't trigger this on main today. The only checkpoints that reach _iter_weights_attn_fp8 are ModelOpt ones, and those use a per-tensor scale. So on main this is hardening plus a prerequisite -- it becomes load-bearing once a per-channel checkpoint can actually route there, which needs the compressed-tensors detection from #390 plus a one-token widening to accept "channel".

I mention it so you can weigh it as "small safe change that unblocks a checkpoint class" rather than "fixes a bug users are hitting", because the latter would be overselling it.

Tests

Adds tests/models/test_qwen3_5_moe_weight.py, modelled on tests/models/test_glm5_next_config.py. Covers both on-disk granularities, fp32 promotion from every storage dtype, and the mismatch raising. The per-channel case asserts row order with distinct values rather than a sum or a set -- permuting the scales would keep every aggregate identical while silently scaling each row by the wrong factor.

There were no test_qwen3_5_moe_* files before this, so nothing is replaced.

Testing done

Verified against unsloth/Qwen3.6-35B-A3B-NVFP4-Fast and nvidia/Qwen3.6-35B-A3B-NVFP4 on an RTX 4080 SUPER (16 GiB, sm_89, TP=1). Full background and measurements in #252.

[salekseev]

Status note, since #418/#427 landed after I opened this.

This one still applies -- _per_row_scale on main is byte-for-byte the original one-liner, so the bug is untouched and the PR merges clean. But the area around it has moved, and I'd rather you decide than have me quietly leave it sitting:

• The new layers/quantization/linear/fp8_tensor.py allocates weight_scale as [out_features] fp32 directly and FP8_CHANNEL is a first-class scheme, so in the new architecture this function's job is done elsewhere and correctly.
• qwen3_5_moe hasn't migrated yet, though. config.py still computes _attn_quant / _lm_head_quant / _expert_quant and _iter_weights_attn_fp8 still routes off them, so the legacy reader -- and this bug -- are still live on main today.

So this is now a fix to a path that a migration would eventually delete. Two reasonable calls:

1. Merge it as a stopgap. It's 15 lines plus a test, no behaviour change for anything main can currently load, and it removes a silent-wrong-output failure mode from the legacy reader while that reader is still in use.
2. Close it and fold the concern into migrating qwen3_5_moe onto layers/quantization, which is where I think the real remaining work is.

I'm happy either way -- say the word. I closed #416 for exactly this reason (its scaffolding is gone entirely; the new config layer already resolves the fp8 head and ignore precedence, more completely than my patch did), and offered to do the qwen3_5_moe migration there. If you'd rather this one go the same way, just close it.

The test in here (tests/models/test_qwen3_5_moe_weight.py) is worth keeping either way if the migration lands -- the per-channel/per-tensor/mismatch cases and the row-order assertion apply to whatever computes that scale.

For whoever picks this up, the broader situation is written up as #437. Short version -- the new layers/quantization config layer resolves this checkpoint class correctly and qwen3_5_moe hasn't been migrated onto it, so _per_row_scale is a live bug in a reader that a migration would eventually delete. If #437 is accepted and I do that work, this PR becomes redundant and I'll close it myself. Until then the two options in my previous comment stand.

[salekseev]

Closing -- #438 carries this. The maintainer confirmed there that _per_row_scale stays (the layer allocates [out] but doesn't broadcast, so the reader still builds the per-row vector) and that the logic here is credited in the PR, and tests/models/test_qwen3_5_moe_weight.py from this branch is in #438's file list.

Nothing to rebase, nothing lost. Thanks for the read.

One note for anyone finding this later: the reachability caveat in the original description no longer applies the way I wrote it. I said this was unreachable on main because only ModelOpt checkpoints reached the fp8 dense reader. #438 replaces that reader with one pass that resolves every Linear through the QuantConfig, so compressed-tensors checkpoints reach the per-row scale path as a matter of course -- which is what makes this a live fix rather than hardening.