[Bugfix][Quantization] Index INT8 MoE per-channel weight scales independently of act quant - #58570

…endently of act quant

Static per-tensor INT8 activations can be paired with per-output-channel
weight scales, but the legacy Triton fused_experts path used
per_act_token_quant to select both activation- and weight-scale layouts.
That made per-channel W13/W2 scales look like one scalar per expert.

Pass per_out_ch_quant through int8_w8a8_moe_quant_config and index
weight vs activation scales independently in the shared non-block W8A8
kernel.

Fixes vllm-project#58532

Signed-off-by: GokayAI <[email protected]>

Co-authored-by: GokayAI <[email protected]>

…Triton

The config/Quark assertions still run on CPU. The launcher and fused_experts
forwarding checks need a working Triton install, so skip them otherwise.

Signed-off-by: GokayAI <[email protected]>

Co-authored-by: GokayAI <[email protected]>

Quark static INT8 reports its activation and weight scale layouts from the
weight qscheme string, which couples the two. Read each layout off the
quant key that describes it instead:

- per_act_token_quant from the activation quant key
- per_out_ch_quant from the weight quant key

This keeps per-channel weights with static per-tensor activations (the
legacy fused_experts path) correct now that the shared Triton kernel
indexes the two layouts independently.

Also consolidate tests: the Quark scale-layout cases become a parametrized
test in tests/quantization/test_quark.py, and the tautological quant-config
factory tests are dropped. Intercept the vllm::fused_experts custom op in
test_fused_experts_forwards_independent_scale_flags, since patching
fused_experts_impl no longer takes effect (and the op only has CUDA/Meta
kernels, so a CPU test must patch at the op boundary).

Signed-off-by: Felix Marty <[email protected]>

Move the scale-layout coverage into tests/quantization/test_quark.py and
drop the standalone tests/kernels/moe/test_int8_moe_scale_layout.py:

- the flag cases stay parametrized over weight/activation quant key pairs
  (per-channel weights with static per-tensor activations report
  per_out_ch_quant=True, per_act_token_quant=False);
- the numerical case is driven through
  QuarkW8A8Int8MoEMethod.get_fused_moe_quant_config() so it exercises the
  production legacy fused_experts path, and it fails on the pre-fix code
  where routed experts' per-channel W13/W2 scales were ignored;
- the two kernel-plumbing tests are dropped: both flags now travel with
  the quant config (see the TritonExperts/BatchedTritonExperts changes),
  and the numerical case covers the behaviour end to end.

Signed-off-by: Felix Marty <[email protected]>

…cher default

invoke_fused_moe_triton_kernel, dispatch_fused_moe_kernel,
fused_experts_impl and invoke_moe_batched_triton_kernel gained an explicit
per_out_ch_quant flag. Defaulting it to False silently changed behaviour
for callers that only set per_channel_quant: those used to get per-channel
weight scales and would now read a single per-expert (per-tensor) scale,
which produces wrong numbers instead of failing.

Default the flag to None and resolve it to the activation layout flag when
unset, so pre-existing callers keep their old behaviour and only callers
that state the weight layout use the new, decoupled indexing. With that,
models/k2_horizon.py no longer needs to pass the flag by hand.

Add a launcher test asserting that per_channel_quant=True without
per_out_ch_quant still resolves to per-channel weight scales.

Signed-off-by: Felix Marty <[email protected]>

…ivation layout

FusedMoEQuantConfig.make and the fp8/int8/int4-w4afp8 quant config factories
gained an explicit per_out_ch_quant flag. Callers that only set
per_act_token_quant - including several FP8 MoE kernel tests and out-of-tree
code - used to get per-channel weight scales and now silently read a single
per-expert (per-tensor) scale, which produces wrong numbers instead of
failing. In CI this broke test_int8_kernel.py::test_w8a8_fp8_fused_moe and
test_triton_moe_ptpc_fp8.py::test_w8a8_fp8_fused_moe.

Default the flag to None and resolve it to per_act_token_quant when unset, so
pre-existing callers keep their old behaviour and only callers that state the
weight layout use the new, decoupled indexing. This matches the launcher
default added in the previous commit.

Signed-off-by: Felix Marty <[email protected]>