[Bugfix][DSV4] attention: fall back to weight_scale when weight_scale_inv absent - #43290

Purpose

DeepseekV4MultiHeadLatentAttentionWrapper.attention_impl at vllm/models/deepseek_v4/attention.py:334 reads self.wo_a.weight_scale_inv unconditionally. Two compressed-tensors FP8 schemes register the per-block weight scale under different parameter names, and only one of the two has the rename:

• CompressedTensorsW8A16Fp8 BLOCK strategy explicitly renames weight_scale → weight_scale_inv in process_weights_after_loading at vllm/model_executor/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a16_fp8.py:130.

• CompressedTensorsW8A8Fp8 BLOCK strategy delegates to fp8_linear.process_weights_after_loading (compressed_tensors_w8a8_fp8.py:165). The DeepGemm kernel implementation at vllm/model_executor/kernels/linear/scaled_mm/deep_gemm.py:78-89 preserves whichever name the parameter had on disk (weight_scale or weight_scale_inv) — it does not rename.

For artifacts where the W8A8 BLOCK path is taken AND the on-disk scale name is weight_scale (the default for llm-compressor's FP8_BLOCK preset), wo_a reaches attention_impl with weight_scale registered but weight_scale_inv absent, and the forward path raises at the first _dummy_run profile pass during torch.compile tracing:

AttributeError: 'ColumnParallelLinear' object has no attribute 'weight_scale_inv'.
Did you mean: 'weight_scale'?

When this bites

A DSV4-class artifact quantized with an llm-compressor recipe that uses the canonical FP8_BLOCK preset (compressed-tensors.quantization.quant_scheme.FP8_BLOCK) with dynamic FP8 input activations — that is, the same recipe RedHat ships in RedHatAI/DeepSeek-V4-Flash-NVFP4-FP8 — produces an artifact with on-disk scales named weight_scale. Loading through current mainline picks CompressedTensorsW8A8Fp8 BLOCK + DeepGemm and the parameter name flows through unchanged.

Change

         wo_a_fp8 = self.wo_a.weight
-        wo_a_scale = self.wo_a.weight_scale_inv
+        # CompressedTensorsW8A16Fp8 BLOCK strategy renames `weight_scale` ->
+        # `weight_scale_inv` in process_weights_after_loading
+        # (compressed_tensors_w8a16_fp8.py:130). The W8A8Fp8 BLOCK strategy
+        # delegates to fp8_linear.process_weights_after_loading, which preserves
+        # whichever name was on disk (`weight_scale` or `weight_scale_inv`).
+        # Fall back so both scheme paths reach the same einsum entry point.
+        wo_a_scale = getattr(self.wo_a, "weight_scale_inv", None)
+        if wo_a_scale is None:
+            wo_a_scale = self.wo_a.weight_scale

Test plan

• No semantic change for any configuration where weight_scale_inv already exists — the explicit-is None fallback only kicks in when weight_scale_inv is absent.
• For W8A8Fp8 BLOCK artifacts where the on-disk name is weight_scale, the forward path now reads the parameter that DeepGemm's process_weights_after_loading actually wrote, matching the kernel's contract.
• Same fallback pattern is already used elsewhere in vLLM where parameter naming is ambiguous between FP8 conventions.

Test result

Local syntax check passes. _dummy_run profile pass proceeds past attention_impl for a W8A8Fp8 BLOCK + DeepGemm + on-disk-weight_scale artifact that previously crashed at this site.

Related

• vLLM PR [Bugfix][compressed-tensors] Wrap is_static_input_scheme with bool() for input_quant=None schemes #43248 (this org): same family of code-smell — defensive handling of compressed-tensors scheme parameter conventions.
• vLLM PR [Bugfix][DSV4] Default scale_fmt to "ue8m0" instead of hard-subscript #43288 (this org): same family — defensive .get() on quantization_config["scale_fmt"] at model.py:909.