You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit fa6c830
Browse filesBrowse the repository at this point in the historyBrowse files
* init
* style
* full support
* support EP better using offsets !
* comments
* get rid of neutralize_ep_sentinels
* remove deepgemm stuff
* fix
* prefix
* move
* fix
* remove comment
* fix unintilized outputs leaking
* revert unnecessary changes
* more unnecessary changes
* revert downcast
* keep it simple
* guard deepgemm cuda version
* fix style
* update
* add deepgemm testing
* moe sentinel support
* fix
* compilable sonicmoe
* mega moe kernel support attempt
* use package for now
* skip ep router and experts pre/post processing
* simpler
* fix
Co-authored-by: Copilot <[email protected]>
* fix
* fix
* dtensor support
* more dtensor
* simpler
* remove comment
* revert
* bc order
* revert extra indent
* revert unnecessary change
* update
* less defensive
* allow all kernels
* alow all kernels
* hub only
* fix
* fix
* test
* test
* sync
* check nvcc
* probe
* fix
* test psum
* test
* test
* probe
* fix
* test
* nan issue
* repro
* repro
* fix
* simplifications
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* empty
* simplify
* test deepseek
* dsv4 only
* download dsv4
* fix test
* push
* test
* fix
* fix ep plan
* fix attempt
* debug
* attempt
* debug
* fixes in modeling
* more modeling changes
* more modeling changes
* initial megamoe works but wrong output, deepgemm fully functional
* mega moe works !
* simplification
* comments
* fix merge
* fix rank divergence
* use _keep_in_fp32_modules
* simplify quantizer
* introduce indexerscorer
* remove testing file
* style
* weight rename for scorer
* style
* address review comments
* style
* align with laguna
* stale comments and styling
* use grouped finegrained fp8 for grouped linear
Co-authored-by: sywangyi [email protected]
* comments and docs
* Revert "[DeepSeek V4] Fix MoE converter substring-matching FP8 scale companions (#45930)"
This reverts commit ecc7e32.
* fix quantizer and use upstream deepgemm
* remove force al kernels
* [deepseek_v4] fix FP8 save-side substring match + fp32 gate through softmax
* core_model_loading: WeightConverter.reverse_transform now appends a
(?=\.|$) token-boundary lookahead to each reversed source so they can't
substring-match longer sibling tokens. Re-fixes#45794 (the FP8Experts
scale companion `mlp.experts.gate_up_proj_scale_inv` was being routed
into the weight-only [Chunk, SplitModulelist] ops on save) without
needing `$` on every model-specific converter — that was #45930's
approach and broke load via `process_source_pattern`.
* Add `ConversionOps.supports_round_trip` opt-in flag; the on-the-fly
`quantization_operation` check in reverse_transform now uses it
instead of a blanket bail-out. Flagged the ops with audited reverse
pairs (Chunk/Concatenate, MergeModulelist/SplitModulelist, Transpose,
Conv3d↔Linear, Ernie pair, _IdentityOp, Fp8Quantize/Fp8Dequantize).
* deepseek_v4 compressors (HCA / Indexer / CSA): drop the trailing
`.to(chunk_gate.dtype)` so the gate stays fp32 from the
`+ position_bias` through the softmax — matches the reference
Compressor.forward which upcasts x to fp32 at entry.
* deepseek_v4 config: only set `attention_factor=1.0` when the compress
group resolves to `rope_type="yarn"`. Suppresses the noisy
"Unrecognized keys in rope_parameters for rope_type=default" warning
on the second post_init pass (where yarn params have already been
consumed).
* Refresh slow integration test expectations to match the fp32-gate
output.
Verified on the published DeepSeek-V4-Flash checkpoint:
RUN_SLOW=1 pytest tests/models/deepseek_v4 -k Integration -> 2 passed.
* always floats scal inv with triton
* minimal changes i quantizer
* force stable abi build for now
* support transposed
* fix yarn
* unnecessary change
* add flash base test and fix the incorrect and crash issue during enab… (#46076)
* fix incorrect output in tests/models/deepseek_v4/test_modeling_deepseek_v4.py::DeepseekV4IntegrationTest::test_v4_flash_fp8_generation and add test for flash-base
Signed-off-by: Wang, Yi <[email protected]>
* update
Signed-off-by: Wang, Yi <[email protected]>
---------
Signed-off-by: Wang, Yi <[email protected]>
* merge
* better errors across deepgemm
* more better errors for deepgemm
* ep tests
* Apply suggestions from code review
Co-authored-by: Arthur <[email protected]>
* scope supports_round_trip to quantization ops only
Drop the flag from the regular shape ops (Chunk, Concatenate, MergeModulelist,
SplitModulelist, Transpose, Conv3d↔Linear, _IdentityOp, the Ernie pair) — those
reverse themselves via their own `reverse_op` property, no flag needed. The
flag only gates `WeightConverter.reverse_transform`'s check on the converter's
`quantization_operation`, so today it's just Fp8Quantize / Fp8Dequantize. Other
quant backends (bnb, torchao, eetq, mxfp4, quanto, ...) stay opt-out until
their round-trip is audited.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* Apply suggestions from code review
Co-authored-by: Arthur <[email protected]>
* Update src/transformers/core_model_loading.py
* style: ruff format
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* custom ep/tp plan for megamoe
* process megamoe after weight loading
* fix megamoe process
* up
* the actual fix???
* up
* nit comment for futur
* style: drop unused exception variable in Fp8Dequantize._dequantize_one
* up
* style: apply make fix-repo docstring formatting
* simplify after merge
* review comments
* revert unncessary
* avoid name contraction
* fixes
* dynamic tp/ep plan swap
* fix deepgemm single error and device assertion docs
* properly support bias in grouped linear
* simplify
* fix dist error handling and min cuda compatbility
* fix least cuda compat
* get alignment from deepgemm and cache it
---------
Signed-off-by: Wang, Yi <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Arthur Zucker <[email protected]>
Co-authored-by: Wang, Yi <[email protected]>
Co-authored-by: Arthur <[email protected]>
Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
0 commit comments