[Bug fix] Fix lm head bias #3185
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
heavengate
reviewed
Aug 4, 2025
| if self.linear_bias_key is None: | ||
| logits = paddle.matmul(logits, self.weight) | ||
| else: | ||
| logits = paddle.incubate.nn.functional.fused_linear(logits, self.weight, self.bias) |
Collaborator
There was a problem hiding this comment.
这两个底层调用的OP不一致么，会有性能diff么
Collaborator
Author
There was a problem hiding this comment.
不一致，paddle.matmul调用cublas，paddle.incubate.nn.functional.fused_linear调用融合的cublasLt，可以将bias的加法融合到epilogue中，理论上比单独调用add算子性能更好
|
Thanks for your contribution! |
yuanlehome
reviewed
Aug 5, 2025
| is_bias=False, | ||
| ) | ||
| if self.bias_key is not None: | ||
| self.bias = self.create_parameter( |
Collaborator
There was a problem hiding this comment.
lm_head是否有bias不应该写在use_ep逻辑下吧？应该是通用的
Collaborator
Author
There was a problem hiding this comment.
TP下面用了ColumnLinear的类，类里面会判断。之所以EP不复用ColumnLinear，原因在于这里直接传入了从fleet获取的num_ranks信息。合并EP和TP可以用fd_config中的ep_rank和tp_rank去判断。限于人力这里没有做重构。
yuanlehome
approved these changes
Aug 5, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add lm head bias for EP