[GCU] Support gcu platform #2702
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
EnflameGCU
force-pushed
the
support_gcu_platform
branch
9 times, most recently
from
9b02b05 to
08a285b
Compare
| causal=self.causal, | ||
| ) | ||
|
|
||
| # res = self.native_sdpa_impl( |
Collaborator
There was a problem hiding this comment.
remove these useless codes
Contributor
Author
There was a problem hiding this comment.
mem_efficient_attn_backend其中的native sdpa实现主要用来调测中精度对比，期望的是保留该部分代码，调测新网络时方便本地快速切换，已切换到通过变量控制
Collaborator
vivienfanghuagood
left a comment
vivienfanghuagood
left a comment
There was a problem hiding this comment.
Please add some detaied information in PR:
- supported model lists & tutorials
- user guide about build & install GCU Fastdeploy
Contributor
Author
已提供，详见PR第一条comment |
EnflameGCU
force-pushed
the
support_gcu_platform
branch
from
08a285b to
592e800
Compare
yongqiangma
reviewed
Jul 7, 2025
| def insert_prefill_inputs(self, req_dicts: List[Request]): | ||
| """ | ||
| Process inputs for prefill tasks and insert it to share_inputs buffer | ||
| TODO(gongshaotian): Refactor this func |
Collaborator
There was a problem hiding this comment.
多个内容的注释是gongshaotian，根据实际情况修改下吧
yongqiangma
reviewed
Jul 7, 2025
| norm_out = self.norm_func( | ||
| x, residual_input, self.ln_weight, self.eps | ||
| ) | ||
| # if residual_input is not None: |
yongqiangma
reviewed
Jul 7, 2025
| raise NotImplementedError( | ||
| "prefix_caching is not support by GCUModelRunner." | ||
| ) | ||
| # cache_kvs_list = [] |
EnflameGCU
force-pushed
the
support_gcu_platform
branch
3 times, most recently
from
e24c60b to
55598cf
Compare
baseline: e7fa57e
EnflameGCU
force-pushed
the
support_gcu_platform
branch
from
c4a7a46 to
0ad9ae0
Compare
yongqiangma
approved these changes
Jul 8, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
新增
GCU硬件适配，包括：platformsgcu opsgcu_model_runner/gcu_workerattentionmoequantizationactivation/normalization/linear/rotary_embedding/sampler目前支持的模型：
详细安装使用请参见使用说明。