[XPU] xpu support think length limit #4539
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
ddchenhao66 seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
|
Thanks for your contribution! |
yuanlehome
reviewed
Oct 22, 2025
|
|
||
| } | ||
|
|
||
| PD_BUILD_OP(limit_thinking_content_length_v1) |
Collaborator
There was a problem hiding this comment.
这里不需要改成 PD_BUILD_STATIC_OP 吗，下同
yuanlehome
reviewed
Oct 22, 2025
| max_think_lens = share_inputs["max_think_lens"] | ||
| step_idx = share_inputs["step_idx"] | ||
| limit_think_status = share_inputs["limit_think_status"] | ||
| print(f"ch66 limit_strategy:{limit_strategy}") |
ddchenhao66
force-pushed
the
length_limit_1
branch
from
2576384 to
25ea457
Compare
iosmers
reviewed
Oct 22, 2025
| else: | ||
| # Disable thinking | ||
| self.share_inputs["max_think_lens"][idx : idx + 1, :] = -1 | ||
| self.share_inputs["limit_think_status"][idx : idx + 1, :] = 0 |
Collaborator
There was a problem hiding this comment.
self.share_inputs["limit_think_status"][idx : idx + 1, :] = 0 可以合并一下？
| if request.get("enable_thinking", False) and request.get("reasoning_max_tokens", None) is not None: | ||
| # Enable thinking | ||
| self.share_inputs["max_think_lens"][idx : idx + 1, :] = request.get("reasoning_max_tokens") | ||
| self.share_inputs["limit_think_status"][idx : idx + 1, :] = 0 |
Collaborator
There was a problem hiding this comment.
self.share_inputs["limit_think_status"][idx : idx + 1, :] = 0 可以合并一下？
| // 强制将当前token替换为结束思考的token | ||
| next_token_lm = line_break_id; | ||
| limit_think_status_lm = 2; | ||
| } |
Collaborator
There was a problem hiding this comment.
这里再加一个else{}? 加一些debug信息？
Collaborator
Author
There was a problem hiding this comment.
不需要，else的场景不需要操作
hong19860320
previously approved these changes
Oct 22, 2025
|
|
||
| WRAPPER_DUMP(ctx); | ||
| if (ctx->dev().type() == api::kCPU) { | ||
| assert(false); |
Collaborator
There was a problem hiding this comment.
后面有空补上 CPU wrapper 的实现吧
| WRAPPER_DUMP_PARAM2(ctx,line_break_id,bs); | ||
| WRAPPER_DUMP(ctx); | ||
| if (ctx->dev().type() == api::kCPU) { | ||
| assert(false); |
DDDivano
previously approved these changes
Oct 22, 2025
qingqing01
approved these changes
Oct 22, 2025
qingqing01
previously approved these changes
Oct 22, 2025
ddchenhao66
dismissed stale reviews from qingqing01, XiaoguangHu01, DDDivano, yuanlehome, and hong19860320
via
25d4408
ddchenhao66
force-pushed
the
length_limit_1
branch
from
25d4408 to
31fc218
Compare
yuanlehome
approved these changes
Oct 23, 2025
qingqing01
approved these changes
Oct 23, 2025
DDDivano
approved these changes
Oct 23, 2025
jeff41404
approved these changes
Oct 23, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
xpu支持限制思考长度，使用kernel提升性能
Motivation
Modifications
Usage or Command
Accuracy Tests
Checklist
[FDConfig],[APIServer],[Engine],[Scheduler],[PD Disaggregation],[Executor],[Graph Optimization],[Speculative Decoding],[RL],[Models],[Quantization],[Loader],[OP],[KVCache],[DataProcessor],[BugFix],[Docs],[CI],[Optimization],[Feature],[Benchmark],[Others],[XPU],[HPU],[GCU],[DCU],[Iluvatar],[Metax]]pre-commitbefore commit.releasebranch, make sure the PR has been submitted to thedevelopbranch, then cherry-pick it to thereleasebranch with the[Cherry-Pick]PR tag.