[Feature] add tool parser #3483
Conversation
|
Thanks for your contribution! |
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for tool parsers to the FastDeploy system to handle function call parsing from model outputs. The changes introduce two new command-line parameters and refactor the reasoning and tool parsing architecture.
Key changes:
- Added
--tool-call-parserand--tool-parser-pluginparameters for configuring tool parsers - Refactored reasoning parsers to return
DeltaMessageobjects instead of tuples - Unified streaming response handling to use a single
delta_messagefield - Updated variable naming from
tool_parserstotool_parser_dictfor consistency
Reviewed Changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
fastdeploy/reasoning/qwen3_reasoning_parsers.py |
Modified to return DeltaMessage objects and added is_reasoning_end method |
fastdeploy/reasoning/ernie_x1_reasoning_parsers.py |
Major refactor of streaming logic to return DeltaMessage objects and removed embedded unit tests |
fastdeploy/reasoning/ernie_vl_reasoning_parsers.py |
Updated to return DeltaMessage objects instead of tuples |
fastdeploy/input/text_processor.py |
Renamed variables and unified response handling with delta_message field |
fastdeploy/input/ernie_processor.py |
Similar changes to text processor plus auto-enabling thinking for ErnieX1 |
fastdeploy/input/ernie_vl_processor.py |
Variable rename for consistency |
fastdeploy/entrypoints/openai/tool_parsers/ernie_x1_tool_parser.py |
Simplified error handling and improved streaming logic |
fastdeploy/entrypoints/openai/serving_*.py |
Updated to handle new unified delta_message response structure |
tests/ |
Updated test mocks to match new variable names and method signatures |
docs/ |
Added documentation for new tool parser parameters |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| end_index = delta_text.find(self.end_token) | ||
| reasoning_content = delta_text[:end_index] | ||
| content = delta_text[end_index + len(self.end_token) :] | ||
| content = delta_text[end_index + len(self.end_token)] |
There was a problem hiding this comment.
Missing slice notation - should be delta_text[end_index + len(self.end_token):] to get the substring from the end position to the end of the string.
| content = delta_text[end_index + len(self.end_token)] | |
| content = delta_text[end_index + len(self.end_token):] |
| if len(delta_token_ids) == 1 and delta_token_ids[0] == self.think_end_token_id: | ||
| return None | ||
| # 思考阶段处理 | ||
| if not previous_text.endswith(self.think_end_token) and self.think_end_token not in previous_text: |
There was a problem hiding this comment.
The condition self.think_end_token not in previous_text is redundant since if previous_text.endswith(self.think_end_token) is false and the token appears anywhere in the text, the first condition would already be false. Consider simplifying to just check not previous_text.endswith(self.think_end_token).
| if not previous_text.endswith(self.think_end_token) and self.think_end_token not in previous_text: | |
| if not previous_text.endswith(self.think_end_token): |
| # 完成当前工具调用处理 | ||
| self.current_tool_id += 1 | ||
| self.current_tool_name_sent = False | ||
| self.streamed_args_for_tool.append("") |
There was a problem hiding this comment.
This line appears to be adding an empty string to the streamed_args_for_tool list when a tool call ends, but the current_tool_id and current_tool_name_sent variables are not being reset. This could lead to incorrect state for subsequent tool calls.
| self.streamed_args_for_tool.append("") | |
| self.streamed_args_for_tool.append("") | |
| self.current_tool_id = -1 | |
| self.current_tool_name_sent = False |
| prompt_token_ids=None, | ||
| completion_token_ids=output.get("token_ids") if request.return_token_ids else None, | ||
| tool_calls=None, | ||
| raw_prediction=output.get("raw_prediction") if request.return_token_ids else None, |
There was a problem hiding this comment.
There's an inconsistent line missing between lines 380-381. The original code had a completion_tokens field that was removed, but the formatting suggests a line may be missing. Please verify the delta_message structure is complete.
| raw_prediction=output.get("raw_prediction") if request.return_token_ids else None, | |
| raw_prediction=output.get("raw_prediction") if request.return_token_ids else None, | |
| completion_tokens=output.get("completion_tokens") if "completion_tokens" in output else None, |
* add tool parser * add x1 enable_thinking * restart ci * fix vl reasoning parser * modify call style * modify call style * add offline enablethinking * fix completion * fix * fix unit test * fix unit test * fix unit test * fix vl reasoning parser * fix vl reasoning parser
* [Feature] Pass through the `chat_template_kwargs` to the data processing module (#3421) * fix chat_template_args * fix args * add offline * add offline * fix * fix * fix default enable_thinking value * fix default enable_thinking value * modify condition * Revert "modify condition" This reverts commit 26430bd. * fix unit test * add Tool Parser (#3272) * add tool-parser * add tool-parser * add tool parser * add tool parser * fix * add offline * add offline * fix * parsers:tool&reasoning * 修改tool parser名称· * update * fix reasoning-parser * add requirements * fix finish reason * fix * fix reasoning-parser * fix * fix * fix * fix * fix --------- Co-authored-by: zhuzixuan <[email protected]> * [Feature] add tool parser (#3483) * add tool parser * add x1 enable_thinking * restart ci * fix vl reasoning parser * modify call style * modify call style * add offline enablethinking * fix completion * fix * fix unit test * fix unit test * fix unit test * fix vl reasoning parser * fix vl reasoning parser * fix unit test --------- Co-authored-by: zhuzixuan <[email protected]>
3 participants
功能说明：
1.增加--tool-call-parser参数，启动服务时可以通过这个参数指定tool parser用于解析function call。
2.增加--tool-parser-plugin参数，启动服务时可以通过这个参数指定parser文件路径来注册parser。