fix: make FunctionGemma prompt formatting strict#502
fix: make FunctionGemma prompt formatting strict#502HenryNdubuaku merged 1 commit intocactus-compute:mainfrom
Conversation
…mple, fix tool response wrapping)
Remove hardcoded “When you decide to call…” guidance and arg example from developer turn
Fix FunctionGemma trigger string (no trailing period) and append tools declarations directly
Wrap tool responses in a developer turn and allow stacking multiple tool responses
Stop wrapping tool outputs in value:; pass through {...} payload as-is
Close pending tool-response developer turn before next user/model turn to avoid malformed prompts
Signed-off-by: Lennart <[email protected]>
|
@lennartvoelz thanks for this! one thing, this fails more tool call tests than the old setup, any insights as to why? |
|
What did you test it against? @HenryNdubuaku |
|
I would say for the standard weights, it is just noise. But there are a few things off with FunctionGemma setup rn. I'll later push some changes that make the model much more stable. As said, I tested against the Hackathon dataset. With my changes, the effect is especially visible for fine tuned models. With the changes in the Gemma implementation, the effect of the extra tokens on latency is much smaller. However, I believe it is better practice to keep it standard (which also works better in this case!). |
|
thanks @lennartvoelz I will merge this now, thanks for testing |
2 participants
Follows the official docs