-
Notifications
You must be signed in to change notification settings - Fork 3.4k
gemini-2.5-pro-preview-06-05 struggles with str_replace #2830
Description
It pretty commonly happens in my code (and I'm sure others) that specific sections end up duplicated in multiple places. Unit tests for example can end up looking pretty repetitive sometimes.
And at least gemini-2.5-pro-preview-06-05 often seems to seriously struggle with str_replace in some of these scenarios, hitting the error pretty often that there are multiple instances of the string. Which is not at all surprising; it'd be quite tricky for a human to craft something that worked with str_replace often. At one point it told me "This is getting frustrating, I'm just going to reread the file and write it back", but then we're using a lot more tokens.
When looking at this (partially tangential) I came to the realization that the current developer MCP was probably inspired by (at least?) the Claude Code version. However, it seems really likely to me that Claude has been explicitly trained on Anthropic's own MCP tooling. Other models I doubt are.
I started trying to hack on this a bit, my first intuition was that str_replace would be a lot easier for a model to use if e.g. it had an after_regexp that only started looking for the matched text after a particular regexp matched in the file - with the suggestion to use a function name for the regex. Also, if it was defined to not error on multiple matches in that case but just replace the first.
Anyways though, I guess my wider points are:
- It seems to me the
developerMCP server is too much of a catchall and would be better served by being split into different tools as it is for Claude (only tangential to this but I think in some use cases having a stricter allow-list forbashwould be really useful) - While Goose is pluggable I suspect for a lot of people the default enabled MCP servers strongly influence that default experience, and it's important to have a good out of the box set.
- It seems like it should be worth expanding the set of text editing tools (Claude also has a simple insert and I bet this would enable wider model compatibility and efficiency