Skip to content

fix(community): normalize max_results before handing it to Firecrawl and fastCRW - #6175

Merged
WillemJiang merged 4 commits into
bytedance:mainfrom
Lesereingrape:fix/firecrawl-fastcrw-max-results-guard
Oct 2, 2026
Merged

WillemJiang merged 4 commits into
bytedance:mainfrom
Lesereingrape:fix/firecrawl-fastcrw-max-results-guard

Conversation

@Lesereingrape

Copy link
Copy Markdown
Contributor

What

firecrawl and fastcrw pass the configured max_results straight into client.search(query, limit=max_results). Every other web-search provider under community/ normalizes that value first — ddg_search, image_search, exa, searxng, serper, serply, sofya, brave, groundroute, tencent_wsa each have a _coerce_* guard; these two were the only ones that did not.

Why it is user-visible

max_results is free-form YAML, so anything lands here:

  • max_results: 0 sends limit=0, the provider answers with an empty result set, and the tool returns [] with no log line — the agent reads that as "the web has nothing about this" instead of "your config is wrong". The ten guarded providers log and fall back to the default for the same input.
  • max_results: true (a typo for 1, or a value YAML resolves as a bool) sends limit=true and the request is rejected, so the model gets Error: … in place of search results.
  • max_results: 3.5 sends a float; int() would silently turn it into 3, which is why the accepted ddg_search guard rejects fractional values outright.
  • max_results: "8" sends a string where an integer is expected.

How it is fixed

A module-level _coerce_max_results in each provider, mirroring the shape already merged for ddg_search (63401d79): reject booleans, fractional floats, non-numeric values and anything <= 0 with a warning naming the bad value, fall back to DEFAULT_MAX_RESULTS = 5, and accept an integer or an integer-form string. Nothing else about the call changes.

Checks (run on 63e399f2, Python 3.12)

$ pytest tests/test_firecrawl_tools.py tests/test_fastcrw_tools.py -q -k normalizes_max_results
12 failed, 4 passed   # against the unfixed sources: the six unusable shapes per provider
16 passed             # after

$ pytest tests/test_firecrawl_tools.py tests/test_fastcrw_tools.py -q
3 failed, 35 passed   # the 3 failures are TestWebFetchTool's DNS lookups; this sandbox has no resolver
                      # and they fail identically on an untouched checkout

$ ruff check <the four files> && ruff format --check <the four files>
All checks passed! / 4 files already formatted

Diff is 4 files, +122 −4: two guards, two call sites, and a parametrized test per provider covering bool / zero / negative / fractional / non-numeric / integer-string / plain int / omitted, each asserting the exact limit= the client receives plus whether a warning is expected.

tavily and searxng have the same gap and are already being handled in #5867, so this change deliberately leaves them alone.

Agent-assisted contribution, disclosed per AGENTS.md.

@github-actions github-actions Bot added risk:medium Medium risk: regular code changes size/M PR changes 100-300 lines labels Oct 2, 2026

@willem-bd willem-bd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean, well-tested fix that faithfully mirrors the merged ddg_search guard: both web_search call sites now normalize max_results (bool / fractional / non-numeric / <=0 rejected with a warning naming the bad value, integer-string accepted, default 5), and I verified these are the only max_results call sites in both files — web_fetch does not use it. The new parametrized tests assert the exact limit= kwarg the client receives plus warning presence, the imports they rely on already exist at head, and leaving tavily/searxng to #5867 avoids conflicting with that PR. Two nits inline: the helper is now triplicated, and one config shape is untested.

DEFAULT_MAX_RESULTS = 5


def _coerce_max_results(value: object) -> int:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is now the third near-identical copy of _coerce_max_results (ddg_search, firecrawl, fastcrw), on top of the ~10 per-provider _coerce_* guards already merged. deerflow.community already hosts shared helpers (url_safety, search_time_range), so a single coerce_max_results(value, default, label) there would keep the bool/float/int rules from drifting apart the next time one copy gets a fix and the others don't. Following the existing per-provider convention is fine, so deferring to a follow-up is reasonable — just noting the duplication has now tripled.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, and it is the reason I hesitated before adding a third copy. I stayed with the per-provider shape because the existing guards are not one function wearing three hats — measured on this tree: ddg_search and image_search are the unbounded form my two copy (reject bool, reject fractional float, <= 0 -> warn + default, no upper bound); serper and serply take default/max_allowed and clamp with min(...) instead of rejecting, and serper falls back without a log line; exa._coerce_positive_int accepts on isinstance(value, int) plus str.isdecimal() and threads an option name through its message.

So a shared coerce_max_results(value, default, label) has to settle three policies at once — whether an upper bound exists, whether a silent fallback is allowed, and whether fractional floats truncate or warn — across six call sites, which is its own change with its own tests rather than something to fold into this diff. Your drift argument is exactly why that refactor deserves to be reviewed on its own. Happy to take it on as a follow-up if you want it; otherwise I will leave this as it stands.

Comment thread backend/tests/test_firecrawl_tools.py Outdated
pytest.param("many", 5, True, id="non-numeric-string"),
pytest.param("8", 8, False, id="integer-string"),
pytest.param(7, 7, False, id="plain-int"),
pytest.param(None, 5, False, id="omitted"),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

omitted uses None as the key-absent sentinel, so the explicit max_results: null YAML shape — key present with a None value, which reaches _coerce_max_results and exercises the int(None) -> TypeError path — is never tested. It behaves correctly today (warning + default), but a dedicated case with a distinct sentinel object instead of None would pin that path down. Same applies to the mirrored test in test_fastcrw_tools.py.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — None was doing double duty, so extra["max_results"] = configured never ran for it and the key-present / value-null shape was never built. 24f8fb5 switches absence to a _OMITTED sentinel and adds an explicit-null case here and in test_fastcrw_tools.py. The two cases differ in the observable that matters: explicit-null warns, omitted does not, which is what pins the int(None) -> TypeError branch down (measured: _coerce_max_results(None) returns 5 with a warning in both providers).

pytest tests/test_firecrawl_tools.py tests/test_fastcrw_tools.py -k normalizes_max_results is 18 passed, up from 16; the only failures left in those two files are the three TestWebFetchTool cases that need a resolver, which this sandbox does not have.

Lesereingrape and others added 2 commits October 2, 2026 17:51
The parametrized `omitted` case reused None as its "key absent" marker, so
`extra["max_results"] = configured` never ran for None and the key-present,
value-null YAML shape never reached _coerce_max_results. Use a distinct
sentinel for absence and add an explicit-null case per provider.
@WillemJiang WillemJiang added this to the 2.2.0 milestone Oct 2, 2026
@WillemJiang
WillemJiang merged commit 8cd7aef into bytedance:main Oct 2, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:medium Medium risk: regular code changes size/M PR changes 100-300 lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants