Skip to content

fix(serper): ignore malformed image URLs#4319

Merged
WillemJiang merged 1 commit into
bytedance:mainfrom
VectorPeak:codex/serper-malformed-ipv6
Jul 21, 2026
Merged

fix(serper): ignore malformed image URLs#4319
WillemJiang merged 1 commit into
bytedance:mainfrom
VectorPeak:codex/serper-malformed-ipv6

Conversation

@VectorPeak

@VectorPeak VectorPeak commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Why

Serper image search normalizes provider-controlled imageUrl and thumbnailUrl values through _safe_public_url(). That helper rejected unsafe schemes and private hosts, but it called urlparse(url) directly.

Python raises ValueError for malformed IPv6-style netlocs such as http://[::1/i.jpg, so one malformed Serper image result could abort image_search_tool() normalization instead of being filtered out like other invalid or unsafe URLs. The sibling Brave provider already treats this case as an invalid URL and returns an empty string.

Call chain / impact:

Serper provider response
  -> image_search_tool(...)
  -> _safe_public_url(imageUrl / thumbnailUrl)
  -> urlparse(url)
  -> ValueError: Invalid IPv6 URL

This affects only Serper image URL normalization. It does not change query construction, Serper API payloads, safe public URL acceptance, private-host filtering, or non-image search behavior.

What changed

  • Wrap Serper _safe_public_url() parsing in try/except ValueError.
  • Return "" for malformed URLs, matching the existing invalid/unsafe URL behavior.
  • Add a regression test for a malformed IPv6 URL, mirroring the sibling Brave coverage.

Surface area

  • Frontend UI — page / component / setting / interaction under frontend/
  • Backend API — endpoint / SSE event / request-response shape under backend/app
  • Agents / LangGraph — agent node, graph wiring, langgraph.json, or prompt change
  • Sandboxdocker/ or sandboxed execution
  • Skills — change under skills/
  • Dependencies — new/upgraded entry in backend/pyproject.toml or frontend/package.json (say what it buys us)
  • Default behavior change — changes existing behavior without the user opting in (default model, default setting, data shape)
  • Docs / tests / CI only — no runtime behavior change

Backend community tool provider only: this changes Serper image URL normalization under backend/packages/harness/deerflow/community/serper.

Screenshots / Recording

N/A — backend tool normalization only.

Bug fix verification

Test path that reproduces the bug:

  • backend/tests/test_serper_tools.py::TestSafePublicUrl::test_malformed_ipv6_url_does_not_raise

Before the fix, the equivalent helper repro on main raised:

ValueError: Invalid IPv6 URL

After the fix, malformed provider URLs are filtered out and valid sibling image results still normalize.

Tool-level behavior proof after the fix:

{
  "query": "cat",
  "total_results": 1,
  "results": [
    {
      "title": "good",
      "image_url": "https://example.com/i.jpg",
      "thumbnail_url": "https://example.com/i.jpg"
    }
  ],
  "usage_hint": "Use the 'image_url' values as reference images in image generation. Download them first if needed."
}

Validation

  • uv run --project backend pytest backend/tests/test_serper_tools.py::TestSafePublicUrl — 23 passed
  • uv run --project backend pytest backend/tests/test_serper_tools.py — 96 passed
  • uv run --project backend ruff check backend/packages/harness/deerflow/community/serper/tools.py backend/tests/test_serper_tools.py — passed
  • uv run --project backend ruff format --check backend/packages/harness/deerflow/community/serper/tools.py backend/tests/test_serper_tools.py — passed
  • git diff --check — passed

AI assistance

Tool(s) used: Codex

How you used it: Identified the malformed URL repro, implemented the narrow Serper guard, added the regression test, and ran focused backend validation.

  • I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.

Co-authored-by: chatgpt-codex-connector[bot] <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com>
@github-actions github-actions Bot added risk:medium Medium risk: regular code changes size/XS PR changes < 20 lines labels Jul 20, 2026
@WillemJiang
WillemJiang merged commit 16a77cb into bytedance:main Jul 21, 2026
11 checks passed
yogyoho added a commit to yogyoho/eai-flow-main that referenced this pull request Jul 22, 2026
…_detection counter clear, serper URL guard, zip colon NTFS ADS)

bytedance#4295: clear _tool_name_counter on evict/reset (prevents stale counter leak across runs). bytedance#4319: wrap urlparse in try/except to ignore malformed image URLs in serper. bytedance#4236: reject colon in zip member names to close NTFS ADS smuggling gap in skill installer.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:medium Medium risk: regular code changes size/XS PR changes < 20 lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants