Skip to content

wikipedia engine builds invalid URL for region="wt-wt" (wt.wikipedia.org does not exist) #417

Closed as not planned

Description

@yeole-rohan

Before you open an issue:

  • I have the latest version. (Check: ddgs version. Update: pip install -U ddgs)
  • I tried reinstalling the library. (pip install -I ddgs)
  • I tried using a proxy

Describe the bug

When backend="auto" is used and the wikipedia engine is selected, it constructs an invalid Wikipedia URL for the region="wt-wt" (worldwide, no regional bias) parameter. This causes a ConnectionError on every search because https://wt.wikipedia.org does not exist.

Root cause — ddgs/engines/wikipedia.py, build_payload, line 35:

_country, lang = region.lower().split("-")

For region="wt-wt" (DuckDuckGo's "no region" sentinel), this produces lang = "wt", which is not a valid ISO 639 language code. The engine then builds:

https://wt.wikipedia.org/w/api.php?action=opensearch&...

That domain does not exist, causing every call with backend="auto" to fail when the wikipedia engine is chosen.

Steps to reproduce the problem:

from ddgs import DDGS

results = DDGS().text(
    "python programming",
    region="wt-wt",   # worldwide, no regional bias
    safesearch="off",
    backend="auto",   # picks wikipedia engine
    max_results=5,
)

Expected: results returned from a working backend
Actual: ConnectionError — cannot connect to https://wt.wikipedia.org

Suggested fix: In wikipedia.py → build_payload, fall back to "en" when lang is not a recognised Wikipedia language code:

_country, lang = region.lower().split("-")
if lang == "wt" or len(lang) > 3:   # wt-wt is DDG's "worldwide" sentinel
    lang = "en"

Workaround (current): Pin backend="duckduckgo" explicitly to skip the wikipedia engine entirely.

Specify this information

  • OS: Ubuntu 24.04.4 LTS on WSL2 (Linux 5.15.167.4-microsoft-standard-WSL2)
  • Environment: Python 3.12.3, virtualenv
  • ddgs version: 9.10.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions