Skip to content

LLM should send a custom user-agent when using -f https://... #1309

@simonw

Description

@simonw

I noticed that Wikipedia blocks the default HTTPX user-agent, as explained here: https://foundation.wikimedia.org/wiki/Policy:Wikimedia_Foundation_User-Agent_Policy

For example:

llm -f https://en.wikipedia.org/wiki/1988_World_Snooker_Championship 'top 3 winners'

Returns:

httpx.HTTPStatusError: Client error '403 Forbidden' for url 'https://en.wikipedia.org/wiki/1988_World_Snooker_Championship' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

Setting a custom user-agent similar to this should fix it:

User-Agent: CoolBot/0.0 (https://example.org/coolbot/; [email protected]) generic-library/0.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions