Skip to content

change all to httpx #26119

Merged
crazywoola merged 16 commits intolanggenius:mainfrom
asukaminato0721:all-httpx
Oct 10, 2025
Merged

change all to httpx #26119
crazywoola merged 16 commits intolanggenius:mainfrom
asukaminato0721:all-httpx

Conversation

@asukaminato0721
Copy link
Copy Markdown
Contributor

Important

  1. Make sure you have read our contribution guidelines
  2. Ensure there is an associated issue and you have been assigned to it
  3. Use the correct syntax to link this PR: Fixes #<issue number>.

fix #24006

by codex

Summary

Screenshots

Before After
... ...

Checklist

  • This change requires a documentation update, included: Dify Document
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.
  • I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

@asukaminato0721 asukaminato0721 marked this pull request as ready for review September 23, 2025 10:20
Copilot AI review requested due to automatic review settings September 23, 2025 10:20
@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Sep 23, 2025
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR migrates HTTP client library usage from requests to httpx across the entire Dify codebase. This modernizes the HTTP client stack and takes advantage of httpx's async capabilities and better timeout handling.

Key changes include:

  • Replace requests with httpx for all HTTP operations
  • Update timeout configurations to use httpx.Timeout objects
  • Modify authentication imports to use httpx equivalents
  • Update OpenTelemetry instrumentation from requests to httpx

Reviewed Changes

Copilot reviewed 23 out of 24 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
api/pyproject.toml Updates dependencies to replace requests-related packages with httpx
api/pyrightconfig.json Updates OpenTelemetry instrumentation from requests to httpx
api/extensions/ext_otel.py Switches OpenTelemetry instrumentation from RequestsInstrumentor to HTTPXClientInstrumentor
api/services/*/remote_retrieval.py Replaces requests.get with httpx.get and updates timeout syntax
api/services/enterprise/base.py Major refactor to use httpx.Client with proper mount handling for proxies
api/core/tools/utils/parser.py Updates HTTP GET request and adds proper response cleanup
api/core/rag/extractor/*.py Migrates various extractors from requests to httpx with proper resource management
api/core/rag/datasource/vdb//.py Updates vector database clients to use httpx and handle connection errors
api/core/plugin/impl/base.py Comprehensive refactor of plugin daemon client to use httpx with streaming support
api/core/extension/api_based_extension_requestor.py Converts extension requestor to use httpx.Client with proper proxy handling
api/tests/unit_tests//test_.py Updates test mocks from requests to httpx
api/tests/integration_tests/vdb/__mock/*.py Removes requests.adapters imports and updates type hints

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@asukaminato0721 asukaminato0721 force-pushed the all-httpx branch 3 times, most recently from 3b31c7c to 272cb1b Compare September 24, 2025 14:19
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Oct 10, 2025
@crazywoola crazywoola merged commit bb6a331 into langgenius:main Oct 10, 2025
12 checks passed
@asukaminato0721 asukaminato0721 deleted the all-httpx branch October 10, 2025 20:09
Eric-Guo added a commit to Eric-Guo/dify that referenced this pull request Oct 13, 2025
Eric-Guo added a commit to Eric-Guo/dify that referenced this pull request Oct 13, 2025
@Eric-Guo
Copy link
Copy Markdown
Contributor

Eric-Guo commented Oct 13, 2025

Codex Review comment:

  • [P0] Preserve plugin request timeout semantics — api/core/plugin/impl/base.py:66-68 Switching _request over to httpx.request without supplying a timeout changes the runtime behaviour: httpx applies a default 5 s connect/read timeout, whereas requests.request previously waited indefinitely. Any plugin daemon call (including the SSE path) that legitimately takes longer than 5 s will now raise httpx.TimeoutException, be caught as a generic RequestError, and surface as PluginDaemonInnerError, breaking long-running plugin invocations. Please pass an explicit timeout that matches the old behaviour (e.g. timeout=None or a configured value) so we don’t regress existing plugins.

Eric-Guo added a commit to Eric-Guo/dify that referenced this pull request Oct 13, 2025
…egitimately takes longer than 5s would right.

Otherwise, will be caught as a generic RequestError, and surface as PluginDaemonInnerError, fix langgenius#26119
Eric-Guo added a commit to Eric-Guo/dify that referenced this pull request Oct 13, 2025
…egitimately takes longer than 5s would right.

Otherwise, will be caught as a generic RequestError, and surface as PluginDaemonInnerError, fix langgenius#26119
Eric-Guo added a commit to Eric-Guo/dify that referenced this pull request Oct 13, 2025
…egitimately takes longer than 5s would right.

Otherwise, will be caught as a generic RequestError, and surface as PluginDaemonInnerError, fix langgenius#26119
Eric-Guo added a commit to Eric-Guo/dify that referenced this pull request Oct 13, 2025
…egitimately takes longer than 5s would right.

Otherwise, will be caught as a generic RequestError, and surface as PluginDaemonInnerError, fix langgenius#26119
Eric-Guo added a commit to Eric-Guo/dify that referenced this pull request Oct 13, 2025
…egitimately takes longer than 5s would right.

Otherwise, will be caught as a generic RequestError, and surface as PluginDaemonInnerError, fix langgenius#26119
Eric-Guo added a commit to Eric-Guo/dify that referenced this pull request Oct 13, 2025
…egitimately takes longer than 5s would right.

Otherwise, will be caught as a generic RequestError, and surface as PluginDaemonInnerError, fix langgenius#26119
LawrenceZHLee pushed a commit to LawrenceZHLee/dify that referenced this pull request Oct 28, 2025
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: crazywoola <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm This PR has been approved by a maintainer size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

requests -> httpx

4 participants