Skip to content

Investigate the possibility of removing httptools #539

@bymoye

Description

@bymoye

httptools has two main uses in the current framework: URL and built-in Client.

yarl is several times faster than httptools in URL parsing。

The results on my computer are as follows:

python -m timeit -u usec -s "import httptools" -- "httptools.parse_url(b'https://user:[email protected]/users?filter=1#a')"
1000000 loops, best of 5: 0.293 usec per loop

python -m timeit -u usec -s "from yarl import URL" -- "URL(b'https://user:[email protected]/users?filter=1#a'.decode())"
2000000 loops, best of 5: 0.124 usec per loop

Then about the built-in Client, which was discussed in #281 .

I investigated and found that httpx is indeed much slower than the current httptools. The main reasons may be.

  1. First of all, httpx is a pure python implementation based on h11.
  2. Then httpx has too many functions.

But maybe there are other options? For example, aiohttp, or other lightweight libraries?

Why try to remove it?
Because in this way, we may remove a "beast" that prevents blacksheep from embracing pypy

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions