I suspect there are a lot of inefficiencies in sccache-dist, but let's start with a glaringly obvious one.

We use rouille as our web framework (which looks kind of like a dead project, but that's a separate issue). When rouille starts, we request that it start a server with a thread pool to handle requests:

https://github.com/tomaka/rouille/blob/e9692e1b19c8d25a742129a16bbe826c686ab274/src/lib.rs#L334-L340

Requests are handled here after they're received from tiny-http:

https://github.com/tomaka/rouille/blob/e9692e1b19c8d25a742129a16bbe826c686ab274/src/lib.rs#L443-L452

tiny-http is the actual web server responsible for serving requests...which also has a thread pool of its own:

https://github.com/tiny-http/tiny-http/blob/179cfea26a3b9413e95ad622b6161ec37d39285d/src/util/task_pool.rs

Plus a separate thread to actually listen on the socket to accept new connections. So processing a request looks like:

1. Get the connection on the listening thread.
2. Push something into tiny-http's task pool.
3. Some other thread turns it into a proper request.
4. Said request gets pushed onto a separate message queue.
5. The main thread running rouille gets notified there's a request.
6. That request gets bounced into rouille's thread pool.
7. Something finally processes that.
8. I think we do a similar dance to pass it back (?).

That's four threads involved in handling an HTTP request. That cannot possibly be efficient.

I don't know what the right way to untangle this is; I'm not at all familiar with what are the options in the Rust HTTP world. I have seen noise about Tide which from a cursory glance looks more efficient than the above setup, but I think that would require significant async/await buy-in.