There are two things that I found out:

1. You introduced new generics like T and the error messages are related to that. Throughout your flow T is supposed to be generic but it becomes a concrete type at some point (Request). Rust can't substitute within a function. Changing it back to Request makes those type errors go away, but I wonder if you had anything particular in mind when introducing generic type parameters?

2. When I change T back to Request, this part throws some nasty errors:

    tokio::spawn(async move {
        futures::StreamExt::for_each_concurrent(
            ReceiverStream::new(recv_req),
            max_concurrency,
            |request: Result<T>| async {
                let request = request.expect("cannot read request");
                let response = handle(&mut client, cache.clone(), request).await;

                send_resp
                    .send(response)
                    .await
                    .expect("cannot send response to queue");
            },
        )
        .await;
    });

First it can't pass a mutable reference to client within a spawned future and async block, and if I introduce an Arc or similar it turns out that BoxService is not Sync... and I honestly don't know how to get rid of that right now.

But before going into that rabbit hole, I thought I'd ask about the generic type parameters and what your intention was :) Maybe that makes a few things more clear to me

Thanks a lot for your comment. I really appreciate the support.

The idea behind the generic type was that I didn't want to be dependent on any single request implementation (that of reqwest or any other). However in hindsight that is unnecessary because we fully control the internals of the library and could change it if we wanted to, which is - quite frankly - unlikely.

With that in mind, let's change it back to the concrete type if it makes things easier.

As for BoxService, my guess is that the canonical way might be to use SyncWrapper as described in tower-rs/tower#697, which also includes a link to a pull request which does just that. What do you think?

Maybe if we use a concrete type and SyncWrapper we might get closer to a working version.

Closing this as I don't see it getting merged anytime soon. I want to revisit this idea at some point once I'm a bit clearer of what I want to achieve by moving to Tower. We got better retry-handling now and the middleware part is still a bit far in the future, so no need to merge a rushed design. Thanks, @ddprrt, for the feedback and support, though!