What about if we reach the condition we enter the ring to retrieve whatever completions are already there and pushed those to a pending_queue to make sure no request is completed synchronously and decrease iou->in_flight accordingly? Only if iou->in_flight reaches the limit with no completions left, would make us go through the threadpool.

It's probably because it's a sleepy Saturday night but I'm not quite sure what you're asking. To call io_uring_enter when we hit the limit?

I'm guessing you maybe intended to write s/What about/How about/?

Yes, sorry about not being clear, even incorrect. My thinking is, by the time the limit is hit it's very likely completions are already available, grab them and keep on submitting to the ring taking advantage of how fast sometimes those ops are performed. I said "enter" and that's not correct as we're not actually using io_uring_enter() to grab the completions (I conflated it with the io_uring_enter(IORING_ENTER_GETEVENTS) we were using to make more completions available when the OVERFLOW state was reached)

Right, I believe I'm following now. I don't think we can call uv__poll_io_uring() from here because that runs user callbacks.

Well, I guess we could if we retrieved the uv_fs_t pointer from the CQE and queued it somewhere, but... that's yet another queue :-)

Well, I guess we could if we retrieved the uv_fs_t pointer from the CQE and queued it somewhere, but... that's yet another queue :-)

Yes, that was what I was thinking when I mentioned about pushing completions to a pending_queue...
So, do you think it would make a difference? Yes, it's another queue, but we would be sending more requests to the ring than to the threadpool which should be have better performance?

Well... maybe. The counterargument is that it's default-disabled code so it's not going to get used or tested much. An easier fix is to make the SQ and CQ bigger; they're quite modestly sized now. I.e.:

diff --git a/src/unix/linux.c b/src/unix/linux.c
index 279ca2c8..c63484d6 100644
--- a/src/unix/linux.c
+++ b/src/unix/linux.c
@@ -773,7 +773,7 @@ static struct uv__io_uring_sqe* uv__iou_get_sqe(struct uv__iou* iou,
       return NULL;
     }
 
-    uv__iou_init(loop->backend_fd, iou, 64, UV__IORING_SETUP_SQPOLL);
+    uv__iou_init(loop->backend_fd, iou, 1024, UV__IORING_SETUP_SQPOLL);
     if (iou->ringfd == -2)
       iou->ringfd = -1;  /* "failed" */
   }

So I've benchmarked this last change. And performance drops significantly in my laptop. I'm running the fs_stat benchmark with these changes. Could you check I'm not doing anything obviously wrong?

diff --git a/src/unix/linux.c b/src/unix/linux.c
index 857a4ef8a..293cc1ce3 100644
--- a/src/unix/linux.c
+++ b/src/unix/linux.c
@@ -773,7 +773,7 @@ static struct uv__io_uring_sqe* uv__iou_get_sqe(struct uv__iou* iou,
       return NULL;
     }
 
-    uv__iou_init(loop->backend_fd, iou, 64, UV__IORING_SETUP_SQPOLL);
+    uv__iou_init(loop->backend_fd, iou, 1024, UV__IORING_SETUP_SQPOLL);
     if (iou->ringfd == -2)
       iou->ringfd = -1;  /* "failed" */
   }
diff --git a/test/benchmark-fs-stat.c b/test/benchmark-fs-stat.c
index c41062241..c9d1da07d 100644
--- a/test/benchmark-fs-stat.c
+++ b/test/benchmark-fs-stat.c
@@ -27,7 +27,8 @@
 
 #define NUM_SYNC_REQS         (10 * 1e5)
 #define NUM_ASYNC_REQS        (1 * (int) 1e5)
-#define MAX_CONCURRENT_REQS   32
+/* To be sure it overflows when ring size of 64 */
+#define MAX_CONCURRENT_REQS   128
 
 #define sync_stat(req, path)                                                  \
   do {                                                                        \
@@ -129,6 +130,7 @@ static void async_bench(const char* path) {
  * out of the equation.
  */
 BENCHMARK_IMPL(fs_stat) {
+  uv_loop_configure(uv_default_loop(), UV_LOOP_USE_IO_URING_SQPOLL);
   const char path[] = ".";
   warmup(path);
   sync_bench(path);

I ran fs_stat with 32 and 128 (in steps of 8) concurrent requests in the following configurations on x86_64 Linux 6.5.0-18-generic:

1. no io_uring, i.e., thread pool
2. io_uring the way it is in v1.x
3. io_uring + this pull request
4. io_uring + this pull request + 1024 sqes, 2048 cqes

For reqs=32, the results are uninteresting: the thread pool is around 50% slower, the other three are indistinguishable from each other.

For reqs=128, things get more interesting. At low concurrency rates (let's say < 32) results are much the same as for reqs=32 but at high concurrency rates (> 64):

• (1) is still slower but closing the gap (15%)
• (4) is 35% slower than (2) and (3) - I think that's what you are seeing to?

fs_stat stress-tests system call overhead, not disk I/O (only hits the dentry cache), so it seems we're falling off some io_uring performance cliff in (4).