fix(db): useLiveInfiniteQuery pagination with async on-demand loadSubset (#1209)

KyleAMathews · claude · autofix-ci[bot] · web-flow · commit 9184dcce6201 · 2026-02-12T14:43:00.000-07:00
* test: add failing test for useLiveInfiniteQuery peek-ahead limit bug This test documents a bug where useLiveInfiniteQuery doesn't request pageSize+1 items from loadSubset for hasNextPage peek-ahead detection. The bug causes hasNextPage to always return false when using on-demand sync mode with Electric collections, because: 1. useLiveInfiniteQuery calls setWindow({ limit: pageSize + 1 }) in useEffect 2. But subscribeToOrderedChanges calls requestLimitedSnapshot BEFORE the useEffect runs, using the original compiled limit (pageSize) 3. The loadSubset function receives limit=pageSize instead of limit=pageSize+1 4. This prevents the peek-ahead strategy from working correctly Related: Discord bug report about useLiveInfiniteQuery + Electric on-demand https://claude.ai/code/session_01FskX2noxNAj1zCiALFQXnC * ci: apply automated fixes * fix(react-db): use pageSize+1 in initial query for peek-ahead detection The initial query was using `.limit(pageSize)` but `setWindow` expects `pageSize + 1` for peek-ahead detection. This caused a race condition where the first `requestLimitedSnapshot` was called with `limit = pageSize` before `setWindow` could adjust it to `pageSize + 1`. The fix uses `pageSize + 1` from the start so the compiled query includes the peek-ahead limit, ensuring `loadSubset` receives the correct limit for `hasNextPage` detection. https://claude.ai/code/session_01FskX2noxNAj1zCiALFQXnC * ci: apply automated fixes * refactor(tests): simplify useLiveInfiniteQuery test assertions - Fix unused parameter lint warnings (allPages -> _allPages) - Simplify test logic using .find() instead of .filter()[0] - Condense redundant comments Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: add changeset for peek-ahead fix Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add e2e test for useLiveInfiniteQuery with on-demand collection Replaces the previous implementation-detail test with a proper e2e test that verifies the actual behavior of useLiveInfiniteQuery with on-demand collections: - Initial page loads correctly with hasNextPage=true - fetchNextPage() actually loads more data via loadSubset - Multiple pages can be fetched with correct items - hasNextPage correctly reflects when no more data exists This test catches bugs where the incremental sync doesn't properly fetch data from the backend when paginating. https://claude.ai/code/session_01FskX2noxNAj1zCiALFQXnC * test: add failing test for async loadSubset pagination bug This test reproduces a bug where useLiveInfiniteQuery doesn't fetch subsequent pages when loadSubset returns a Promise (async mode). Root cause identified in collection-subscriber.ts: - When loadSubset returns a Promise, pendingOrderedLoadPromise is set - loadMoreIfNeeded returns early while the promise is pending - When the promise resolves, pendingOrderedLoadPromise is cleared - BUT loadMoreIfNeeded is NOT re-triggered to check if more data is needed This affects Electric on-demand mode where all data comes from async loadSubset calls. The initial page loads correctly, but fetchNextPage fails to trigger additional loadSubset calls. https://claude.ai/code/session_01FskX2noxNAj1zCiALFQXnC * fix(db): re-trigger loadMoreIfNeeded when async loadSubset completes When useLiveInfiniteQuery uses an on-demand collection with async loadSubset, the second page was never loaded because: 1. When setWindow() was called for the next page, maybeRunGraph's callback was never called because the graph had no pending work This fix ensures the graph run callback is called at least once even when there's no pending work, so setWindow() can trigger loadMoreIfNeeded for lazy loading scenarios. https://claude.ai/code/session_01FskX2noxNAj1zCiALFQXnC * refactor: simplify tests, extend async coverage, fix changeset - Extract createOnDemandCollection helper to reduce test duplication - Extend async on-demand test to verify all 3 pages and hasNextPage=false - Add peek-ahead boundary test for pageSize+1 items - Replace silent catch block with explicit re-throw in test helper - Add @tanstack/db to changeset (independently versioned) - Remove redundant comments and tighten callback-guarantee comment Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: apply automated fixes --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
diff --git a/.changeset/fix-infinite-query-peek-ahead.md b/.changeset/fix-infinite-query-peek-ahead.md
@@ -0,0 +1,8 @@
+---
+'@tanstack/react-db': patch
+'@tanstack/db': patch
+---
+
+Fix `useLiveInfiniteQuery` peek-ahead detection for `hasNextPage`. The initial query now correctly requests `pageSize + 1` items to detect whether additional pages exist, matching the behavior of subsequent page loads.
+
+Fix async on-demand pagination by ensuring the graph callback fires at least once even when there is no pending graph work, so that `loadMoreIfNeeded` is triggered after `setWindow()` increases the limit.
diff --git a/packages/db/src/query/live/collection-config-builder.ts b/packages/db/src/query/live/collection-config-builder.ts
@@ -336,13 +336,22 @@ export class CollectionConfigBuilder<
 
       // Always run the graph if subscribed (eager execution)
       if (syncState.subscribedToAllCollections) {
+        let callbackCalled = false
         while (syncState.graph.pendingWork()) {
           syncState.graph.run()
           // Flush accumulated changes after each graph step to commit them as one transaction.
           // This ensures intermediate join states (like null on one side) don't cause
           // duplicate key errors when the full join result arrives in the same step.
           syncState.flushPendingChanges?.()
           callback?.()
+          callbackCalled = true
+        }
+
+        // Ensure the callback runs at least once even when the graph has no pending work.
+        // This handles lazy loading scenarios where setWindow() increases the limit or
+        // an async loadSubset completes and we need to re-check if more data is needed.
+        if (!callbackCalled) {
+          callback?.()
         }
 
         // On the initial run, we may need to do an empty commit to ensure that
diff --git a/packages/react-db/src/useLiveInfiniteQuery.ts b/packages/react-db/src/useLiveInfiniteQuery.ts
@@ -184,10 +184,14 @@ export function useLiveInfiniteQuery<TContext extends Context>(
 
   // Create a live query with initial limit and offset
   // Either pass collection directly or wrap query function
+  // Use pageSize + 1 for peek-ahead detection (to know if there are more pages)
   const queryResult = isCollection
     ? useLiveQuery(queryFnOrCollection)
     : useLiveQuery(
-        (q) => queryFnOrCollection(q).limit(pageSize).offset(0),
+        (q) =>
+          queryFnOrCollection(q)
+            .limit(pageSize + 1)
+            .offset(0),
         deps,
       )
 
diff --git a/packages/react-db/tests/useLiveInfiniteQuery.test.tsx b/packages/react-db/tests/useLiveInfiniteQuery.test.tsx
@@ -14,7 +14,7 @@ type Post = {
   category: string
 }
 
-const createMockPosts = (count: number): Array<Post> => {
+function createMockPosts(count: number): Array<Post> {
   const posts: Array<Post> = []
   for (let i = 1; i <= count; i++) {
     posts.push({
@@ -28,6 +28,79 @@ const createMockPosts = (count: number): Array<Post> => {
   return posts
 }
 
+type OnDemandCollectionOptions = {
+  id: string
+  allPosts: Array<Post>
+  autoIndex?: `off` | `eager`
+  asyncDelay?: number
+}
+
+/**
+ * Creates an on-demand collection with a loadSubset handler that supports
+ * sorting, cursor-based pagination, and limit. Returns the collection and
+ * a reference to recorded loadSubset calls for test assertions.
+ */
+function createOnDemandCollection(opts: OnDemandCollectionOptions) {
+  const loadSubsetCalls: Array<LoadSubsetOptions> = []
+  const { id, allPosts, autoIndex, asyncDelay } = opts
+
+  const collection = createCollection<Post>({
+    id,
+    getKey: (post: Post) => post.id,
+    syncMode: `on-demand`,
+    startSync: true,
+    ...(autoIndex ? { autoIndex } : {}),
+    sync: {
+      sync: ({ markReady, begin, write, commit }) => {
+        markReady()
+
+        return {
+          loadSubset: (subsetOpts: LoadSubsetOptions) => {
+            loadSubsetCalls.push({ ...subsetOpts })
+
+            let filtered = [...allPosts].sort(
+              (a, b) => b.createdAt - a.createdAt,
+            )
+
+            if (subsetOpts.cursor) {
+              const whereFromFn = createFilterFunctionFromExpression(
+                subsetOpts.cursor.whereFrom,
+              )
+              filtered = filtered.filter(whereFromFn)
+            }
+
+            if (subsetOpts.limit !== undefined) {
+              filtered = filtered.slice(0, subsetOpts.limit)
+            }
+
+            function writeAll(): void {
+              begin()
+              for (const post of filtered) {
+                write({ type: `insert`, value: post })
+              }
+              commit()
+            }
+
+            if (asyncDelay !== undefined) {
+              return new Promise<void>((resolve) => {
+                setTimeout(() => {
+                  writeAll()
+                  resolve()
+                }, asyncDelay)
+              })
+            }
+
+            writeAll()
+            return true
+          },
+        }
+      },
+    },
+  })
+
+  return { collection, loadSubsetCalls }
+}
+
 describe(`useLiveInfiniteQuery`, () => {
   it(`should fetch initial page of data`, async () => {
     const posts = createMockPosts(50)
@@ -629,7 +702,7 @@ describe(`useLiveInfiniteQuery`, () => {
         {
           pageSize: 10,
           initialPageParam: 0,
-          getNextPageParam: (lastPage, allPages, lastPageParam) =>
+          getNextPageParam: (lastPage, _allPages, lastPageParam) =>
             lastPage.length === 10 ? lastPageParam + 1 : undefined,
         },
       )
@@ -838,7 +911,7 @@ describe(`useLiveInfiniteQuery`, () => {
         {
           pageSize: 10,
           initialPageParam: 100,
-          getNextPageParam: (lastPage, allPages, lastPageParam) =>
+          getNextPageParam: (lastPage, _allPages, lastPageParam) =>
             lastPage.length === 10 ? lastPageParam + 1 : undefined,
         },
       )
@@ -987,6 +1060,221 @@ describe(`useLiveInfiniteQuery`, () => {
     expect(result.current.isFetchingNextPage).toBe(false)
   })
 
+  it(`should request limit+1 (peek-ahead) from loadSubset for hasNextPage detection`, async () => {
+    // Verifies that useLiveInfiniteQuery requests pageSize+1 items from loadSubset
+    // to detect whether there are more pages available (peek-ahead strategy)
+    const PAGE_SIZE = 10
+    const { collection, loadSubsetCalls } = createOnDemandCollection({
+      id: `peek-ahead-limit-test`,
+      allPosts: createMockPosts(PAGE_SIZE), // Exactly PAGE_SIZE posts
+    })
+
+    const { result } = renderHook(() => {
+      return useLiveInfiniteQuery(
+        (q) =>
+          q
+            .from({ posts: collection })
+            .orderBy(({ posts: p }) => p.createdAt, `desc`),
+        {
+          pageSize: PAGE_SIZE,
+          getNextPageParam: (lastPage) =>
+            lastPage.length === PAGE_SIZE ? lastPage.length : undefined,
+        },
+      )
+    })
+
+    await waitFor(() => {
+      expect(result.current.isReady).toBe(true)
+    })
+
+    const callWithLimit = loadSubsetCalls.find(
+      (call) => call.limit !== undefined,
+    )
+    expect(callWithLimit).toBeDefined()
+    expect(callWithLimit!.limit).toBe(PAGE_SIZE + 1)
+
+    // With exactly PAGE_SIZE posts, hasNextPage should be false (no peek-ahead item returned)
+    expect(result.current.hasNextPage).toBe(false)
+    expect(result.current.data).toHaveLength(PAGE_SIZE)
+  })
+
+  it(`should detect hasNextPage via peek-ahead with exactly pageSize+1 items in on-demand collection`, async () => {
+    // Boundary test: with exactly pageSize+1 items, the peek-ahead item should
+    // signal hasNextPage=true but NOT appear in user-visible data
+    const PAGE_SIZE = 10
+    const { collection } = createOnDemandCollection({
+      id: `peek-ahead-boundary-test`,
+      allPosts: createMockPosts(PAGE_SIZE + 1),
+    })
+
+    const { result } = renderHook(() => {
+      return useLiveInfiniteQuery(
+        (q) =>
+          q
+            .from({ posts: collection })
+            .orderBy(({ posts: p }) => p.createdAt, `desc`),
+        {
+          pageSize: PAGE_SIZE,
+          getNextPageParam: (lastPage) =>
+            lastPage.length === PAGE_SIZE ? lastPage.length : undefined,
+        },
+      )
+    })
+
+    await waitFor(() => {
+      expect(result.current.isReady).toBe(true)
+    })
+
+    // Peek-ahead item detected: hasNextPage should be true
+    expect(result.current.hasNextPage).toBe(true)
+    // But user-visible data should be exactly pageSize (peek-ahead excluded)
+    expect(result.current.data).toHaveLength(PAGE_SIZE)
+    expect(result.current.pages).toHaveLength(1)
+    expect(result.current.pages[0]).toHaveLength(PAGE_SIZE)
+  })
+
+  it(`should work with on-demand collection and fetch multiple pages`, async () => {
+    // End-to-end test: on-demand collection where ALL data comes from loadSubset
+    // (no initial data). Simulates the real Electric on-demand scenario.
+    const PAGE_SIZE = 10
+    const { collection, loadSubsetCalls } = createOnDemandCollection({
+      id: `on-demand-e2e-test`,
+      allPosts: createMockPosts(25), // 2 full pages + 5 items
+      autoIndex: `eager`,
+    })
+
+    const { result } = renderHook(() => {
+      return useLiveInfiniteQuery(
+        (q) =>
+          q
+            .from({ posts: collection })
+            .orderBy(({ posts: p }) => p.createdAt, `desc`),
+        {
+          pageSize: PAGE_SIZE,
+          getNextPageParam: (lastPage) =>
+            lastPage.length === PAGE_SIZE ? lastPage.length : undefined,
+        },
+      )
+    })
+
+    await waitFor(() => {
+      expect(result.current.isReady).toBe(true)
+    })
+
+    // Page 1: 10 items
+    expect(result.current.pages).toHaveLength(1)
+    expect(result.current.data).toHaveLength(PAGE_SIZE)
+    expect(result.current.hasNextPage).toBe(true)
+    expect(result.current.data[0]!.id).toBe(`1`)
+    expect(result.current.data[9]!.id).toBe(`10`)
+
+    // Fetch page 2
+    act(() => {
+      result.current.fetchNextPage()
+    })
+
+    await waitFor(() => {
+      expect(result.current.pages).toHaveLength(2)
+    })
+
+    expect(loadSubsetCalls.length).toBeGreaterThan(1)
+    expect(result.current.data).toHaveLength(20)
+    expect(result.current.hasNextPage).toBe(true)
+    expect(result.current.pages[1]![0]!.id).toBe(`11`)
+    expect(result.current.pages[1]![9]!.id).toBe(`20`)
+
+    // Fetch page 3 (partial page)
+    act(() => {
+      result.current.fetchNextPage()
+    })
+
+    await waitFor(() => {
+      expect(result.current.pages).toHaveLength(3)
+    })
+
+    expect(result.current.data).toHaveLength(25)
+    expect(result.current.pages[2]).toHaveLength(5)
+    expect(result.current.hasNextPage).toBe(false)
+    expect(result.current.pages[2]![0]!.id).toBe(`21`)
+    expect(result.current.pages[2]![4]!.id).toBe(`25`)
+  })
+
+  it(`should work with on-demand collection with async loadSubset`, async () => {
+    // Same as the sync on-demand test, but loadSubset returns a Promise
+    // to simulate async network requests (the real Electric scenario).
+    const PAGE_SIZE = 10
+    const { collection, loadSubsetCalls } = createOnDemandCollection({
+      id: `on-demand-async-test`,
+      allPosts: createMockPosts(25),
+      autoIndex: `eager`,
+      asyncDelay: 10,
+    })
+
+    const { result } = renderHook(() => {
+      return useLiveInfiniteQuery(
+        (q) =>
+          q
+            .from({ posts: collection })
+            .orderBy(({ posts: p }) => p.createdAt, `desc`),
+        {
+          pageSize: PAGE_SIZE,
+          getNextPageParam: (lastPage) =>
+            lastPage.length === PAGE_SIZE ? lastPage.length : undefined,
+        },
+      )
+    })
+
+    await waitFor(() => {
+      expect(result.current.isReady).toBe(true)
+    })
+
+    await waitFor(() => {
+      expect(result.current.data).toHaveLength(PAGE_SIZE)
+    })
+
+    expect(result.current.pages).toHaveLength(1)
+    expect(result.current.hasNextPage).toBe(true)
+
+    const initialCallCount = loadSubsetCalls.length
+
+    // Fetch page 2
+    act(() => {
+      result.current.fetchNextPage()
+    })
+
+    expect(result.current.isFetchingNextPage).toBe(true)
+
+    await waitFor(
+      () => {
+        expect(result.current.data).toHaveLength(20)
+      },
+      { timeout: 500 },
+    )
+
+    expect(result.current.pages).toHaveLength(2)
+    expect(loadSubsetCalls.length).toBeGreaterThan(initialCallCount)
+    expect(result.current.hasNextPage).toBe(true)
+
+    // Fetch page 3 (partial page) to verify async path handles end-of-data
+    const callCountBeforePage3 = loadSubsetCalls.length
+
+    act(() => {
+      result.current.fetchNextPage()
+    })
+
+    await waitFor(
+      () => {
+        expect(result.current.data).toHaveLength(25)
+      },
+      { timeout: 500 },
+    )
+
+    expect(result.current.pages).toHaveLength(3)
+    expect(result.current.pages[2]).toHaveLength(5)
+    expect(loadSubsetCalls.length).toBeGreaterThan(callCountBeforePage3)
+    expect(result.current.hasNextPage).toBe(false)
+  })
+
   it(`should track isFetchingNextPage when async loading is triggered`, async () => {
     // Define all data upfront
     const allPosts = createMockPosts(30)
@@ -1062,8 +1350,10 @@ describe(`useLiveInfiniteQuery`, () => {
                   }
                   // Re-sort after combining
                   filtered.sort((a, b) => b.createdAt - a.createdAt)
-                } catch {
-                  // Fallback to original filtered if cursor parsing fails
+                } catch (e) {
+                  throw new Error(`Test loadSubset: cursor parsing failed`, {
+                    cause: e,
+                  })
                 }
               } else if (opts.limit !== undefined) {
                 // Apply limit only if no cursor (cursor handles limit internally)
