Skip to content

[Feature] UI - Usage: Auto-paginate daily spend data#23622

Merged
yuneng-jiang merged 5 commits intolitellm_internal_dev_03_13_2026from
litellm_usage_page_auto_pagination
Mar 14, 2026
Merged

[Feature] UI - Usage: Auto-paginate daily spend data#23622
yuneng-jiang merged 5 commits intolitellm_internal_dev_03_13_2026from
litellm_usage_page_auto_pagination

Conversation

@yuneng-jiang
Copy link
Copy Markdown
Contributor

Summary

Problem (Before Fix)

  • EntityUsage.tsx (teams, orgs, customers, tags, agents, users) only fetched page 1 of paginated daily spend endpoints — users with >1000 daily records saw incomplete data.
  • UsagePageView.tsx fetched all pages but blocked the UI until every page was loaded, leaving users staring at a spinner.

Fix

  • Created a reusable usePaginatedDailyActivity hook that fetches pages sequentially (500ms delay between requests), updates charts progressively after each page, and auto-cancels on unmount or parameter change.
  • Wired the hook into both UsagePageView.tsx (as fallback after aggregated endpoint) and EntityUsage.tsx (fixing the page-1-only bug).
  • Added a progress indicator ("Loading spend data... page X/Y") with a Stop button, and a partial-data notice when cancelled.

Testing

  • All 168 existing tests across 13 test files pass.

Type

🆕 New Feature
🐛 Bug Fix

… rendering

Previously, EntityUsage only fetched page 1 of paginated daily spend endpoints,
showing incomplete data. UsagePageView fetched all pages but blocked the UI until
completion. This adds a reusable usePaginatedDailyActivity hook that fetches pages
sequentially with 500ms delays, updates charts progressively, and supports
cancellation on unmount or user action.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@vercel
Copy link
Copy Markdown

vercel bot commented Mar 14, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Mar 14, 2026 6:29pm

Request Review

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 14, 2026

Greptile Summary

This PR fixes a real bug (EntityUsage only fetched page 1 of paginated daily-spend endpoints) and improves UsagePageView by replacing a blocking full-fetch with a usePaginatedDailyActivity hook that streams data progressively and lets users cancel mid-flight. The new hook, progress indicator, and Stop button are well-structured, but several issues need attention before merging.

Key findings:

  • Four console.log("debugTags", ...) statements remain in EntityUsage.tsx's getTopAPIKeys() function and will pollute the production browser console for every user on the Key Activity tab.
  • Two console.log calls in the UsagePageView component body (logging currentUser JSON) fire on every render, including every pagination tick — a performance concern in addition to production noise.
  • EntityUsage.tsx does not destructure the loading return value from usePaginatedDailyActivity, so there is no loading indicator during the initial page-1 fetch. Users see all metric cards at $0 / 0 requests / 0 tokens and blank charts until page 1 resolves, which is a UX regression.
  • RENDER_BATCH_SIZE = 3 means the first chart update after page 1 is delayed by at least 900 ms (3 pages × 300 ms inter-page delay, plus network latency), despite the feature being described as "progressive" loading. The PR description also states "500ms delay" while the code constant is 300 ms.
  • Several deeper issues (stale Promise leaks on cancel, isDateChanging not clearing on empty results, spurious page-1 requests on date change while aggregated endpoint is unavailable) were identified in previous review threads and remain open.

Confidence Score: 2/5

  • Not safe to merge — debug logging in production and a missing loading state in EntityUsage are pre-merge blockers, and several hook-level issues from prior review threads remain unaddressed.
  • The core pagination logic and cancellation mechanism are sound, and the test suite is solid. However, the PR ships with multiple console.log debug statements that will appear in every user's browser console, a missing loading state in EntityUsage.tsx that causes a flash of zeroed-out metric cards on every data load, and a batch-size configuration that undermines the "progressive" UX goal. Combined with the unresolved issues from previous review threads (Promise leaks, isDateChanging never clearing on empty results, spurious extra requests on date change), this PR needs another pass before it is production-ready.
  • EntityUsage.tsx (debug logs + missing loading state) and UsagePageView.tsx (debug logs in component body) need the most attention before merging.

Important Files Changed

Filename Overview
ui/litellm-dashboard/src/components/UsagePage/hooks/usePaginatedDailyActivity.ts New reusable hook for sequential page fetching with cancellation; has known issues from previous threads (stale Promise leak on cancel, intermediate state flicker, etc.) plus a RENDER_BATCH_SIZE that delays first chart update by 900ms+.
ui/litellm-dashboard/src/components/UsagePage/components/EntityUsage/EntityUsage.tsx Refactored to use usePaginatedDailyActivity hook, fixing the page-1-only bug; however, it omits the loading return value (causing a flash of empty/zero data on initial load) and retains four debug console.log("debugTags", ...) statements that will appear in production.
ui/litellm-dashboard/src/components/UsagePage/components/UsagePageView.tsx Wires up usePaginatedDailyActivity as an aggregated-endpoint fallback with progressive rendering and a Stop button; two debug console.log calls in the component body fire on every render, and the isDateChanging clearance guard has a known empty-results bug.
ui/litellm-dashboard/src/components/UsagePage/components/UsagePageView.test.tsx Comprehensive test suite covering aggregated fallback, multi-page pagination, admin/non-admin flows, and view switching; all network calls are mocked, no real requests.
ui/litellm-dashboard/tests/CreateKeyPage.expiredToken.test.tsx New mock-only tests for expired token redirect and valid token rendering; correctly mocks daily activity calls required by the newly wired components, no real network calls.

Sequence Diagram

sequenceDiagram
    participant U as UI Component
    participant H as usePaginatedDailyActivity
    participant API as Daily Activity API

    U->>H: enabled=true, fetchFn, args
    H->>H: increment fetchId, reset state
    H->>H: setLoading(true)
    H->>API: fetchFn(args, page=1)
    API-->>H: { results, metadata: { total_pages: N } }
    H->>H: setData(firstPage)
    H->>H: setLoading(false)

    alt total_pages == 1
        H->>U: { data, loading:false, isFetchingMore:false }
    else total_pages > 1
        H->>H: setIsFetchingMore(true)
        loop pages 2..N
            H->>H: await delay(300ms)
            H->>API: fetchFn(args, page=i)
            API-->>H: pageData
            H->>H: accumulate results + sumMetadata
            alt isLastPage OR isBatchBoundary (every 3 pages)
                H->>U: setData(accumulated), setProgress
            end
        end
        H->>H: setIsFetchingMore(false)
        H->>U: { data:full, isFetchingMore:false }
    end

    alt User clicks Stop
        U->>H: cancel()
        H->>H: cancelledRef=true, setCancelled(true), clearTimeout
        H->>U: { cancelled:true, partial data }
    end

    alt Component unmounts or params change
        H->>H: cleanup: increment fetchId (isStale=true), clearTimeout
        H->>H: In-flight fetches silently discard results
    end
Loading

Comments Outside Diff (2)

  1. ui/litellm-dashboard/src/components/UsagePage/components/EntityUsage/EntityUsage.tsx, line 218-257 (link)

    Debug console.log statements left in production code

    Four console.log("debugTags", ...) calls remain in getTopAPIKeys(). These will be included in the production bundle and will pollute the browser DevTools console of every user who views the Key Activity tab:

    • Line 218: console.log("debugTags", { spendData });
    • Line 223: console.log("debugTags", { entities });
    • Line 236: console.log("debugTags", { tagDictionary });
    • Line 257: console.log("debugTags", { keySpend });

    These appear to be debug instrumentation left over from development — "debugTags" is the tag prefix used throughout. All four should be removed before merging.

  2. ui/litellm-dashboard/src/components/UsagePage/components/UsagePageView.tsx, line 84-85 (link)

    Debug console.log calls in component body fire on every render

    These two console.log calls sit directly in the component function body (not inside a useEffect), so they execute on every render — date changes, user selection changes, data updates, and progressive pagination ticks all trigger them:

    console.log(`currentUser: ${JSON.stringify(currentUser)}`);
    console.log(`currentUser max budget: ${currentUser?.max_budget}`);

    JSON.stringify(currentUser) will serialize the full user object on every render, which is also a non-trivial allocation. Both lines should be removed.

Last reviewed commit: d26faeb

Comment on lines +398 to +420
{(isFetchingMore || cancelled || agentIsFetchingMore || agentCancelled) && (
<div className="flex items-center gap-2 text-sm text-gray-500 mb-2">
{isFetchingMore && (
<>
<LoadingOutlined spin className="text-xs" />
<span>Loading spend data... (page {progress.currentPage}/{progress.totalPages})</span>
<button onClick={cancel} className="text-blue-600 hover:text-blue-800 underline text-xs">Stop</button>
</>
)}
{cancelled && (
<span className="text-yellow-600 text-xs">
Showing partial data ({progress.currentPage}/{progress.totalPages} pages loaded)
</span>
)}
{agentIsFetchingMore && entityType === "team" && (
<>
<LoadingOutlined spin className="text-xs" />
<span>Loading agent data... (page {agentProgress.currentPage}/{agentProgress.totalPages})</span>
<button onClick={agentCancel} className="text-blue-600 hover:text-blue-800 underline text-xs">Stop</button>
</>
)}
</div>
)}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agentCancelled state included in outer condition but never rendered

The outer condition evaluates agentCancelled, so when only agentCancelled is true (all other states are false), the container <div> is rendered — but there is no inner block that renders content for agentCancelled. This means a user who clicks "Stop" on agent data loading will see no feedback at all (empty div), which is confusing and inconsistent with the main spend data's cancelled state which shows a yellow partial-data notice.

Suggested change
{(isFetchingMore || cancelled || agentIsFetchingMore || agentCancelled) && (
<div className="flex items-center gap-2 text-sm text-gray-500 mb-2">
{isFetchingMore && (
<>
<LoadingOutlined spin className="text-xs" />
<span>Loading spend data... (page {progress.currentPage}/{progress.totalPages})</span>
<button onClick={cancel} className="text-blue-600 hover:text-blue-800 underline text-xs">Stop</button>
</>
)}
{cancelled && (
<span className="text-yellow-600 text-xs">
Showing partial data ({progress.currentPage}/{progress.totalPages} pages loaded)
</span>
)}
{agentIsFetchingMore && entityType === "team" && (
<>
<LoadingOutlined spin className="text-xs" />
<span>Loading agent data... (page {agentProgress.currentPage}/{agentProgress.totalPages})</span>
<button onClick={agentCancel} className="text-blue-600 hover:text-blue-800 underline text-xs">Stop</button>
</>
)}
</div>
)}
{(isFetchingMore || cancelled || agentIsFetchingMore || (agentCancelled && entityType === "team")) && (
<div className="flex items-center gap-2 text-sm text-gray-500 mb-2">
{isFetchingMore && (
<>
<LoadingOutlined spin className="text-xs" />
<span>Loading spend data... (page {progress.currentPage}/{progress.totalPages})</span>
<button onClick={cancel} className="text-blue-600 hover:text-blue-800 underline text-xs">Stop</button>
</>
)}
{cancelled && (
<span className="text-yellow-600 text-xs">
Showing partial data ({progress.currentPage}/{progress.totalPages} pages loaded)
</span>
)}
{agentIsFetchingMore && entityType === "team" && (
<>
<LoadingOutlined spin className="text-xs" />
<span>Loading agent data... (page {agentProgress.currentPage}/{agentProgress.totalPages})</span>
<button onClick={agentCancel} className="text-blue-600 hover:text-blue-800 underline text-xs">Stop</button>
</>
)}
{agentCancelled && entityType === "team" && (
<span className="text-yellow-600 text-xs">
Showing partial agent data ({agentProgress.currentPage}/{agentProgress.totalPages} pages loaded)
</span>
)}
</div>
)}

Comment on lines +191 to +197
} catch (error) {
if (!isStale()) {
console.error("Error fetching daily activity:", error);
setLoading(false);
setIsFetchingMore(false);
}
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No error state exposed — callers cannot distinguish empty data from a fetch failure

When the page-1 fetch throws, the hook swallows the error (only logs to console), resets loading and isFetchingMore, and leaves data as EMPTY_DATA. Callers receive { data: EMPTY_DATA, loading: false, ... } which is indistinguishable from a successful empty response.

In UsagePageView.tsx, this is important because the paginated hook is used as a fallback when the aggregated endpoint fails — if it then also fails silently, users see an empty chart with no error message. Consider adding an error: Error | null field to the return type and setting it here:

      } catch (error) {
        if (!isStale()) {
          console.error("Error fetching daily activity:", error);
          setError(error instanceof Error ? error : new Error(String(error)));
          setLoading(false);
          setIsFetchingMore(false);
        }
      }

Comment on lines +202 to +206
return () => {
fetchIdRef.current++;
};
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [enabled, fetchFn, ...args]);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spreading args into useEffect dep array is fragile

Using ...args in the dependency array is a known anti-pattern (hence the eslint-disable on the preceding line). React expects a statically-sized dependency array. If a future caller passes args of a different length between renders, the array changes size and React's element-by-element Object.is comparison can silently fail to detect changes — or re-run when nothing meaningful changed.

A safer pattern is to stabilize the comparison with a string key derived from the args, or to refactor the hook to accept individual named parameters instead of a variadic args[]. This would remove the need for the eslint-disable comment entirely.

Copy link
Copy Markdown
Contributor

@alvinttang alvinttang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review: Auto-paginate daily spend data

Good refactor — extracting the pagination logic into usePaginatedDailyActivity is a solid improvement over the inline useEffect + manual accumulation. A few observations:

1. args spread in useEffect deps causes infinite re-renders

In usePaginatedDailyActivity.ts line ~198, the effect dependency is:

}, [enabled, fetchFn, ...args]);

Since args is constructed inline in the parent as [accessToken, startTime, endTime, entityFilterArg], every render creates a new array with new object references (the Date objects from useMemo are stable, but entityFilterArg returns a new array selectedTags each time if .length > 0). This will cause the effect to fire on every render when tags are selected, because selectedTags is a new array reference each time.

Consider either:

  • Using JSON.stringify(args) as the dependency key, or
  • Having the caller pass a stable args reference via useMemo

In EntityUsage.tsx, entityFilterArg is already memoized, so it may be okay there, but in UsagePageView.tsx the args array [accessToken, startTime, endTime, effectiveUserId] is re-created each render. Since the primitive values are stable, spreading them individually is fine — but this is fragile if someone passes an object later.

2. agentCancelled shows in status bar even when entityType !== "team"

In EntityUsage.tsx, the outer condition checks agentCancelled:

{(isFetchingMore || cancelled || agentIsFetchingMore || agentCancelled) && (

But the inner rendering for agentCancelled is missing — there's no block that shows partial-data text for agentCancelled. If the agent fetch is cancelled, the outer condition is true, but nothing renders inside, leaving an empty div with gap-2.

3. No error state surfaced to the user

If a page fetch fails mid-pagination, the hook catches the error, logs it, and stops. But the UI has no indication that something went wrong — it just silently shows partial data without the yellow "partial data" badge (since cancelled is false; it was an error, not a user cancellation). Consider adding an error state to the hook return.

4. The as unknown as EntitySpendData casts are a code smell

const spendData = spendDataRaw as unknown as EntitySpendData;

This double-cast (as unknown as) bypasses TypeScript's structural checking entirely. If the hook returned a properly typed generic, this wouldn't be needed. Consider making the hook generic: usePaginatedDailyActivity<T>.

5. Missing cleanup of setTimeout in pagination loop

The PAGE_FETCH_DELAY_MS delay uses:

await new Promise((resolve) => setTimeout(resolve, PAGE_FETCH_DELAY_MS));

When the effect cleanup runs (fetchIdRef.current++), any in-flight setTimeout will still fire and attempt the next fetch. The isStale() check after the delay handles this, but the timeout itself is not cleared. Under rapid re-renders this means you could have many orphaned timeouts queued up. Consider using AbortController or storing the timeout ID so cleanup can clearTimeout.

6. Minor: handleDateChange in UsagePageView lost setLoading(true)

The old handleDateChange called setLoading(true) for instant feedback. The new version only sets setIsDateChanging(true). Since loading is now derived from aggregatedLoading || paginatedResult.loading, there may be a brief moment where isDateChanging is true but loading is false (before the useEffect fires the aggregated fetch), which could cause the loading skeleton/overlay to flicker.

Overall this is a nice architectural improvement. The main risk is the effect dependency stability issue (point 1).

The test's partial vi.mock of @/components/networking was missing the daily
activity call exports now imported by EntityUsage via ENTITY_FETCH_FNS.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
Comment on lines +222 to +226
useEffect(() => {
if (aggregatedFailed && !paginatedResult.loading && paginatedResult.data.results.length > 0) {
setIsDateChanging(false);
}
}, [aggregatedFailed, paginatedResult.loading, paginatedResult.data.results.length]);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isDateChanging never cleared when paginated fallback returns empty results

The useEffect that clears isDateChanging guards on paginatedResult.data.results.length > 0:

if (aggregatedFailed && !paginatedResult.loading && paginatedResult.data.results.length > 0) {
  setIsDateChanging(false);
}

When a user has no activity in the selected date range AND the aggregated endpoint is unavailable, the paginated hook will finish loading (loading = false) but results.length will be 0. The condition is never satisfied, so isDateChanging remains true indefinitely. The ChartLoader overlays (lines 702, 782, 832) keep spinning and the user never sees the "no data" state.

The length guard is unnecessary — completion of the loading phase (regardless of whether results are empty) should be enough to clear the spinner:

Suggested change
useEffect(() => {
if (aggregatedFailed && !paginatedResult.loading && paginatedResult.data.results.length > 0) {
setIsDateChanging(false);
}
}, [aggregatedFailed, paginatedResult.loading, paginatedResult.data.results.length]);
if (aggregatedFailed && !paginatedResult.loading) {
setIsDateChanging(false);
}

1. Replace ...args spread in useEffect deps with JSON.stringify(args) key
   to prevent infinite re-renders when callers pass unstable array references.
2. Add missing agentCancelled partial-data message in EntityUsage so the
   outer condition no longer renders an empty div.
3. Store setTimeout ID in a ref and clearTimeout on cleanup/cancel to avoid
   orphaned timers under rapid re-renders.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
Comment on lines +206 to +210
const paginatedResult = usePaginatedDailyActivity({
fetchFn: userDailyActivityCall,
args: [accessToken, startTime, endTime, effectiveUserId],
enabled: aggregatedFailed && !!accessToken && !!startTime && !!endTime,
});
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spurious paginated page-1 request on every date-range change while aggregated endpoint is failed

When aggregatedFailed is true (from a previous request cycle) and the user changes the date range, the sequence is:

  1. React re-renders with the new startTime/endTimeaggregatedFailed is still true from the previous render's committed state.
  2. The paginated hook sees enabled = true and a new argsKey → its effect fires, launching a page-1 request with the new dates.
  3. The aggregated useEffect also runs and calls setAggregatedFailed(false) (line 200).
  4. On the next render aggregatedFailed = falseenabled = false → the hook's cleanup increments fetchIdRef.current, making isStale() return true and discarding the response.

The result is one wasted page-1 network call every time the user changes the date range while the aggregated endpoint remains unavailable. In rate-limited or slow environments this adds unnecessary latency and backend load.

A simple fix is to reset aggregatedFailed synchronously (via a useRef) before the effect fires, so enabled is already false on the render that precedes the new fetch:

const aggregatedFailedRef = useRef(false);

// In the aggregated useEffect, set the ref synchronously:
aggregatedFailedRef.current = false;
setAggregatedFailed(false);
setAggregatedData(null);
// ...

// And derive `enabled` from the ref, not state:
const paginatedEnabled =
  aggregatedFailedRef.current && !!accessToken && !!startTime && !!endTime;

Alternatively, you could gate the paginated hook on an additional startTime/endTime ref snapshot that is compared after the aggregated result is known, but the useRef approach above is simpler.

Comment on lines +151 to +173
const run = async () => {
const currentArgs = argsRef.current;
setLoading(true);
setIsFetchingMore(false);
setProgress({ currentPage: 1, totalPages: 1 });

try {
// Inject page=1 as the 4th argument.
const argsWithPage = [...currentArgs.slice(0, 3), 1, ...currentArgs.slice(3)];
const firstPage = await fetchFn(...argsWithPage);

if (isStale()) return;

setData(firstPage);

const totalPages = firstPage.metadata?.total_pages || 1;

setProgress({ currentPage: 1, totalPages });

if (totalPages <= 1) {
setLoading(false);
return;
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

loading is set to false before isFetchingMore is set to true, exposing a brief intermediate state

After the first page resolves and there are more pages:

setLoading(false);     // line 176
setIsFetchingMore(true); // line 177

These are two separate setState calls inside an async function body. React batches them in React 18 concurrent mode, but callers can still see a render where both loading and isFetchingMore are false simultaneously — i.e. the hook appears to be "done" with no data beyond page 1. Callers in UsagePageView.tsx derive the global loading flag as:

const loading = aggregatedLoading || paginatedResult.loading;

If a spinner is conditional on loading, it briefly disappears between the first-page commit and the isFetchingMore=true commit, causing a flicker. Consider using a single setData/setLoading/setIsFetchingMore batch, or use useReducer to make the transition atomic.

Comment on lines +195 to +202
accumulatedResults = [...accumulatedResults, ...pageData.results];
accumulatedMetadata = sumMetadata(
accumulatedMetadata,
pageData.metadata,
);
accumulatedMetadata.total_pages = totalPages;
accumulatedMetadata.has_more = page < totalPages;
accumulatedMetadata.page = page;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Metadata double-counts page-1 totals when accumulating subsequent pages

accumulatedMetadata is initialized from firstPage.metadata (which already contains the page-1 totals). The sumMetadata helper then adds each subsequent page's totals on top of that. So the final metadata correctly reflects pages 1 through N.

However, firstPage.metadata.total_pages, has_more, and page fields are preserved from the initial spread but then overwritten on each subsequent page iteration:

accumulatedMetadata.total_pages = totalPages;   // line 200 — ok, fixed value
accumulatedMetadata.has_more = page < totalPages; // line 201 — correct at last page
accumulatedMetadata.page = page;                 // line 202 — this tracks the *latest fetched* page

On the very last iteration page === totalPages, so has_more = false and page = totalPages. This is technically correct, but accumulatedMetadata.page is set to the last fetched page number, not 1, which means callers that read metadata.page expecting "the current page" will see an unexpectedly large value. If any caller checks metadata.page === 1 to detect a single-page result, it will silently break. Consider either omitting these fields from the accumulated metadata or documenting the semantics explicitly.

Replace subtle loading text with antd Alert banners that clearly communicate
pagination status, and batch state flushes to reduce chart re-renders.

- Replace inline loading text with warning Alert banners showing progress,
  "open a new tab" link with ExportOutlined icon, and primary Stop button
- Batch setData calls every 5 pages instead of per-page to cut re-renders ~80%
- Reduce fetch delay from 500ms to 300ms for faster data loading
- Add "Charts will update periodically" messaging to set expectations
- Fix pre-existing TS error: Button icon prop was using render function instead of ReactNode

Co-Authored-By: Claude Opus 4.6 <[email protected]>
Comment on lines +118 to +152
const cancel = useCallback(() => {
cancelledRef.current = true;
setCancelled(true);
setIsFetchingMore(false);
if (delayTimerRef.current !== null) {
clearTimeout(delayTimerRef.current);
delayTimerRef.current = null;
}
}, []);

useEffect(() => {
if (!enabled) {
setData(EMPTY_DATA);
setLoading(false);
setIsFetchingMore(false);
setProgress({ currentPage: 0, totalPages: 0 });
setCancelled(false);
return;
}

const currentFetchId = ++fetchIdRef.current;
cancelledRef.current = false;
setCancelled(false);

const isStale = () =>
fetchIdRef.current !== currentFetchId || cancelledRef.current;

/** Cancellable delay that clears itself on cleanup. */
const delay = (ms: number) =>
new Promise<void>((resolve) => {
delayTimerRef.current = setTimeout(() => {
delayTimerRef.current = null;
resolve();
}, ms);
});
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clearTimeout on the delay leaves its Promise permanently unresolved, leaking the async run() closure

cancel() (and the effect cleanup at line 234) calls clearTimeout(delayTimerRef.current), but the Promise returned by delay() has no reject path and its resolve callback is only stored inside the timeout. Once the timeout is cleared, the Promise will never settle, and the run() async function remains suspended at await delay(...) indefinitely.

Each date-range change or Stop click while a delay is in progress leaves one leaked async-function frame on the heap. In long-running sessions with heavy pagination, this compounds. The fix is to keep a reference to resolve so cancel() can settle the Promise immediately:

const delayResolveRef = useRef<(() => void) | null>(null);

const delay = (ms: number) =>
  new Promise<void>((resolve) => {
    delayResolveRef.current = resolve;
    delayTimerRef.current = setTimeout(() => {
      delayTimerRef.current = null;
      delayResolveRef.current = null;
      resolve();
    }, ms);
  });

Then in cancel() and the effect cleanup:

clearTimeout(delayTimerRef.current);
delayTimerRef.current = null;
delayResolveRef.current?.(); // settle the promise so run() can exit
delayResolveRef.current = null;

Because isStale() is checked immediately after await delay(...), resolving early is safe — run() will return without calling any state setters.

…h banner

- Reduce RENDER_BATCH_SIZE from 5 to 3 for more frequent chart updates
- Add LoadingOutlined spinner at the start of all fetching Alert banners

Co-Authored-By: Claude Opus 4.6 <[email protected]>
Comment on lines +120 to 131
const {
data: spendDataRaw,
isFetchingMore,
progress,
cancelled,
cancel,
} = usePaginatedDailyActivity({
fetchFn,
args: [accessToken, startTime, endTime, entityFilterArg],
enabled,
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

loading state omitted — no indicator shown during initial page-1 fetch

loading is not destructured from the hook's return value:

const {
  data: spendDataRaw,
  isFetchingMore,  // true only for pages 2+
  progress,
  cancelled,
  cancel,
} = usePaginatedDailyActivity({ ... });

While page 1 is being fetched (loading = true, isFetchingMore = false), there is no visual feedback. The component renders immediately with EMPTY_DATA — all metric cards show $0, 0 requests, 0 tokens, and the charts are blank. Only after page 1 resolves does real data appear.

The old implementation using useState + useEffect would have started with the same empty-data state, but the original code's loading was block-fetching (no interim render of old data), whereas now the empty-zero render is prominently shown while waiting. For users on slower connections this is a noticeable regression.

The simplest fix is to add a loading skeleton or spinner when loading is true — e.g. wrapping the tab panels in a conditional that checks the loading state.

Suggested change
const {
data: spendDataRaw,
isFetchingMore,
progress,
cancelled,
cancel,
} = usePaginatedDailyActivity({
fetchFn,
args: [accessToken, startTime, endTime, entityFilterArg],
enabled,
});
const {
data: spendDataRaw,
loading,
isFetchingMore,
progress,
cancelled,
cancel,
} = usePaginatedDailyActivity({
fetchFn,
args: [accessToken, startTime, endTime, entityFilterArg],
enabled,
});

Comment on lines +12 to +13
/** Number of pages to accumulate before flushing to React state (reduces re-renders). */
const RENDER_BATCH_SIZE = 3;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RENDER_BATCH_SIZE = 3 delays first post-page-1 update by at least 900 ms

With RENDER_BATCH_SIZE = 3 and PAGE_FETCH_DELAY_MS = 300, the batch boundary check is:

const isBatchBoundary = (page - 1) % RENDER_BATCH_SIZE === 0;
// fires at pages 4, 7, 10, ...

For a response with ≥ 3 pages, users see the page-1 result immediately, then no further chart update for at least 3 × 300 ms = 900 ms (plus network latency for pages 2–4) before the next batch flush at page 4. During this gap, the progress counter increments (3/N → 4/N) but the chart doesn't update, which may confuse users into thinking the indicator is broken.

Also worth noting: the PR description mentions "500ms delay between requests" but the constant is 300. If 300 is the intended value, the description should be updated; if 500 was intended, the constant should be adjusted.

Consider lowering RENDER_BATCH_SIZE to 1 or 2 so charts update more visibly as data streams in, while still capping renders from very large paginations.

@yuneng-jiang yuneng-jiang merged commit 2c840f1 into litellm_internal_dev_03_13_2026 Mar 14, 2026
47 of 65 checks passed
@ishaan-berri ishaan-berri deleted the litellm_usage_page_auto_pagination branch March 26, 2026 22:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants