Skip to content

Commit b85d680

Browse files
authored
perf(desktop): stop the Projects fan refetching on re-entry and running after leave (#6458)
Entering Projects fires a large fan: an exhaustive paginated relay enumeration (projects/repos/tombstones), five 2,000-event work-item queries plus assignment-operation scans, per-repo activity summaries, and a local-repository filesystem scan. Measured on a large community (101 issues / 258 PRs): | query | measured cost | |---|---| | work-items (5 × 2,000-event REQs + assignment scans) | 3.5–3.9s | | activity summaries | 4.1s | | repository activity | 1.0–2.2s | | local repository scan | 1.7s | Two lifecycle bugs made the fan far more expensive than it needs to be: - **Freshness windows guaranteed a full refetch on nearly every re-entry** (60s enumeration, 30s work-items/activity, 10s local scan) — i.e., the costs above were re-paid on almost every visit. Every local write path already invalidates its keys explicitly (issue/PR mutations, project creation, repo sync), so the short windows only served remote-actor freshness. Raised to 5m/2m/2m with a 30m enumeration cache: re-entries now paint from cache, and the fan re-runs at most every 2–5 minutes. - **Leaving Projects left the whole fan running**, competing with the next surface's channel fetches on the same relay connection. AbortSignal is now threaded through the enumeration and assignment pagination loops (optional params — behavior identical without a signal), and leaving the surface cancels the work-items query. Deliberately NOT cancelled: the enumeration (the always-mounted sidebar projects section observes it and its 30m cache is valuable), repo snapshots and local scans (native work that can't abort — cancelling would discard the finished result and force the same clones again), and activity summaries (a single bounded request). Abort behavior is covered by red-first unit tests on both pagination loops. Remaining follow-up (out of scope): the queries themselves want a relay-side aggregate instead of shipping thousands of events to compute counts client-side. --------- Signed-off-by: Max Lampert <[email protected]>
1 parent b0466ac commit b85d680

10 files changed

Lines changed: 172 additions & 13 deletions

‎desktop/src/features/projects/assignmentOperationFetch.test.mjs‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -285,3 +285,38 @@ test("a failed assignment query surfaces as a failed section instead of silent l
285285

286286
assert.ok(result.issues.failedSections.includes("assignments"));
287287
});
288+
289+
test("fetchAssignmentOperationEvents stops paginating once its signal aborts", async () => {
290+
// A permanently-full page would paginate forever without the cursor; abort
291+
// after the first page and require the loop to stop with AbortError.
292+
const controller = new AbortController();
293+
let fetches = 0;
294+
const fullPage = (until) =>
295+
Array.from({ length: 500 }, (_, index) => ({
296+
id: `${until ?? "head"}-${index}`.padEnd(64, "0"),
297+
kind: 1,
298+
pubkey: "a".repeat(64),
299+
created_at: 1_000_000 - fetches * 1_000 - index,
300+
content: JSON.stringify({ type: "assign" }),
301+
tags: [],
302+
}));
303+
const fetchEvents = async (filter) => {
304+
fetches += 1;
305+
// Bound the fake: without the abort support the loop would paginate
306+
// forever (each page has a fresh cursor) and OOM the test run.
307+
if (fetches > 3) throw new Error("kept paginating after abort");
308+
const page = fullPage(filter.until);
309+
controller.abort();
310+
return page;
311+
};
312+
313+
await assert.rejects(
314+
fetchAssignmentOperationEvents(
315+
["issue".padEnd(64, "1")],
316+
fetchEvents,
317+
controller.signal,
318+
),
319+
(error) => error.name === "AbortError",
320+
);
321+
assert.equal(fetches, 1);
322+
});

‎desktop/src/features/projects/assignmentOperationFetch.ts‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,14 +68,17 @@ export async function fetchAssignmentOperationEvents(
6868
fetchEvents: (
6969
filter: FetchEventsInput,
7070
) => Promise<RelayEvent[]> = relayClient.fetchEvents.bind(relayClient),
71+
signal?: AbortSignal,
7172
): Promise<RelayEvent[]> {
7273
if (issueIds.length === 0) return [];
7374
const chunks: string[][] = [];
7475
for (let i = 0; i < issueIds.length; i += ISSUE_ID_CHUNK_SIZE) {
7576
chunks.push(issueIds.slice(i, i + ISSUE_ID_CHUNK_SIZE));
7677
}
7778
const pages = await Promise.all(
78-
chunks.map((chunk) => fetchIssueCommentsExhaustively(chunk, fetchEvents)),
79+
chunks.map((chunk) =>
80+
fetchIssueCommentsExhaustively(chunk, fetchEvents, signal),
81+
),
7982
);
8083
const seen = new Map<string, RelayEvent>();
8184
for (const page of pages) {
@@ -91,11 +94,15 @@ export async function fetchAssignmentOperationEvents(
9194
async function fetchIssueCommentsExhaustively(
9295
issueIds: string[],
9396
fetchEvents: (filter: FetchEventsInput) => Promise<RelayEvent[]>,
97+
signal?: AbortSignal,
9498
): Promise<RelayEvent[]> {
9599
const seen = new Map<string, RelayEvent>();
96100
let limit = ASSIGNMENT_PAGE_LIMIT;
97101
let until: number | undefined;
98102
for (;;) {
103+
// Leaving the Projects surface cancels its queries; stop queuing pages
104+
// behind the next surface's fetches.
105+
signal?.throwIfAborted();
99106
const page = await fetchEvents({
100107
kinds: [KIND_TEXT_NOTE],
101108
"#e": issueIds,

‎desktop/src/features/projects/hooks.ts‎

Lines changed: 52 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -168,13 +168,18 @@ export function eventToProject(
168168
}
169169

170170
export async function fetchProjects(
171-
fetchExhaustively: FetchProjectEventsExhaustively = fetchProjectEventsExhaustively,
171+
fetchExhaustively?: FetchProjectEventsExhaustively,
172+
signal?: AbortSignal,
172173
): Promise<Project[]> {
173174
// Delegates to `buildProjectsFromFetcher` in `projectEnumeration.ts`, which
174175
// is the pure, Tauri-free core of this operation. That helper's javadoc
175176
// explains the fail-closed tombstone contract and the NIP-OA owner-deletion
176177
// relay-side-suppression decision.
177-
return buildProjectsFromFetcher(fetchExhaustively, {
178+
const fetcher: FetchProjectEventsExhaustively =
179+
fetchExhaustively ??
180+
((kinds, extraFilter) =>
181+
fetchProjectEventsExhaustively(kinds, extraFilter, undefined, signal));
182+
return buildProjectsFromFetcher(fetcher, {
178183
relayOrigin: getCachedRelayOrigin(),
179184
hiddenAddresses: new Set(readHiddenProjectCards()),
180185
});
@@ -637,22 +642,44 @@ async function deleteProject(project: Project): Promise<void> {
637642

638643
export const projectsQueryKey = ["projects"] as const;
639644

645+
/**
646+
* Freshness windows for the Projects surface. Every local write path
647+
* invalidates its keys explicitly (issue/PR mutations, project creation,
648+
* repo sync), so short windows bought nothing for own-actions and charged a
649+
* full relay fan-out — project enumeration is an exhaustive paginated scan,
650+
* work items are five 2,000-event queries — on nearly every Projects
651+
* re-entry. Remote actors' changes surface within the window. gcTime keeps
652+
* the enumeration cached across visits so re-entering Projects paints from
653+
* cache instead of blocking on the scan.
654+
*/
655+
export const PROJECTS_STALE_TIME_MS = 5 * 60_000;
656+
// Window-guarded like react-query's own server default (Infinity): an
657+
// explicit finite gcTime schedules a real, non-unref'd timeout per cache
658+
// entry, which keeps node test processes alive for the full 30 minutes.
659+
export const PROJECTS_GC_TIME_MS =
660+
typeof window === "undefined" ? Number.POSITIVE_INFINITY : 30 * 60_000;
661+
export const PROJECT_WORK_ITEMS_STALE_TIME_MS = 2 * 60_000;
662+
export const PROJECT_ACTIVITY_STALE_TIME_MS = 2 * 60_000;
663+
export const PROJECT_LOCAL_REPOS_STALE_TIME_MS = 2 * 60_000;
664+
640665
export function useProjectsQuery() {
641666
return useQuery({
642667
queryKey: projectsQueryKey,
643-
queryFn: () => fetchProjects(),
644-
staleTime: 60_000,
668+
queryFn: ({ signal }) => fetchProjects(undefined, signal),
669+
staleTime: PROJECTS_STALE_TIME_MS,
670+
gcTime: PROJECTS_GC_TIME_MS,
645671
});
646672
}
647673

648674
export function useProjectQuery(projectId: string) {
649675
return useQuery({
650676
queryKey: projectsQueryKey,
651-
queryFn: () => fetchProjects(),
677+
queryFn: ({ signal }) => fetchProjects(undefined, signal),
652678
select: (projects) =>
653679
projects.find((project) => projectMatchesRouteId(project, projectId)) ??
654680
null,
655-
staleTime: 60_000,
681+
staleTime: PROJECTS_STALE_TIME_MS,
682+
gcTime: PROJECTS_GC_TIME_MS,
656683
});
657684
}
658685

@@ -793,7 +820,8 @@ export function useProjectLocalRepositoriesQuery(reposDir?: string | null) {
793820
return useQuery({
794821
queryKey: ["projects", "local-repositories", reposDir ?? "default"],
795822
queryFn: () => listProjectLocalRepositories({ reposDir }),
796-
staleTime: 10_000,
823+
// Filesystem scan; repo sync/clone flows invalidate this key on change.
824+
staleTime: PROJECT_LOCAL_REPOS_STALE_TIME_MS,
797825
retry: 1,
798826
});
799827
}
@@ -828,9 +856,22 @@ export function useProjectPullRequestsQuery(
828856
export function useProjectsWorkItemsQuery(projects: Project[]) {
829857
return useQuery({
830858
enabled: projects.length > 0,
831-
queryKey: ["projects", "work-items", projects.map((project) => project.id)],
832-
queryFn: () => fetchProjectsWorkItems(projects),
833-
staleTime: 30_000,
859+
queryKey: [
860+
"projects",
861+
"work-items",
862+
projects.map((project) => project.id),
863+
// Repo attach/detach changes the fan-out inputs without changing
864+
// project ids; keying on addresses too prevents a pre-attach result
865+
// from serving as fresh for the whole staleTime window.
866+
projects
867+
.flatMap((project) =>
868+
project.repositories.map((repository) => repository.repoAddress),
869+
)
870+
.sort(),
871+
],
872+
queryFn: ({ signal }) =>
873+
fetchProjectsWorkItems(projects, undefined, signal),
874+
staleTime: PROJECT_WORK_ITEMS_STALE_TIME_MS,
834875
});
835876
}
836877

@@ -935,7 +976,7 @@ export function useProjectActivitySummariesQuery(projects: Project[]) {
935976
enabled: repoAddresses.length > 0,
936977
queryKey: ["projects", "activity-summaries", repoAddresses],
937978
queryFn: () => fetchProjectActivitySummaries(projects),
938-
staleTime: 30_000,
979+
staleTime: PROJECT_ACTIVITY_STALE_TIME_MS,
939980
});
940981
}
941982

‎desktop/src/features/projects/projectEnumeration.test.mjs‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -245,3 +245,31 @@ test("buildProjectsFromFetcher still suppresses deleted heads via the scoped fet
245245
const projects = await buildProjectsFromFetcher(fetchExhaustively);
246246
assert.deepEqual(projects, [], "deleted repo must not surface as a project");
247247
});
248+
249+
test("enumerateProjectEvents stops paginating once its signal aborts", async () => {
250+
// 3 full pages of 2 → without an abort the enumeration would fetch all of
251+
// them plus boundary drains. Abort after the first page: the loop must
252+
// throw before requesting another page.
253+
const events = [
254+
relayEvent("a", 1_000),
255+
relayEvent("b", 900),
256+
relayEvent("c", 800),
257+
relayEvent("d", 700),
258+
relayEvent("e", 600),
259+
relayEvent("f", 500),
260+
];
261+
const controller = new AbortController();
262+
let pageFetches = 0;
263+
const fetchPage = async (filter) => {
264+
pageFetches += 1;
265+
const page = await fetcherFor(events)(filter);
266+
controller.abort();
267+
return page;
268+
};
269+
270+
await assert.rejects(
271+
enumerateProjectEvents(fetchPage, [30617], 2, undefined, controller.signal),
272+
(error) => error.name === "AbortError",
273+
);
274+
assert.equal(pageFetches, 1);
275+
});

‎desktop/src/features/projects/projectEnumeration.ts‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ export async function enumerateProjectEvents(
4444
kinds: number[],
4545
pageSize: number,
4646
extraFilter?: ProjectEventExtraFilter,
47+
signal?: AbortSignal,
4748
): Promise<RelayEvent[]> {
4849
if (!Number.isSafeInteger(pageSize) || pageSize <= 0) {
4950
throw new Error(
@@ -55,6 +56,10 @@ export async function enumerateProjectEvents(
5556
let until: number | undefined;
5657

5758
for (;;) {
59+
// Each relay REQ is bounded, but this loop is not: leaving the Projects
60+
// surface cancels its queries, and the abort must stop the enumeration
61+
// from queuing further pages behind the next surface's fetches.
62+
signal?.throwIfAborted();
5863
const page = await fetchPage({
5964
...extraFilter,
6065
kinds,
@@ -65,6 +70,7 @@ export async function enumerateProjectEvents(
6570
if (page.length < pageSize) return [...eventsById.values()];
6671

6772
const oldest = Math.min(...page.map((event) => event.created_at));
73+
signal?.throwIfAborted();
6874
const boundary = await fetchPage({
6975
...extraFilter,
7076
kinds,
@@ -94,12 +100,14 @@ export function fetchProjectEventsExhaustively(
94100
kinds: number[],
95101
extraFilter?: ProjectEventExtraFilter,
96102
pageSize = PROJECT_ENUMERATION_PAGE_SIZE,
103+
signal?: AbortSignal,
97104
): Promise<RelayEvent[]> {
98105
return enumerateProjectEvents(
99106
(filter) => relayClient.fetchEvents(filter),
100107
kinds,
101108
pageSize,
102109
extraFilter,
110+
signal,
103111
);
104112
}
105113

‎desktop/src/features/projects/projectWorkItems.ts‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,7 @@ export async function fetchProjectsWorkItems<TProject extends ProjectReference>(
8282
fetchEvents: (
8383
filter: FetchEventsInput,
8484
) => Promise<RelayEvent[]> = relayClient.fetchEvents.bind(relayClient),
85+
signal?: AbortSignal,
8586
): Promise<ProjectsWorkItemsResult<TProject>> {
8687
const repoAddresses = [
8788
...new Set(
@@ -129,10 +130,17 @@ export async function fetchProjectsWorkItems<TProject extends ProjectReference>(
129130
.filter((event) => event.kind === KIND_GIT_ISSUE)
130131
.map((event) => event.id),
131132
fetchEvents,
133+
signal,
132134
),
133135
),
134136
]);
135137

138+
// The five eager queries above are single bounded REQs the relay client
139+
// cannot abort mid-flight; only the assignment pagination is abort-aware.
140+
// What cancellation CAN save here is the reduce work below and caching a
141+
// result for a surface the user already left.
142+
signal?.throwIfAborted();
143+
136144
if (rootResult.status === "rejected") {
137145
throw rootResult.reason instanceof Error
138146
? rootResult.reason

‎desktop/src/features/projects/repositoryActivityHooks.ts‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ import * as React from "react";
33

44
import {
55
fetchRepositoryActivitySummaries,
6+
PROJECT_ACTIVITY_STALE_TIME_MS,
67
type Project,
78
} from "@/features/projects/hooks";
89

@@ -27,6 +28,6 @@ export function useRepositoryActivitySummariesQuery(projects: Project[]) {
2728
enabled: repoAddresses.length > 0,
2829
queryKey: ["projects", "activity-summaries", "repositories", repoAddresses],
2930
queryFn: () => fetchRepositoryActivitySummaries(repositories),
30-
staleTime: 30_000,
31+
staleTime: PROJECT_ACTIVITY_STALE_TIME_MS,
3132
});
3233
}

‎desktop/src/features/projects/ui/ProjectsScreen.tsx‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,30 @@
1+
import { useQueryClient } from "@tanstack/react-query";
2+
import * as React from "react";
3+
14
import { ProjectsView } from "@/features/projects/ui/ProjectsView";
25

36
export function ProjectsScreen() {
7+
const queryClient = useQueryClient();
8+
React.useEffect(() => {
9+
return () => {
10+
// Leaving the surface: stop the work-items fan. Its assignment-operation
11+
// pagination is abort-aware and unbounded, so cancellation genuinely
12+
// stops relay work that would otherwise compete with the next surface's
13+
// channel fetches. Cached data stays; the next visit refetches per
14+
// staleTime.
15+
//
16+
// Deliberately NOT cancelled:
17+
// - projects enumeration: the always-mounted sidebar projects section
18+
// observes this key, so it must keep filling its 30-minute cache.
19+
// - repo snapshots / local repositories: native clone/scan work cannot
20+
// be aborted mid-flight; cancellation would only discard the finished
21+
// result and force the same clones again on the next visit.
22+
// - activity summaries: a single bounded request — it completes either
23+
// way, so cancelling just wastes the response.
24+
void queryClient.cancelQueries({ queryKey: ["projects", "work-items"] });
25+
};
26+
}, [queryClient]);
27+
428
return (
529
<div className="relative flex min-h-0 min-w-0 flex-1 flex-col overflow-hidden">
630
<ProjectsView />

‎desktop/src/shared/api/relayQueryInvalidation.test.mjs‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,9 @@ test("relay invalidation separates relay project queries from local repo work",
6363
["project", "project-1", "pull-requests"],
6464
["projects", "issues", ["project-1"]],
6565
["projects", "activity-summaries", ["addr-1"]],
66+
// Work items fan out over the relay and are fresh for two minutes;
67+
// reconnect auto-heal must be able to repair a partial result.
68+
["projects", "work-items", ["project-1"], ["addr-1"]],
6669
]) {
6770
assert.equal(isRelayDependentQueryKey(queryKey), true, queryKey.join("/"));
6871
}

‎desktop/src/shared/api/relayQueryInvalidation.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,10 @@ const RELAY_PROJECT_QUERY_PARTS = new Set<string>([
3939
"activity-summaries",
4040
"issues",
4141
"pull-requests",
42+
// Fresh for two minutes (PROJECT_WORK_ITEMS_STALE_TIME_MS) and tolerant of
43+
// partial fan-out failure — reconnect auto-heal is what repairs a partial
44+
// result inside that window.
45+
"work-items",
4246
]);
4347

4448
const LOCAL_PROJECT_QUERY_PARTS = new Set<string>([

0 commit comments

Comments
 (0)