Skip to content

refactor: unify SSE+SWR data flow with useSSECacheSync hook#1719

Merged
yottahmd merged 3 commits intomainfrom
refactor/sse
Mar 4, 2026
Merged

refactor: unify SSE+SWR data flow with useSSECacheSync hook#1719
yottahmd merged 3 commits intomainfrom
refactor/sse

Conversation

@yottahmd
Copy link
Copy Markdown
Collaborator

@yottahmd yottahmd commented Mar 3, 2026

Replace per-component dual SSE+SWR coordination (isPaused, manual cache sync, data merge) with a shared useSSECacheSync hook that pushes SSE data into SWR's cache. This fixes false "external changes" popups after saves, stale data on SSE reconnect, and remote node SSE timeouts.

Summary by CodeRabbit

Release Notes

  • New Features

    • Enhanced real-time data synchronization across the application to keep information current.
  • Bug Fixes & Improvements

    • Improved reliability of data fetching with automatic fallback to polling when real-time connections are unavailable.
    • Streamlined data handling to ensure consistent, unified information sources throughout the application.
    • Optimized cache synchronization to reduce redundant data updates and improve responsiveness.

…nflict popups

Replace per-component dual SSE+SWR coordination (isPaused, manual cache
sync, data merge) with a shared useSSECacheSync hook that pushes SSE data
into SWR's cache. This fixes false "external changes" popups after saves,
stale data on SSE reconnect, and remote node SSE timeouts.
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Mar 3, 2026

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d0bf604c-efc0-401b-8933-f245f21154a6

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This pull request refactors frontend data-fetching patterns across multiple components to use a unified SSE-with-polling-fallback mechanism. A new useSSECacheSync hook is introduced to synchronize SSE updates with SWR cache, replacing manual polling and the deprecated useLastValidData hook. The backend consolidates SSE header setup in the handler before proxying occurs.

Changes

Cohort / File(s) Summary
Backend SSE Setup Consolidation
internal/service/frontend/sse/handler.go, internal/service/frontend/sse/proxy.go
Moved SSE header and write-deadline initialization to precede the remote-node check in handleSSE, eliminating duplicate setup code. Proxy handler now relies on headers being pre-configured rather than calling SetSSEHeaders itself.
Frontend Hook Infrastructure
ui/src/hooks/useSSECacheSync.ts, ui/src/hooks/useLastValidData.ts
Introduced new useSSECacheSync hook providing sseFallbackOptions() to configure SWR polling fallback based on SSE state, and useSSECacheSync() to synchronize cache with SSE updates. Deleted deprecated useLastValidData hook that cached last-valid data with manual reset logic.
DAG Feature Components
ui/src/features/dag-runs/components/dag-run-details/DAGRunDetailsPanel.tsx, ui/src/features/dags/components/dag-details/DAGDetailsModal.tsx, ui/src/features/dags/components/dag-details/DAGDetailsPanel.tsx, ui/src/features/dags/components/dag-editor/DAGSpec.tsx, ui/src/features/dags/components/dag-execution/DAGExecutionHistory.tsx
Replaced manual polling and useLastValidData with sseFallbackOptions() and useSSECacheSync() hooks; consolidated data source to SWR query result with SSE synchronization via cache mutations.
Frontend Pages
ui/src/pages/dag-runs/index.tsx, ui/src/pages/dags/dag/index.tsx, ui/src/pages/dags/index.tsx, ui/src/pages/docs/components/DocEditor.tsx, ui/src/pages/docs/index.tsx, ui/src/pages/index.tsx, ui/src/pages/queues/index.tsx
Unified data fetching to use SWR with SSE fallback; removed explicit polling configuration and separate polling-data sources; added useSSECacheSync() calls to keep cache coherent with SSE state, replacing previous useLastValidData-based patterns.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 22.22% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: introducing a useSSECacheSync hook to unify SSE and SWR data flow across the codebase, replacing per-component manual coordination.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch refactor/sse

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)
ui/src/features/dag-runs/components/dag-run-details/DAGRunDetailsPanel.tsx (1)

139-143: ⚠️ Potential issue | 🟠 Major

Avoid replacing panel content with LoadingIndicator on transient revalidation.

Line 139 now gates rendering on !data, so the panel can blank out and show a full loading indicator during updates/switches instead of retaining stale content.

💡 Proposed fix (keep last renderable data)
-  const data = isSubDAGRun ? subDAGQuery.data : dagRunQuery.data;
+  const data = isSubDAGRun ? subDAGQuery.data : dagRunQuery.data;
+  const lastDataRef = React.useRef<typeof data>(undefined);
+  if (data) {
+    lastDataRef.current = data;
+  }
+  const displayData = data ?? lastDataRef.current;
@@
-  if (!data) {
+  if (!displayData) {
     return (
       <div className="flex items-center justify-center h-full">
         <LoadingIndicator />
       </div>
     );
   }
@@
-            dagRun={data.dagRunDetails}
+            dagRun={displayData.dagRunDetails}

Based on learnings: Applies to ui/**/*.{ts,tsx} : "NEVER use full-page loading overlays or LoadingIndicator components that hide content; instead show stale data while fetching updates".

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@ui/src/features/dag-runs/components/dag-run-details/DAGRunDetailsPanel.tsx`
around lines 139 - 143, The component DAGRunDetailsPanel currently returns a
full-screen LoadingIndicator when data is falsy, causing the panel to blank
during transient revalidation; instead preserve and render the last known data
while fetching and only show the LoadingIndicator when there is genuinely no
prior data. Change the render guard around data in DAGRunDetailsPanel to keep
stale data: track previousData (via a useRef or useState) or use the query
hook's isLoading/isFetching flags, render previousData when data is undefined
but previousData exists, and only render the full LoadingIndicator when neither
data nor previousData exist; remove any logic that unmounts panel content on
transient revalidation.
ui/src/features/dags/components/dag-details/DAGDetailsPanel.tsx (1)

166-170: ⚠️ Potential issue | 🟠 Major

Do not replace the details panel with LoadingIndicator.

Line 169 hides panel content behind a loading state. Preserve stale content and use inline loading cues during refreshes instead of swapping out the panel.

As per coding guidelines ui/**/*.{ts,tsx}: “NEVER use full-page loading overlays or LoadingIndicator components that hide content; instead show stale data while fetching updates”.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@ui/src/features/dags/components/dag-details/DAGDetailsPanel.tsx` around lines
166 - 170, The current DAGDetailsPanel returns a full-panel LoadingIndicator
when data?.dag is missing, which hides stale content; instead modify
DAGDetailsPanel to never replace the panel with a full-page loader—render the
existing panel UI even when data?.dag is undefined or being refreshed and show
inline refresh/loading cues (e.g., a small spinner or skeleton in the
header/fields) while fetching. Locate the early-return that checks data?.dag and
remove it; ensure the component renders the same container/structure and
conditionally displays inline loading state based on the fetch status (use the
existing data, a stale fallback, or isFetching flag) so the panel content
remains visible during updates.
ui/src/pages/dags/index.tsx (1)

314-348: ⚠️ Potential issue | 🟠 Major

Avoid hiding the panel behind LoadingIndicator on missing data.

Line 347 fully replaces the list content with a loading indicator. Keep stale/previous list content visible and use inline loading affordances instead of swapping out the panel content.

As per coding guidelines ui/**/*.{ts,tsx}: “NEVER use full-page loading overlays or LoadingIndicator components that hide content; instead show stale data while fetching updates”.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@ui/src/pages/dags/index.tsx` around lines 314 - 348, The current conditional
replaces the entire panel with <LoadingIndicator /> when data is missing;
instead always render the existing panel (DAGErrors and DAGTable) and only show
an inline loading affordance when isLoading is true. Concretely, remove the
outer ternary that returns <LoadingIndicator /> and render DAGErrors and
DAGTable unconditionally (using data.dags || [] and data.errors || [] as now),
and add an inline spinner or disabled/loading state inside DAGTable or adjacent
to its header when isLoading is true; if data can be undefined during fetch,
keep previous data in local state (e.g., a retainedDags/ref) or render empty
arrays rather than hiding the panel so stale content remains visible while
fetching.
🧹 Nitpick comments (1)
ui/src/features/dags/components/dag-editor/DAGSpec.tsx (1)

185-187: Await post-save revalidation to reduce stale flashes.

Line 186 triggers revalidation but does not await completion. Awaiting here gives a cleaner post-save state transition.

Proposed patch
-    mutateSpec();
+    await mutateSpec();
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@ui/src/features/dags/components/dag-editor/DAGSpec.tsx` around lines 185 -
187, The code calls mutateSpec() to revalidate the SWR cache but does not await
it, causing potential stale UI flashes; update the post-save flow (where
mutateSpec() is invoked, e.g., in the save handler or function that performs the
save) to await the revalidation by using await mutateSpec() (or return the
promise) so the UI waits for the server-side revalidation to complete before
transitioning out of the saving state.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@ui/src/features/dags/components/dag-editor/DAGSpec.tsx`:
- Around line 101-105: The SSE cache transform currently forces spec to an empty
string which overwrites the existing cached spec when the SSE payload omits it;
update the transform passed to useSSECacheSync (the call using sseResult and
mutateSpec) so that spec is only set when sseData.spec is defined (e.g., leave
it undefined or omit the field when sseData.spec === undefined) to preserve the
cached value and avoid false external-modification conflicts in
useContentEditor; reference the useSSECacheSync call and the sseData.spec field
from the DAGSSEResponse shape when making this change.

In `@ui/src/hooks/useSSECacheSync.ts`:
- Around line 39-52: The effect only re-maps SSE data when the data reference
changes, so when transform changes the SWR cache can become stale; inside the
useEffect that references prevDataRef, add a prevTransformRef
(useRef<TransformType | null>) and update it alongside prevDataRef, and change
the condition to also trigger when prevTransformRef.current !== transform (or
when transform is non-null and differs) so you recompute cacheData = transform ?
transform(sseResult.data) : sseResult.data and call mutate; also add transform
to the dependency list (it already is) and ensure you set
prevTransformRef.current = transform after mutating to avoid repeated work.

---

Outside diff comments:
In `@ui/src/features/dag-runs/components/dag-run-details/DAGRunDetailsPanel.tsx`:
- Around line 139-143: The component DAGRunDetailsPanel currently returns a
full-screen LoadingIndicator when data is falsy, causing the panel to blank
during transient revalidation; instead preserve and render the last known data
while fetching and only show the LoadingIndicator when there is genuinely no
prior data. Change the render guard around data in DAGRunDetailsPanel to keep
stale data: track previousData (via a useRef or useState) or use the query
hook's isLoading/isFetching flags, render previousData when data is undefined
but previousData exists, and only render the full LoadingIndicator when neither
data nor previousData exist; remove any logic that unmounts panel content on
transient revalidation.

In `@ui/src/features/dags/components/dag-details/DAGDetailsPanel.tsx`:
- Around line 166-170: The current DAGDetailsPanel returns a full-panel
LoadingIndicator when data?.dag is missing, which hides stale content; instead
modify DAGDetailsPanel to never replace the panel with a full-page loader—render
the existing panel UI even when data?.dag is undefined or being refreshed and
show inline refresh/loading cues (e.g., a small spinner or skeleton in the
header/fields) while fetching. Locate the early-return that checks data?.dag and
remove it; ensure the component renders the same container/structure and
conditionally displays inline loading state based on the fetch status (use the
existing data, a stale fallback, or isFetching flag) so the panel content
remains visible during updates.

In `@ui/src/pages/dags/index.tsx`:
- Around line 314-348: The current conditional replaces the entire panel with
<LoadingIndicator /> when data is missing; instead always render the existing
panel (DAGErrors and DAGTable) and only show an inline loading affordance when
isLoading is true. Concretely, remove the outer ternary that returns
<LoadingIndicator /> and render DAGErrors and DAGTable unconditionally (using
data.dags || [] and data.errors || [] as now), and add an inline spinner or
disabled/loading state inside DAGTable or adjacent to its header when isLoading
is true; if data can be undefined during fetch, keep previous data in local
state (e.g., a retainedDags/ref) or render empty arrays rather than hiding the
panel so stale content remains visible while fetching.

---

Nitpick comments:
In `@ui/src/features/dags/components/dag-editor/DAGSpec.tsx`:
- Around line 185-187: The code calls mutateSpec() to revalidate the SWR cache
but does not await it, causing potential stale UI flashes; update the post-save
flow (where mutateSpec() is invoked, e.g., in the save handler or function that
performs the save) to await the revalidation by using await mutateSpec() (or
return the promise) so the UI waits for the server-side revalidation to complete
before transitioning out of the saving state.

ℹ️ Review info

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9385988 and e0d174f.

📒 Files selected for processing (16)
  • internal/service/frontend/sse/handler.go
  • internal/service/frontend/sse/proxy.go
  • ui/src/features/dag-runs/components/dag-run-details/DAGRunDetailsPanel.tsx
  • ui/src/features/dags/components/dag-details/DAGDetailsModal.tsx
  • ui/src/features/dags/components/dag-details/DAGDetailsPanel.tsx
  • ui/src/features/dags/components/dag-editor/DAGSpec.tsx
  • ui/src/features/dags/components/dag-execution/DAGExecutionHistory.tsx
  • ui/src/hooks/useLastValidData.ts
  • ui/src/hooks/useSSECacheSync.ts
  • ui/src/pages/dag-runs/index.tsx
  • ui/src/pages/dags/dag/index.tsx
  • ui/src/pages/dags/index.tsx
  • ui/src/pages/docs/components/DocEditor.tsx
  • ui/src/pages/docs/index.tsx
  • ui/src/pages/index.tsx
  • ui/src/pages/queues/index.tsx
💤 Files with no reviewable changes (1)
  • ui/src/hooks/useLastValidData.ts

Comment thread ui/src/features/dags/components/dag-editor/DAGSpec.tsx Outdated
Comment thread ui/src/hooks/useSSECacheSync.ts Outdated
yottahmd added 2 commits March 4, 2026 00:24
Add centralized SSEManager singleton that manages EventSource instances
with ref-counted deduplication, a cap of 3 concurrent connections with
LRU eviction, and a 15s connect timeout for browser-queued connections.
Evicted or timed-out connections gracefully fall back to SWR polling via
the existing sseFallbackOptions mechanism.
@yottahmd yottahmd merged commit b748ae5 into main Mar 4, 2026
6 checks passed
@yottahmd yottahmd deleted the refactor/sse branch March 4, 2026 09:37
@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 4, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.52%. Comparing base (9385988) to head (32af3b5).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #1719   +/-   ##
=======================================
  Coverage   69.52%   69.52%           
=======================================
  Files         396      396           
  Lines       43783    43783           
=======================================
  Hits        30440    30440           
+ Misses      10894    10891    -3     
- Partials     2449     2452    +3     

see 9 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9385988...32af3b5. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant