Skip to content

console: add a memory by object card to the redesigned cluster page - #39510

Merged
leedqin merged 1 commit into
cluster-page-resource-usagefrom
cluster-page-memory-by-object
Oct 21, 2026
Merged

leedqin merged 1 commit into
cluster-page-resource-usagefrom
cluster-page-memory-by-object

Conversation

@leedqin

@leedqin leedqin commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

Fourth PR in the cluster details redesign stack, on top of #39508. The design replaces the resource intensive objects table with a memory by object chart, so the Metrics tab shows where a replica's memory goes next to the resource usage card. Fixes CNS-175

Description

  • On the new page's Metrics tab, a card titled Top 10 Objects by Memory (or Objects by Memory when there are fewer than 10) replaces the resource intensive objects table.
  • It reads the same query as the table: the largest objects (up to 10) on the replica the page's Replica select names, or on the cluster's largest replica under All replicas, each as a share of that replica's heap limit. LargestReplicaBoundary and buildLargestClusterReplicaQuery take an optional replica name, which reads that replica with the same heap limit.
  • One bar of the heap limit is split by object, with ticks at 0 to 100%. Hovering a segment names the object. The legend gives each object's size and share, formatted as the table did. Objects are colored by rank with the Freshness page's palette (generateRainbowPalette).
  • Each object links to its Maintained Objects drawer (maintained-objects/:objectId). While maintained-objects-ui-50 is off that route doesn't exist, so the link goes to the object's workflow graph, as the table's rows did. A dropped object's dataflow keeps its name and has no link.
  • Without a heap limit there are no shares to draw, so the card lists each object's size alone, with no bar. That covers a replica that hasn't reported a limit yet, and the local process orchestrator, which never reports one. The older query sorts by size, so the top 10 stay the largest in that case.
  • The card's title and frame stay in place while it loads, when the objects fail to load, and when the replica has no objects in memory, which it says.
  • The largest replica's loading and error states move from LargestMaintainedQueries into LargestReplicaBoundary, which the table and the card share. The boundary takes an optional wrapper for its fallbacks, which the card uses for its frame.
  • The largest replica's heap limit now covers all of its processes: the per-process heap_limit times the size's process count from mz_cluster_replica_sizes. Object sizes are replica-wide, so on a multi-process replica every share was inflated by the process count, which also affected the classic table. The process count comes from the size rather than a sum over mz_cluster_replica_metrics rows, because a process whose metrics fetch failed has no limit to sum. Checked on a staging environment: a 2-process replica's limit doubles, and EXPLAIN adds only a differential join on mz_cluster_replica_sizes_ind.

The design plots memory by object over time. Per-object memory is only reported as a current value by default: mz_object_arrangement_size_history isn't collected unless arrangement_size_history_collection_interval is set, and it samples too coarsely for short windows. So the card shows the current snapshot for now.

Not carried over from the table: the freshness column and the hint that memory grows until hydration finishes. The card follows #39508's Replica select rather than having its own dropdown.

Verification

image

MemoryByObject.test.tsx covers:

  • the title, the replica, and each object's size and share;
  • the named replica instead of the largest;
  • links to the Maintained Objects drawer, and to the workflow graph while that page is off;
  • a dropped object's dataflow shown without a link;
  • sizes without a bar when the replica reports no heap limit;
  • the card's title while loading, and when the replica has no objects;
  • nothing rendered at all for a cluster without replicas, once the replica query has answered;
  • the busy-replica message, inside the card, when the objects fail to load;
  • objects read from mz_object_arrangement_sizes on v26.35 and later, with a dropped object labelled by its id;
  • the bar stopping at the heap limit.

LargestMaintainedQueries.test.tsx passes unchanged after the shared boundary moved out.

largestClusterReplica.test.sql.ts covers a replica whose size runs four processes, and a named replica read with its own heap limit.

🤖 Generated with Claude Code

@leedqin
leedqin added this pull request to stack #39509 October 2, 2026 23:31
@leedqin
leedqin force-pushed the cluster-page-memory-by-object branch from 51e5abe to 32c8f7d Compare October 2, 2026 23:42
@leedqin
leedqin force-pushed the cluster-page-memory-by-object branch from 32c8f7d to 9c821e9 Compare October 2, 2026 23:51
@leedqin
leedqin force-pushed the cluster-page-memory-by-object branch from 9c821e9 to 66f1ba1 Compare October 3, 2026 00:01
@leedqin
leedqin force-pushed the cluster-page-memory-by-object branch 2 times, most recently from f4c71cc to e754e35 Compare October 3, 2026 16:59
@leedqin
leedqin force-pushed the cluster-page-memory-by-object branch 2 times, most recently from 82c98dc to 31eae67 Compare October 5, 2026 15:19
@leedqin
leedqin marked this pull request as ready for review October 5, 2026 15:31
@leedqin
leedqin requested a review from a team as a code owner October 5, 2026 15:31
@leedqin
leedqin requested a review from jdonelson October 5, 2026 15:31
@linear-code

linear-code Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

CNS-173

CNS-175

<VStack spacing={4} alignItems="stretch">
<HStack spacing={2} alignItems="baseline" flexWrap="wrap">
<Text as="h3" textStyle="heading-sm">
Memory consumption by object ({objects.length})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The count is the number the query returns, which caps at 10, so a cluster with hundreds of objects shows "(10)". Something like "Top 10 objects by memory" would avoid reading it as a total.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to "Top 10 objects by memory".

@jdonelson

Copy link
Copy Markdown
Contributor

I have some layout-related feedback:

Screenshot 2026-10-07 at 4 39 40 PM Screenshot 2026-10-07 at 4 39 52 PM

The legends for these 2 graphs are tough to read. On the freshness graph it overflows to the left. Maybe remove them both and move the color swatches to the Freshness table, like we did on the new Freshness page? You'd have to make each object use the same color on both graphs.

@leedqin
leedqin force-pushed the cluster-page-memory-by-object branch from a7b956d to 792d44c Compare October 8, 2026 03:48
@leedqin

leedqin commented Oct 8, 2026

Copy link
Copy Markdown
Contributor Author

I have some layout-related feedback: @jdonelson

I have a followup PR coming to remove this Freshness section so you can add your Freshness tab/ graph once your page is shipped and I agree that we can use the Freshness Swatch legend

@leedqin
leedqin force-pushed the cluster-page-memory-by-object branch from 792d44c to 1492640 Compare October 8, 2026 04:11
@leedqin

leedqin commented Oct 8, 2026

Copy link
Copy Markdown
Contributor Author

Thanks. This PR now colors the card's objects by rank with the Freshness page's palette (generateRainbowPalette), so it matches that page. The legends and the Freshness table are unchanged here.

@leedqin
leedqin force-pushed the cluster-page-memory-by-object branch from 4d5b210 to a11a8cd Compare October 17, 2026 16:35
@leedqin
leedqin force-pushed the cluster-page-memory-by-object branch from a11a8cd to fe36754 Compare October 17, 2026 18:04
@leedqin
leedqin force-pushed the cluster-page-memory-by-object branch from fe36754 to c78babc Compare October 19, 2026 15:51
@leedqin
leedqin force-pushed the cluster-page-memory-by-object branch 2 times, most recently from 6076e1a to f712927 Compare October 19, 2026 22:03
@leedqin
leedqin force-pushed the cluster-page-memory-by-object branch 2 times, most recently from 2723318 to a9f5a42 Compare October 20, 2026 18:32
@leedqin
leedqin force-pushed the cluster-page-memory-by-object branch from a9f5a42 to 09347d7 Compare October 20, 2026 19:39
@leedqin
leedqin force-pushed the cluster-page-memory-by-object branch from 09347d7 to 6ee99fb Compare October 20, 2026 20:31
@leedqin
leedqin force-pushed the cluster-page-memory-by-object branch from 6ee99fb to d991319 Compare October 20, 2026 23:23
@def-

def- commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

QA LLM Review

1. MEDIUM -- After one load error the card ignores the Replica select and blames whichever replica is picked next

console/src/platform/clusters/ClusterPage/MemoryByObject.tsx:58

The card now follows the page's Replica select, but LargestReplicaBoundary's two AppErrorBoundarys never reset when replicaName changes. If one replica's objects fail to load, switching to another replica leaves the card stuck on the "might mean it's busy" alert, now naming the new replica, which was never queried. Polling stops too, so the card only recovers after navigating away or reloading.

Details

Sentry's ErrorBoundary (AppErrorBoundary, console/src/components/AppErrorBoundary.tsx:52) has no resetKeys, and it clears its error state only through resetError(), which LargestMaintainedQueriesError never calls. MemoryByObject is rendered without a key (ClusterPage/ClusterMetrics.tsx:81), so changing the select re-renders the same boundaries. The inner boundary at LargestMaintainedQueries.tsx:184 keeps its error and renders its fallback with the new largestReplica.name. PolledMemoryByObject stays unmounted, so the new replica's objects are never fetched.

I reproduced this with a vitest probe at this head. replicaName="r1" with a failing objects query renders "…from replica r1, which might mean it's busy". After switching to "r2", whose handlers succeed, the card still shows the alert, now reading "…from replica r2…", and "% of heap limit on r2" never appears. In a control run where r1 succeeds, the same switch shows r2's objects. This is the situation in which a user is most likely to try another replica: the alert says the first one is busy, and on pre-v26.35 environments the query runs on that replica itself.

A fix is to key the boundary by selection, for example <LargestReplicaBoundary key={replicaName ?? ""} …> in MemoryByObject. Also keying the inner boundary by largestReplica.name would cover the "All replicas" case when the largest replica changes between polls.

@leedqin
leedqin force-pushed the cluster-page-memory-by-object branch from d991319 to ffc7a58 Compare October 21, 2026 15:15
@leedqin
leedqin force-pushed the cluster-page-memory-by-object branch 2 times, most recently from 335a591 to 1b57241 Compare October 21, 2026 21:16
Fourth in the cluster details redesign stack. On the new page's Metrics
tab, a memory by object card replaces the resource intensive objects
table. It reads the same query: the largest objects on the cluster's
largest replica, as shares of that replica's heap limit. One bar of the
heap limit is split by object, and a legend gives each object's size and
share. Objects are colored by rank with the Freshness page's palette.

The design plots this over time, but per-object memory is only reported as
a current value by default (mz_object_arrangement_size_history isn't
collected unless arrangement_size_history_collection_interval is set), so
the card shows the current snapshot.

Each object links to its Maintained Objects drawer, or to its workflow
graph while the maintained-objects-ui-50 flag is off.

The largest replica's loading and error states move into
LargestReplicaBoundary so the classic table and the card share them.

The largest replica's heap limit now covers all of its processes: the
per-process limit times the size's process count from
mz_cluster_replica_sizes. Object sizes are replica-wide, so on a
multi-process replica every share was inflated by the process count, in
the classic table too.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants