feat: async chart pipeline with worker-owned grouping and perf improvements#125
Merged
Conversation
…ine) Snapshot of in-progress async-worker refactor before reworking swap-axis to move all per-setting computation (incl. grouping) into the worker. Includes new ui/src/lib/swap.ts and temporary debug logs (removed during the rework). Spec: requirement-swap-async.md
Arrangement-aware, non-mutating grouping (identityKeys -> targetKeys) that generalizes useDataPoint.grouped. Foundation for moving grouping into the worker. Unused for now.
The worker now owns the full raw dataset and treats arrangement, group, sort, scale and showLabels all as compute params. After the one-time init clone (and again only on a dataset switch), no setting change does any main-thread data work — swap/group/sort/scale just post a param and render the returned ChartData. - transform.worker.ts: own raw rows; init/setArrangement project+group via projectAndGroup and reply with groupNames; compute reads grouped.get(group). Remove the broken swap message and swapAxisFields. - useChartPipeline.ts: rawData/arrangement/labels/groupName inputs; reinit only on dataset change; arrangement posts setArrangement; group/sort/scale/ labels recompute off the cache; expose groupNames. Drop triggerSwap. - useDataPoint.ts: drop the 500k main-thread grouping computeds; add per-dataset arrangement state + activeArrangement; source groups from the worker via setGroupNames; keep resultGroups/activeGroupId/selectGroup working. - Dashboard.vue: pass raw data + arrangement + derived labels + groupName; mirror the pipeline's groupNames into useDataPoint. - AxisSwapper.vue: set the arrangement key instead of mutating rows/labels. - swap.ts: drop the now-unused mutating swapAxisFields.
Groups are populated asynchronously by the worker on first `ready`.
Register a one-shot watcher (Vue 3.5 `{ once: true }`) on
`resultGroups.value.length` so the `?g=N` deep-link is applied once
groups arrive, without delaying `?d=` / settings which remain synchronous.
Replace groupName: Ref<string> param with activeGroupId: Ref<number> in useChartPipeline. Resolve the group name internally via currentGroupName() (reads from the pipeline's own groupNames ref), so pumpQueue() called inside the ready handler uses the correct name synchronously — without waiting for a downstream watcher to propagate it. Previously, groupNames [] → [...] on first ready caused activeGroupName to transition '' → groupNames[0] in a later microtask, which tripped the params watch and triggered a full second recompute on every page load and every swap. Now the params watch tracks activeGroupId (an integer), which does not change on the first ready, so only one compute batch is queued. User group-select (activeGroupId change) still triggers a single recompute as expected. Also removes the now-unused activeGroupName computed from useDataPoint, and collapses a double blank line in AxisSwapper.vue.
Switch dataSets to shallowRef+markRaw so rows stay plain objects; postMessage clones them natively instead of the JSON round-trip (~7300ms → ~700ms for 500k rows). Drop console.info(e) in the worker that serialized the full message to devtools on every init. Pre-populate chart slots in reinit() via listChartSignatures so per-card skeletons appear immediately while the worker clones and groups data. Remove the full-page LoadingSkeleton from the init phase — header and metadata are visible from the moment data loads; only chart slots pulse. Fix toStatSignature to handle absent per/unit fields without generating misleading signature strings.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
shallowRef+markRawfor datasets, nativepostMessageclone replaces JSON round-trip (~7300ms → ~700ms for 500k rows)groupName: Ref<string>→activeGroupId: Ref<number>so the params watcher doesn't fire on group-name propagation?g=URL param to defer until worker groups are ready (Vue 3.5{ once: true }watcher)Key changes
useChartPipeline.ts— dual epoch (dataEpoch/jobEpoch),reinit/setArrangement/recomputepaths, exposegroupNamestransform.worker.ts— owns raw rows;init/setArrangementreply withgroupNames;computereads from internal cacheuseDataPoint.ts— drop 500k main-thread grouping computeds; source groups from worker viasetGroupNamestransform.ts— add pureprojectAndGrouphelperuseUrlRouter.ts— one-shot watcher for?g=paramAxisSwapper.vue— set arrangement key, no row mutationchartConfig.ts— x data zoom threshold configTest Plan
?g=1— correct group selected after worker readygo test ./...passes