Skip to content

Provide group / ungroup with nested element tree#263

Merged
hackerwins merged 38 commits into
mainfrom
slides-group-design
May 18, 2026
Merged

Provide group / ungroup with nested element tree#263
hackerwins merged 38 commits into
mainfrom
slides-group-design

Conversation

@hackerwins

@hackerwins hackerwins commented May 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Adds Google-Slides-style group / ungroup to the slides editor (Mod+Alt+G / Mod+Shift+Alt+G, drill-in via double-click, Esc to pop), including recursive renderer / hit-test / snap / interactions and a tight ancestor-transform pipeline shared across paint, edit, and PPTX import.
  • Closes the Group / ungroup entry previously deferred to v2 in slides.md; design captured in slides-group.md, implementation tracked in docs/tasks/archive/2026/05/20260517-slides-group-{todo,lessons}.md (16 tasks, TDD).
  • Brings full parity to YorkieSlidesStore over nested Yorkie.Array<Element>, including connector partition and data.refSize so resizing a group visibly scales its children (the OOXML <a:chExt> / <a:ext> analog).

Highlights

  • Model + math (packages/slides/src/model/): new GroupElement variant + model/group.ts with groupToTransform, applyGroupTransform, normalizeToGroupLocal, applyInverseMatrix, applyInversePoint, composeGroupMatrix, composeAncestorTransform, findElementPath, isGroupDescendantOf, flattenElements. Property tests cover round-trip invariants.
  • Store (MemSlidesStore + YorkieSlidesStore): group() / ungroup() with same-parent / placeholder / empty-group / cycle invariants; element mutations walk findElementPath; addElement(parentGroupId?); empty-group auto-removal; connector cross-group exclusion with onToast; data.refSize set at creation.
  • Renderer + hit-test: drawElement recurses into group children; applies ctx.scale(frame.w/refSize.w, frame.h/refSize.h). hitTestSlide returns { elementId, ancestorPath }.
  • Selection state machine: scope: string[] + click / doubleClick / escape per Google Slides drill-in rules.
  • Scope-aware interactions: drag / resize / rotate / nudge + align / distribute / rotateBy use frame-space.ts to lift to world coords and convert back at commit.
  • Shortcuts + menus: Mod+Alt+G / Mod+Shift+Alt+G / Esc catalog + keymap; context menu Group / Ungroup; toolbar Arrange dropdown entries. Right-click goes through Selection.click so the outermost group is picked and Ungroup is enabled.
  • PPTX import (import/pptx/shape.ts): <p:grpSp> preserved as GroupElement; group.frame is the rotation-aware AABB of children's world frames so handles tightly bound the visible content even when <a:chOff>/<a:chExt> differ from <a:off>/<a:ext>.
  • Multi-user: equivalence tests + integration tests for concurrent group / ungroup convergence (gated by YORKIE_RPC_ADDR).

Out of scope / follow-ups

Documented in slides-group.md Known Limitations:

  • PDF export for slides isn't implemented yet (separate subsystem). When added, copying the renderer's recursive transform pattern is enough.
  • selectAt in view/editor/interactions/select.ts is dead code after the click-handler rewire; safe delete in a follow-up.
  • Cross-group connectors (one endpoint outside the selection) are excluded with a toast in v1; symmetric handling is v1.1.

Test plan

  • pnpm verify:fast — green (slides 1162, frontend 398, docs 791, sheets 191, backend 143).
  • pnpm slides build — green (frontend reads @wafflebase/slides from dist).
  • PPTX <p:grpSp> import preserves groups with bbox-equivalence to old flatten path (property test, depth 2, ≤ 0.5 px).
  • Selection drill-in state machine — 20 unit tests covering every spec table row.
  • CRDT convergence — yorkie-slides-group-concurrent.integration.ts (4 scenarios; gated on YORKIE_RPC_ADDR).
  • Manual smoke after merge — pnpm dev, create a group, resize/rotate/drill-in/ungroup; PPTX import on a real deck; verify handles match rendered shapes on rotated/scaled groups.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Added Group/Ungroup functionality for slide elements with keyboard shortcuts (Cmd+Alt+G to group, Cmd+Shift+Alt+G to ungroup).
    • Implemented Google Slides-style drill-in selection for grouped elements (click to select group, double-click to enter).
    • PPTX imports now preserve group structures instead of flattening them.
    • Added toast notifications for grouping edge cases.

Review Change Stack

@coderabbitai

coderabbitai Bot commented May 18, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

Implements Google Slides-style group/ungroup for the slides editor: adds recursive GroupElement type with group-local child coordinates, store APIs for grouping/ungrouping elements with connector partitioning, PPTX group preservation, recursive rendering and hit-testing with drill-in selection scope, scope-aware drag/rotate/resize/snap interactions, Yorkie-backed nested mutations, and comprehensive test coverage.

Changes

Group/Ungroup Feature Implementation

Layer / File(s) Summary
Design documentation and task tracking
docs/design/README.md, docs/design/slides/slides-group.md, docs/design/slides/slides.md, docs/tasks/*.md, docs/tasks/archive/...
Design spec for group/ungroup scope, drill-in selection, PPTX preservation, rendering/hit-test/export changes; task archive entries documenting implementation plan and lessons learned.
Group element types and affine transform utilities
packages/slides/src/model/element.ts, packages/slides/src/model/group.ts
GroupElement with data.children: Element[] and optional refSize for group-local coordinate resizing; transform math: compose, invert, apply, find element paths, check descendant cycles, flatten nested elements.
Store interface for grouping/ungrouping
packages/slides/src/store/store.ts
addElement accepts optional parentGroupId; new group(elementIds) wraps elements into a group and returns {groupId, excludedConnectorIds}; ungroup(groupId) dissolves a group and returns promoted child ids.
MemSlidesStore group/ungroup and tree-aware mutations
packages/slides/src/store/memory.ts
Core implementation: recursive element path lookup for all mutations; group() with rotation-aware AABB, child frame normalization, connector partitioning by endpoint containment; ungroup() with group transform baking and empty ancestor pruning; tree helpers for parent-array resolution.
PPTX shape tree parsing with group preservation
packages/slides/src/import/pptx/shape.ts
parseGrpSp() preserves <p:grpSp> as GroupElement, computes group AABB over child world frames, converts children to group-local coordinates, handles free connector endpoints, falls back to <a:off>/<a:ext> for empty groups.
Slides package public API exports
packages/slides/src/index.ts
Re-export GroupElement, GroupTransform, group transform utilities, HitResult, hitTestSlide, collectSnapCandidates.
Canvas rendering with recursive group drawing and refSize scaling
packages/slides/src/view/canvas/element-renderer.ts, packages/slides/src/view/canvas/slide-renderer.ts
drawElement() recurses into group children under group transform; applies refSize-derived scaling for group-local↔frame-space mapping; flattenElements() DFS for connector endpoint lookups.
Hit-testing, frame-space conversions, and drill-in scope selection
packages/slides/src/view/editor/hit-test-elements.ts, packages/slides/src/view/editor/frame-space.ts, packages/slides/src/view/editor/selection.ts
hitTestSlide() recursively traverses groups, transforms points to group-local space, returns leaf hit with ancestorPath; scopeAncestorTransform() / toWorldFrame() / fromWorldFrame() handle coordinate conversion; Selection class manages drill-in scope with click/doubleClick/escape transitions.
Editor pointer, keyboard, and context menu with scope awareness
packages/slides/src/view/editor/editor.ts, packages/slides/src/view/editor/interactions/*.ts
Pointer handlers use hitTestSlide + Selection.click for drill-in; drag/rotate/resize capture world frames and live paint via paintLiveScoped; keyboard shortcuts: Cmd+Alt+G group, Cmd+Shift+Alt+G ungroup, Esc pops scope; context menu gates Group/Ungroup by canGroup/canUngroup helpers; snap candidates respect scope with ancestor transform composition.
Frontend toast wiring and toolbar UI integration
packages/frontend/src/app/slides/slides-view.tsx, packages/frontend/src/app/slides/toolbar/arrange-menu.tsx, packages/frontend/src/app/slides/toolbar/object-section.tsx
SlidesView wires onToast ref into editor; ArrangeMenu adds Group/Ungroup dropdown section with enable/disable gating; ObjectSection computes canUngroup for the single selected element via findElementPath.
YorkieSlidesStore recursive Yorkie proxy unwrapping and tree-aware mutations
packages/frontend/src/app/slides/yorkie-slides-store.ts, packages/frontend/src/types/slides-document.ts
readElement() recursively unwraps Yorkie proxies for groups and children; mutations route via path lookup; updateElementData mutates Yorkie data in-place to preserve nested arrays; group/ungroup operate on Yorkie trees; YorkieGroupElement type added.
Comprehensive test coverage
packages/slides/test/..., packages/frontend/tests/...
Unit tests for group transforms, store group/ungroup/nested mutations, PPTX import fixtures, canvas rendering, hit-testing/frame-space/selection drill-in, editor interactions (keyboard/pointer/menu), Yorkie equivalence/convergence; property-based tests for transform round-trips and frame invariance.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • wafflebase/wafflebase#243: PPTX import <p:grpSp> handling—overlaps in group geometry/transform composition at packages/slides/src/import/pptx/*.
  • wafflebase/wafflebase#239: Keyboard Escape handling for editor state transitions—overlaps in packages/slides/src/view/editor/interactions/keyboard.ts.
  • wafflebase/wafflebase#241: Connector endpoint mutation and rendering lookups—overlaps in connector handling now required to work through grouped element paths.

Poem

🐰 A tale of nesting frames so deep,
Where rabbits grouped their slides to keep,
With drill-in clicks and transforms bright,
The groups emerge—a wondrous sight!
From PPTX to canvas done,
Group/ungroup—the feature's won! 🎉

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch slides-group-design

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 11

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)
packages/frontend/src/app/slides/yorkie-slides-store.ts (1)

1414-1465: ⚠️ Potential issue | 🟠 Major | 🏗️ Heavy lift

Connector maintenance helpers ignore connectors nested inside groups.

Both detachConnectorsTargeting and recomputeDependentConnectorFrames only scan s.elements (root). Nested connectors won’t be detached/recomputed when nested targets are removed or moved.

Suggested direction
+function walkElements(elements: ProxyArray, fn: (el: ProxyArray[number]) => void): void {
+  for (const el of elements) {
+    fn(el);
+    if (el.type === 'group') {
+      walkElements((el.data as { children: ProxyArray }).children, fn);
+    }
+  }
+}

Use walkElements(...) in both helpers instead of root-only loops.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/frontend/src/app/slides/yorkie-slides-store.ts` around lines 1414 -
1465, Both helpers only iterate s.elements and thus miss connectors nested in
groups; change detachConnectorsTargeting and recomputeDependentConnectorFrames
to walk the entire element tree with walkElements(s, cb) instead of for (const
el of s.elements). Inside the walk callback, keep the existing type checks (if
el.type !== 'connector' continue), reuse lookup = this.slideElementsLookup(s),
perform the same endpoint checks and mutations (including setting c.frame =
computeConnectorFrame(plain, lookup) after changes) so nested connector elements
get detached and have their frames recomputed just like root-level ones.
packages/slides/src/view/editor/editor.ts (2)

1188-1209: ⚠️ Potential issue | 🟠 Major | 🏗️ Heavy lift

enterEditMode still assumes top-level text elements.

Line 1190 uses slide.elements.find(...), so grouped/nested text can’t enter edit mode. Also, mounting with Line 1208 element.frame uses local coordinates for nested elements, which misplaces the editor overlay.

Suggested direction
-    const element = slide.elements.find((e) => e.id === elementId);
+    const element = findElement(slide.elements, elementId);
     if (!element || element.type !== 'text') return;
+    const frameForOverlay = toWorldFrame(element.frame, this.selection.getScope(), slide);
@@
-      frame: element.frame,
+      frame: frameForOverlay,

Also verify withTextElement(slideId, elementId, ...) supports nested paths; if not, it needs the same tree-aware lookup/update strategy.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/slides/src/view/editor/editor.ts` around lines 1188 - 1209,
enterEditMode currently finds elements only at slide.elements and passes
element.frame (local coords) to mountTextBox, which breaks for grouped/nested
text; update the element lookup to a tree-aware search (e.g., a helper like
findElementInTree(slide, elementId) that traverses groups/children) and use that
to set selection and editingElementId; compute the element's
absolute/frame-in-canvas coordinates before calling mountTextBox (e.g.,
accumulate parent transforms/offsets or call an existing localToGlobal utility)
and pass that absolute frame to mountTextBox; finally, confirm or update
withTextElement to accept a nested path or use the same tree-aware updater so
commits target the correct nested text node.

513-524: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Reset drill-in scope when changing slides.

Line 518 only clears ids. With drill-in enabled, stale scope survives slide switch and can poison the next scoped operation. Clear both ids and scope during slide change.

Suggested fix
-    this.selection.clear();
+    this.selection.click(null, {});
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/slides/src/view/editor/editor.ts` around lines 513 - 524, In
setCurrentSlide, also reset the drill-in scope so stale scope does not survive
slide switches: after clearing selection ids (this.selection.clear()), clear the
drill-in scope by calling a dedicated method like this.selection.clearScope()
(or, if that method doesn't exist, explicitly set this.selection.scope =
undefined/null) so both ids and scope are reset before switching slides; update
or add the clearScope helper on the selection object if needed and invoke it in
setCurrentSlide.
🧹 Nitpick comments (2)
packages/slides/test/view/editor/interactions/keyboard.test.ts (1)

666-736: ⚡ Quick win

Add Ctrl-based coverage for Mod+Alt+G and Mod+Shift+Alt+G.

Line 670 and Line 703 currently validate only metaKey paths. Since these shortcuts are Mod-based, add ctrlKey variants to lock Windows/Linux parity.

Proposed test additions
 describe('keyboard — Cmd+Alt+G group', () => {
@@
   it('Cmd+Alt+G groups ≥2 selected elements', () => {
@@
   });
+
+  it('Ctrl+Alt+G groups ≥2 selected elements (Windows/Linux Mod path)', () => {
+    const { editor: e, store, slideId, aId, bId } = makeTwoElementFixture();
+    editor = e;
+    editor.setSelection([aId, bId]);
+    document.dispatchEvent(new KeyboardEvent('keydown', {
+      key: 'g', ctrlKey: true, altKey: true, bubbles: true,
+    }));
+    const slide = store.read().slides.find((s) => s.id === slideId)!;
+    expect(slide.elements).toHaveLength(1);
+    expect(slide.elements[0].type).toBe('group');
+  });
 });
 
 describe('keyboard — Cmd+Shift+Alt+G ungroup', () => {
@@
   it('Cmd+Shift+Alt+G ungroups a selected group and selects children', () => {
@@
   });
+
+  it('Ctrl+Shift+Alt+G ungroups a selected group (Windows/Linux Mod path)', () => {
+    const { editor: e, store, slideId, aId, bId } = makeTwoElementFixture();
+    editor = e;
+    let groupId = '';
+    store.batch(() => {
+      groupId = store.group(slideId, [aId, bId]).groupId;
+    });
+    editor.setSelection([groupId]);
+    document.dispatchEvent(new KeyboardEvent('keydown', {
+      key: 'g', ctrlKey: true, altKey: true, shiftKey: true, bubbles: true,
+    }));
+    const slide = store.read().slides.find((s) => s.id === slideId)!;
+    expect(slide.elements).toHaveLength(2);
+  });
 });

As per coding guidelines, "**/*.{test,spec}.{js,ts,jsx,tsx}: Write tests for critical business logic and edge cases".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/slides/test/view/editor/interactions/keyboard.test.ts` around lines
666 - 736, The tests for Mod+Alt+G and Mod+Shift+Alt+G only exercise metaKey
(Cmd) paths; add equivalent ctrlKey (Ctrl) variants to ensure Windows/Linux
parity by duplicating the keyboard dispatches in the four test cases ("Cmd+Alt+G
groups ≥2 selected elements", "Cmd+Alt+G is a no-op when fewer than 2 elements
are selected", "Cmd+Shift+Alt+G ungroups a selected group and selects children",
"Cmd+Shift+Alt+G is a no-op when selection is not a single group") and sending a
KeyboardEvent with ctrlKey: true (and metaKey: false) using the same
key/modifier combos (g + alt + optional shift) so the editor’s
grouping/ungrouping handlers are exercised for Ctrl-based shortcuts as well.
packages/slides/src/model/element.ts (1)

165-185: ⚡ Quick win

Encode the “no placeholder on group” invariant in the type.

GroupElement currently inherits placeholderRef? from ElementBase, even though the comment says it is invalid. This allows invalid group objects to compile outside MemSlidesStore.group().

💡 Proposed fix
 export type GroupElement = ElementBase & {
   type: 'group';
+  placeholderRef?: never;
   data: {
     children: Element[]; // frames are in group-local coords (0..refSize × 0..refSize)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/slides/src/model/element.ts` around lines 165 - 185, GroupElement
currently still inherits placeholderRef from ElementBase allowing invalid group
objects; update the GroupElement type to encode "no placeholder on group" by
removing/negating placeholderRef from the inherited shape—for example, use
Omit<ElementBase, 'placeholderRef'> when composing GroupElement or explicitly
override placeholderRef on GroupElement as never/undefined so the compiler
rejects any group with a placeholderRef; adjust the declaration around the
GroupElement type (and its use of ElementBase/Element) accordingly so
type-checking enforces the invariant.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/design/slides/slides-group.md`:
- Around line 174-180: Several fenced code blocks (the tree diagram and the code
examples for paintElement and hitTest) are missing language tags triggering
markdownlint MD040; update each ``` fence (including the root/tree block and the
code blocks containing paintElement(ctx, el, parent = identity) and
hitTest(point, elements, parentMatrix)) to include an appropriate language
identifier (e.g., ```text for the diagram and ```text or ```js for the examples)
so the blocks are properly tagged; apply the same change to the other
occurrences noted (lines around 288-299 and 308-317).

In `@packages/frontend/src/app/slides/yorkie-slides-store.ts`:
- Around line 1345-1346: In withTextElement(), don't overwrite the entire event
data (currently done via eAny.data = { blocks: clone(next ?? blocks) }); instead
merge the new blocks into the existing data object so other text metadata keys
are preserved: read the current eAny.data (or default to {}), clone the blocks
(clone(next ?? blocks)) and set data = { ...existingData, blocks: clonedBlocks }
(or mutate existingData.blocks) so only blocks are updated; reference symbols:
withTextElement, eAny, data, blocks, next, clone.
- Around line 1296-1299: ungroup() currently rebakes only the element.frame (see
bakedChildren mapping) so connector elements whose start/end have kind ===
'free' remain in group-local coordinates and jump after ungrouping; update the
rebake logic in the bakedChildren creation (or a helper used by ungroup) to also
transform connector endpoint coordinates when their start/end.kind === 'free' by
applying applyGroupTransform to those endpoint positions (use the same
plainGroup transform used for frame) so both frame and connector free endpoints
are converted from group-local to document coordinates.

In `@packages/frontend/tests/app/slides/yorkie-slides-equivalence.test.ts`:
- Around line 190-311: Add a new test in the same suite using runBoth that
creates a slide, adds two shapes and a connector with one free endpoint (use
store.addElement with type 'connector' and data.endpoints including a free
endpoint), then call store.batch(() => { groupId = store.group(slideId,
[shapeAId, shapeBId, connectorId]).groupId; }); then store.batch(() => {
store.ungroup(slideId, groupId); }); finally assert deep equality via
stripGroupIds(yo) vs stripGroupIds(mem) and verify the connector endpoint
coordinates are normalized/rewritten consistently (inspect yo.slides[0].elements
to find the connector element and check its endpoints), mirroring the pattern
used in existing tests (runBoth, store.group, store.ungroup, stripGroupIds).

In `@packages/frontend/tests/app/slides/yorkie-slides-store.test.ts`:
- Line 281: The assertion assert.equal(groupId, groupId) is tautological and
should be replaced with a meaningful check: either remove it or assert a real
postcondition about groupId (for example assert.ok(typeof groupId === 'string')
or assert.equal(groupId, expectedGroupId) or assert.match/groupId against an
expected pattern) that reflects the behavior under test; locate the test in
packages/frontend/tests/app/slides/yorkie-slides-store.test.ts and update the
assertion referencing the variable groupId (or replace it with an assertion
about store state or side-effects related to groupId) so the test actually
verifies expected behavior.

In `@packages/slides/src/model/group.ts`:
- Around line 118-127: The inverse helpers (e.g., applyInverseMatrix) currently
divide by det unguarded causing Infinity/NaN for singular transforms; fix by
checking the determinant (const det = t.a*t.d - t.b*t.c) against a small epsilon
(e.g., Math.abs(det) < 1e-12) and short-circuit when singular—either return the
original frame unchanged or return a safe no-op result—otherwise proceed to
compute inv and call applyMatrix; apply the same guard to the other inverse
helper at the 176-193 block so both functions handle zero/near-zero determinant
cases.

In `@packages/slides/src/store/memory.ts`:
- Around line 229-260: The connector recomputation and lookup code in addElement
(and the other spots noted) only builds a Map from the slide root array
(slide.elements) and thus misses connectors and targets that live inside groups;
change these paths to first produce a flattened element list/map for the whole
slide tree (recursively gather slide.elements and all group children) and then
reuse that flattened lookup when calling computeConnectorFrame in addElement,
when recomputing endpoints in updateConnectorEndpoint, when refreshing dependent
connectors in updateElementFrame, and when detaching connectors in
detachConnectorsTargeting so connector frame computation and target lookups are
tree-aware rather than root-array-only.
- Around line 738-751: The ungroup baking only transforms child.frame but misses
connector geometry stored in group-local coords, so update the bakedChildren
creation (the map that builds bakedChildren using applyGroupTransform) to also
detect connector elements and transform their endpoint/geometry fields into
parent space: for each child (inside the group.data.children map) after
computing the new frame via applyGroupTransform(child.frame, group as
GroupElement), if the child is a connector with free endpoints (the
connector/line element shape that stores endpoints/points in its props), run the
same group-to-parent transform on those endpoint/point coordinates and replace
them in the cloned child before returning it; leave non-free or non-connector
children unchanged. Ensure the transformed connector geometry is included in the
object you splice into parentArray so connectors retain correct shape after
parentArray.splice(groupIndex, 1, ...bakedChildren).

In `@packages/slides/src/store/store.ts`:
- Around line 76-78: Update the stale doc comment on the group() API to reflect
the current behavior: state that group() returns an object { groupId,
excludedConnectorIds } where excludedConnectorIds may contain connector IDs
(i.e., is not always [] anymore) because connector exclusion has been
implemented; edit the comment above the group() function in store.ts to remove
the “always [] until Task 11” text and briefly describe when
excludedConnectorIds will be populated so callers understand it can be
non-empty.

In `@packages/slides/src/view/editor/frame-space.ts`:
- Around line 52-70: scopeAncestorTransform currently only checks the
innermostId; instead validate the entire scope chain by ensuring the found path
for innermostId actually ends with the exact sequence of ids in scope and that
each corresponding element is a group. After computing path =
findElementPath(slide.elements, innermostId), verify path.map(e =>
e.id).slice(-scope.length) equals scope (throw an error if it doesn't), and also
check each element in the matched suffix is of type 'group' before returning
fullAncestors/composeAncestorTransform; reference symbols:
scopeAncestorTransform, scope, innermostId, findElementPath, path,
fullAncestors, innermostEl.

In `@packages/slides/test/import/pptx/group-preserving.test.ts`:
- Around line 51-69: The function collectLeafWorldFrames applies ancestor
transforms in outer→inner order causing incorrect world frames for nested
scaled/rotated groups; change the transform application to inner→outer by
iterating ancestors in reverse (e.g., for each ancestor from the last to the
first) when calling applyGroupTransform on the element's local frame so the
innermost group transform is applied first; keep the recursion that builds
ancestors ([...ancestors, el]) and ensure the Frame results are collected after
applying the reversed-order transforms.

---

Outside diff comments:
In `@packages/frontend/src/app/slides/yorkie-slides-store.ts`:
- Around line 1414-1465: Both helpers only iterate s.elements and thus miss
connectors nested in groups; change detachConnectorsTargeting and
recomputeDependentConnectorFrames to walk the entire element tree with
walkElements(s, cb) instead of for (const el of s.elements). Inside the walk
callback, keep the existing type checks (if el.type !== 'connector' continue),
reuse lookup = this.slideElementsLookup(s), perform the same endpoint checks and
mutations (including setting c.frame = computeConnectorFrame(plain, lookup)
after changes) so nested connector elements get detached and have their frames
recomputed just like root-level ones.

In `@packages/slides/src/view/editor/editor.ts`:
- Around line 1188-1209: enterEditMode currently finds elements only at
slide.elements and passes element.frame (local coords) to mountTextBox, which
breaks for grouped/nested text; update the element lookup to a tree-aware search
(e.g., a helper like findElementInTree(slide, elementId) that traverses
groups/children) and use that to set selection and editingElementId; compute the
element's absolute/frame-in-canvas coordinates before calling mountTextBox
(e.g., accumulate parent transforms/offsets or call an existing localToGlobal
utility) and pass that absolute frame to mountTextBox; finally, confirm or
update withTextElement to accept a nested path or use the same tree-aware
updater so commits target the correct nested text node.
- Around line 513-524: In setCurrentSlide, also reset the drill-in scope so
stale scope does not survive slide switches: after clearing selection ids
(this.selection.clear()), clear the drill-in scope by calling a dedicated method
like this.selection.clearScope() (or, if that method doesn't exist, explicitly
set this.selection.scope = undefined/null) so both ids and scope are reset
before switching slides; update or add the clearScope helper on the selection
object if needed and invoke it in setCurrentSlide.

---

Nitpick comments:
In `@packages/slides/src/model/element.ts`:
- Around line 165-185: GroupElement currently still inherits placeholderRef from
ElementBase allowing invalid group objects; update the GroupElement type to
encode "no placeholder on group" by removing/negating placeholderRef from the
inherited shape—for example, use Omit<ElementBase, 'placeholderRef'> when
composing GroupElement or explicitly override placeholderRef on GroupElement as
never/undefined so the compiler rejects any group with a placeholderRef; adjust
the declaration around the GroupElement type (and its use of
ElementBase/Element) accordingly so type-checking enforces the invariant.

In `@packages/slides/test/view/editor/interactions/keyboard.test.ts`:
- Around line 666-736: The tests for Mod+Alt+G and Mod+Shift+Alt+G only exercise
metaKey (Cmd) paths; add equivalent ctrlKey (Ctrl) variants to ensure
Windows/Linux parity by duplicating the keyboard dispatches in the four test
cases ("Cmd+Alt+G groups ≥2 selected elements", "Cmd+Alt+G is a no-op when fewer
than 2 elements are selected", "Cmd+Shift+Alt+G ungroups a selected group and
selects children", "Cmd+Shift+Alt+G is a no-op when selection is not a single
group") and sending a KeyboardEvent with ctrlKey: true (and metaKey: false)
using the same key/modifier combos (g + alt + optional shift) so the editor’s
grouping/ungrouping handlers are exercised for Ctrl-based shortcuts as well.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f2ff0204-8c9a-4aa2-8ae5-d3080dba278d

📥 Commits

Reviewing files that changed from the base of the PR and between 3df315b and 3384fd6.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (47)
  • docs/design/README.md
  • docs/design/slides/slides-group.md
  • docs/design/slides/slides.md
  • docs/tasks/README.md
  • docs/tasks/archive/2026/05/20260517-pptx-image-background-lessons.md
  • docs/tasks/archive/2026/05/20260517-pptx-image-background-todo.md
  • docs/tasks/archive/2026/05/20260517-slides-group-lessons.md
  • docs/tasks/archive/2026/05/20260517-slides-group-todo.md
  • docs/tasks/archive/2026/05/20260517-slides-textbox-click-scale-lessons.md
  • docs/tasks/archive/2026/05/20260517-slides-textbox-click-scale-todo.md
  • docs/tasks/archive/README.md
  • packages/frontend/src/app/slides/slides-view.tsx
  • packages/frontend/src/app/slides/toolbar/arrange-menu.tsx
  • packages/frontend/src/app/slides/toolbar/object-section.tsx
  • packages/frontend/src/app/slides/yorkie-slides-store.ts
  • packages/frontend/src/types/slides-document.ts
  • packages/frontend/tests/app/slides/yorkie-slides-equivalence.test.ts
  • packages/frontend/tests/app/slides/yorkie-slides-group-concurrent.integration.ts
  • packages/frontend/tests/app/slides/yorkie-slides-store.test.ts
  • packages/slides/package.json
  • packages/slides/src/import/pptx/shape.ts
  • packages/slides/src/index.ts
  • packages/slides/src/model/element.ts
  • packages/slides/src/model/group.ts
  • packages/slides/src/store/memory.ts
  • packages/slides/src/store/store.ts
  • packages/slides/src/view/canvas/element-renderer.ts
  • packages/slides/src/view/canvas/slide-renderer.ts
  • packages/slides/src/view/editor/editor.ts
  • packages/slides/src/view/editor/frame-space.ts
  • packages/slides/src/view/editor/hit-test-elements.ts
  • packages/slides/src/view/editor/interactions/keyboard.ts
  • packages/slides/src/view/editor/interactions/select.ts
  • packages/slides/src/view/editor/selection.ts
  • packages/slides/src/view/editor/shortcuts-catalog.ts
  • packages/slides/src/view/editor/snap-candidates.ts
  • packages/slides/test/import/pptx/group-preserving.test.ts
  • packages/slides/test/model/group.test.ts
  • packages/slides/test/store/group-mutations.test.ts
  • packages/slides/test/store/group-nested-mutations.test.ts
  • packages/slides/test/view/canvas/group-render.test.ts
  • packages/slides/test/view/editor/editor.test.ts
  • packages/slides/test/view/editor/frame-space.test.ts
  • packages/slides/test/view/editor/hit-test-elements.test.ts
  • packages/slides/test/view/editor/interactions/keyboard.test.ts
  • packages/slides/test/view/editor/selection-drillin.test.ts
  • packages/slides/test/view/editor/snap-candidates.test.ts

Comment thread docs/design/slides/slides-group.md Outdated
Comment thread packages/frontend/src/app/slides/yorkie-slides-store.ts Outdated
Comment thread packages/frontend/src/app/slides/yorkie-slides-store.ts Outdated
Comment thread packages/frontend/tests/app/slides/yorkie-slides-store.test.ts Outdated
Comment thread packages/slides/src/store/memory.ts
Comment thread packages/slides/src/store/memory.ts
Comment thread packages/slides/src/store/store.ts Outdated
Comment thread packages/slides/src/view/editor/frame-space.ts
Comment thread packages/slides/test/import/pptx/group-preserving.test.ts
@hackerwins
hackerwins force-pushed the slides-group-design branch from 3384fd6 to c2e3ebe Compare May 18, 2026 16:46
@hackerwins

Copy link
Copy Markdown
Collaborator Author

Addressed CodeRabbit's 11 findings across two commits:

197d3c59 — Correctness bugs (Major)

  • M1 — ungroup() leaves connector free endpoints in stale group-local coords. Both MemSlidesStore.ungroup() and YorkieSlidesStore.ungroup() now apply applyGroupTransformToPoint(p.x, p.y, groupToTransform(group)) to start/end when kind === 'free' before splicing children. Symmetric fix in both stores.
  • M2 — Singular group transform produces NaN/Infinity. applyInverseMatrix and applyInversePoint now throw on |det| < 1e-9. Both MemSlidesStore.group() / YorkieSlidesStore.group() clamp the AABB to MIN_GROUP_DIM = 1px so a singular group is never created in the first place. 4 new tests in group.test.ts.
  • M3 — scopeAncestorTransform only validated the innermost scope id. It now verifies the full ancestor chain. Stale intermediate scope ids throw rather than silently miscomputing.
  • M4 — Connector bookkeeping was root-array-only. elementsLookup, detachConnectorsTargeting, updateElementFrame dependent refresh, and addElement frame recompute now use flattenElements from model/group.ts so connectors nested inside groups are tracked.
  • BonuswithTextElement no longer overwrites non-blocks data fields. Existing data is spread before merging the new blocks.

Equivalence test added: connector with free endpoint goes through group() → ungroup() and lands back at the original world coordinates.

d8f57dab — Doc + test polish (Minor)

  • m5 — Fenced code blocks in slides-group.md got text language tags (markdownlint MD040).
  • m6SlidesStore.group() JSDoc no longer says excludedConnectorIds is "always []"; describes the actual partition.
  • m7 — Tautological assert.equal(groupId, groupId) replaced with assert.ok(groupId.length > 0).
  • m8collectLeafWorldFrames test helper now applies ancestors inner→outer instead of outer→inner.

pnpm verify:fast — 439 frontend + 1162 slides + 791 docs + 191 sheets + 143 backend tests, all green.

hackerwins and others added 26 commits May 19, 2026 02:12
Adds a v1 design doc that turns the slide element array into a
recursive tree by introducing a first-class `GroupElement`. Child
frames live in group-local coordinates so a group's resize / rotate
is one frame mutation rather than fanning out to children, and the
PPTX `<p:grpSp>` transform math already in the importer is reused
inverted for runtime paint and inverse-applied for import
normalization. Selection follows Google Slides drill-in semantics
(double-click descends, Esc pops out), and PDF export shares the
recursive renderer so paint and export stay identical by
construction.

This closes the "Group / ungroup" item the original slides.md spec
deferred to v2; the matching v2 Non-Goals entry should move once the
implementation lands.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
The plan decomposes the slides-group.md design into 16 TDD-style
tasks across five phases: model and store mutations (Tasks 1-4),
recursive renderer / hit-test / snap (Tasks 5-7), drill-in selection
and shortcut / context-menu / toolbar wiring (Tasks 8-10), connector
partitioning during group() (Task 11), Yorkie adapter and multi-user
convergence tests (Tasks 12-13), PPTX import preservation and
recursive PDF export (Tasks 14-15), and a final task to update
slides.md and archive the work.

Includes the empty-todo lessons companion to be filled in during
implementation.
Foundation for the slides group/ungroup feature. Adds GroupElement
to the Element union so the data model can represent nested groups,
and provides four helpers in model/group.ts (applyGroupTransform,
normalizeToGroupLocal, findElementPath, isDescendantOf) that compose
with the existing PPTX affine-transform math without touching
import/pptx/group.ts. No behaviour change to existing elements.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
- Rename isDescendantOf → isGroupDescendantOf and narrow second
  parameter from Element to GroupElement; makes the cycle-prevention
  contract explicit: the function only walks group subtrees and is
  only meaningful when the target is itself a group.
- Fix misleading "returns true for a direct child" test which only
  exercised the self-check; replace with a genuine direct-group-child
  assertion so all three cases (self, direct child, deep descendant)
  are covered by distinct tests.
- Remove redundant inline comment inside groupToTransform that
  duplicated the JSDoc above.
- Simplify double-negative in normalizeToGroupLocal:
  -(-t.b * t.tx + t.a * t.ty) → (t.b * t.tx - t.a * t.ty).
- Add invariant comment on GroupElement explaining that placeholderRef
  is invalid on groups to prevent future mistakes.
- Update design doc findElementPath signature to match implementation
  (elements: Element[] not slide: Slide).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Implements Task 2 of the slides group/ungroup feature. The `group()`
method validates that all candidate elements exist, share the same
parent, and carry no placeholderRef, then computes the world-AABB from
rotated corners, wraps candidates in a new GroupElement (with
children in group-local coords via normalizeToGroupLocal), and inserts
the group at the front-most selected element's z-order position inside
batch() for single-step undo.

Also adds the `ungroup()` stub (throws "not implemented") and both
signatures to the SlidesStore interface so Task 3 can land without a
breaking interface change.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Move the canonical compose/ancestor helpers into model/group.ts so there
is a single source of truth for slide-group affine math.  The four
private helpers in memory.ts (composeLocalTransforms,
computeAncestorTransform, applyTransformToFrame,
applyInverseTransformToFrame) duplicated logic already in pptx/group.ts
and model/group.ts; replacing them with thin wrappers over the public
API removes ~60 lines and makes the invariants visible at a glance.

The dead Invariant-5 loop (a pair of nested for-loops whose body was
only a comment) is deleted; a single comment explains that cycle
prevention is structurally guaranteed by the shared-parent invariant, so
no runtime check is needed.  The O(n²) indexOf inside .map() is replaced
with an index-based map, and the missing-parentId path in
resolveAncestorTransform now throws instead of silently returning
identity so future refactors cannot hide path-lookup bugs.

A new test locks in the rotation-aware AABB path: a 100×40 shape rotated
90° grouped with an axis-aligned 50×50 shape must produce a frame of
{x:30, y:-30, w:120, h:180, rotation:0}.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
`applyInverseFrameWithTransform` in memory.ts duplicated the
determinant/cofactor arithmetic already present in
`normalizeToGroupLocal`. Extract a public `applyInverseMatrix` helper
in model/group.ts that takes a raw GroupTransform, then refactor
`normalizeToGroupLocal` to delegate to it. Remove the private copy
from memory.ts entirely — callers now import the single implementation
via the model layer.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Children's world frames must survive a group→ungroup round-trip.
`applyGroupTransform(child.frame, group)` bakes the group's affine
transform (translation + rotation) into each child's frame so the
child lands in the group's parent coordinate space — one level only.
Nested GroupElement children have their own `frame` transformed the
same way; their grandchildren stay untouched in their own group-local
space.

Adds five deterministic tests (z-order, rotation composition, error
paths, undo step) and one fast-check property test (50 runs) that
asserts world frames are preserved within floating-point tolerance
across group→ungroup.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Drops the leftover "stub — implemented in Task 3" note now that the
real ungroup is live, and documents the one-level bake invariant so
callers know children land in the immediate parent's coord space (not
absolute slide-root space) when the group itself is nested.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
All MemSlidesStore element mutations (updateElementFrame, updateElementData,
reorderElement, removeElement, removeElements, withTextElement, the three
connector updaters) now use findElementPath so they locate and mutate the
target element at any nesting depth, not just at the slide root.

addElement gains an optional parentGroupId parameter: when provided the new
element is appended to that group's children instead of the slide root.

After any removeElement / removeElements call, pruneEmptyAncestorGroups walks
back up the ancestor path and removes any group that became empty, recursing
until it hits a non-empty group or the slide root. This keeps the tree free
of empty ghost groups without requiring callers to handle cleanup.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Close inconsistency where connectors added into a group skipped the
defensive frame recompute that slide-root connectors received; move the
recompute after both branches so it applies unconditionally.

Mark two flat-walk sites in the Yorkie store (removeElement, reorderElement)
and the keyboard handler (z-order arrow keys) with TODO(group) comments so
they are not forgotten when UI consumers become group-aware in P2/P3.

Replace the `!` non-null assertion in removeElementsByPaths with an
explicit guard and a descriptive throw, so future refactors don't get
caught out by a silent assumption.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Groups become a paint-time recursion: drawElement saves the group's frame
transform (translate + optional rotate/flip) and then iterates its children,
calling drawElement recursively for each. Per-type painters (shape, text,
image) are unchanged — they still receive element-local coordinates and are
unaware they're inside a group. Arbitrary nesting depth is handled naturally
by the recursion.

elementsLookup is now built from flattenElements() instead of the flat
slide.elements array so connector endpoints targeting shapes nested inside
groups resolve correctly. flattenElements() (new export in model/group.ts
and re-exported from index.ts) performs a DFS walk of the element tree and
returns every element at every depth.

Connectors inside groups are left as a TODO per the v1 invariant: group()
never includes connectors as children (Task 11), so the existing early-return
path for connectors is safe. A comment documents the limitation.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Task 8's drill-in selection state machine needs to know the full chain
of ancestor groups containing the clicked element, not just the leaf id.
Introducing hitTestSlide() walks the element tree DFS front-to-back,
transforms each world-space test point into each group's local coordinate
space via the inverse group matrix, and recurses into children.

The existing topmostUnderPoint() in select.ts is reduced to a one-liner
that delegates to hitTestSlide(), preserving all current callers' semantics
(they only consume the leaf-most elementId).

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
The snap engine previously called `boundingBox` nowhere for snap
candidates — the editor's inline collection passed raw `e.frame`
values to `snapDelta`, so rotated shapes and groups snapped against
their un-rotated frame edges rather than the visible rotated bbox.

This commit introduces `collectSnapCandidates` in a new module
`snap-candidates.ts`. At slide-root scope each element (including
groups as a single entry) contributes its rotated AABB; children of
groups are not exposed at root scope. For non-empty `scope` (the
drill-in variant Task 9 will exercise), direct children of the
innermost group are collected with frames composed through the full
ancestor-group transform chain, so they arrive in world coordinates
and still contribute their rotated AABB.

The editor's `startDrag` call site is refactored to call
`collectSnapCandidates(startSlide, [], selectedIds)` instead of the
previous inline `.filter/.map`. `collectSnapCandidates` is also
re-exported from the public `packages/slides/src/index.ts` for
consistency with other editor helpers.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Replace the per-ancestor applyGroupTransform loop in
collectScopedCandidates with a single composeAncestorTransform call
followed by one applyMatrix per child. This is one matrix multiply per
scope level instead of one full apply per ancestor, and makes the
intent clearer. Add a two-level-deep test so Task 9 has a regression
guard against composed-transform regressions.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Google Slides lets users double-click into a group to select children
without ungrouping. This commit adds that hierarchical selection state:

- `Selection.scope` tracks the ancestor-group chain the user has drilled
  into (outer → inner); empty means slide root.
- `click()`, `doubleClick()`, and `escape()` implement the state
  transitions: single click picks outermost ancestor or same-scope
  sibling; double-click drills one level deeper; Escape pops the scope
  stack. Shift+click toggles within the current scope.
- `pickAtScope()` centralises the "outermost vs next-level-down"
  decision, making future callers (drag, resize, rotate) share the same
  logic rather than re-implementing it.
- `applyInversePoint` is moved from a private inline in
  `hit-test-elements.ts` to `model/group.ts` (re-exported from
  `index.ts`) as a Task 6 code-review follow-up — the math is already
  present in `applyInverseMatrix` and belongs next to the rest of the
  group transform utilities.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Drilled-in elements (scope != []) store their frames in group-local
coordinates. Mouse events arrive in world (slide-root) coords. Without
conversion, dragging a grouped child would write a world-space delta
into a local-space frame, visually snapping the element out of position.

Centralise the conversion in `frame-space.ts` (scopeAncestorTransform,
toWorldFrame, fromWorldFrame). These wrap composeAncestorTransform and
applyInverseMatrix from model/group.ts so no matrix math is duplicated.

editor.ts: startDrag, startResize, startRotate all now
  - find elements via the full tree (findElement helper, not flat
    slide.elements.find) so nested elements are reachable,
  - capture world frames at drag start,
  - track world frames throughout the interaction,
  - convert world → local via fromWorldFrame before committing.

paintLiveScoped replaces paintLive: builds a synthetic slide with LOCAL
frames for the canvas renderer (which traverses group hierarchies) and
passes WORLD-frame pseudo-elements to renderOverlay so selection handles
appear at visually correct positions.

keyboard.ts nudge: uses findElementPath + toWorldFrame/fromWorldFrame so
arrow-key moves inside drilled-in groups stay in group-local space.

Phase A + B landed. Phase C (overlay bbox for scoped selection at rest)
not yet handled — the repaintOverlay path still reads stored local frames
for scoped elements, so handles may be offset when not in a live drag.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Task 9 Phase C: repaintOverlay used a flat `slide.elements.filter()`
which only scans slide-root elements. When the user drills into a group
and selects a child, the child id is not in the top-level array, so the
filter returned nothing and no handles were rendered until the user
started dragging (where paintLiveScoped already did the correct
recursive lookup + world-frame conversion).

Fix: replace the flat filter with a recursive `findElement()` lookup for
each selected id, then convert the stored (group-local) frame to world
coords via `toWorldFrame(el.frame, scope, slide)` — the same pattern
paintLiveScoped uses during live drags. For scope=[] the conversion is
the identity, preserving existing behavior for slide-root elements.

Two new tests in editor.test.ts verify that:
- The nw handle lands at the child's WORLD position, not its group-local
  position, after drilling in and selecting a child at rest.
- A child inside a rotated group still gets a CSS-rotated outline,
  confirming the world frame is computed (not the raw stored frame).

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Remove the flat-DFS `topmostUnderPoint` duplicate from editor.ts and
replace all three call sites with `hitTestSlide` from hit-test-elements.ts,
which descends into group children using group-local coordinate transforms
(the Task 6 implementation). This makes right-click, single-click, and
double-click resolve to the correct nested element instead of always
stopping at slide-root.

Route single-click through `Selection.click(hitResult, mods)` so the
drill-in state machine is honoured: clicking inside a group at scope=[]
selects the outermost group rather than the leaf child (Google Slides
behaviour). Route double-click through `Selection.doubleClick(hitResult)`
so the scope descends one level per double-click; if the newly-scoped
element is a text box, text edit mode is entered as before.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Adds the keyboard, context-menu, and toolbar entry-points for
group/ungroup so users have discoverable ways to invoke the store
operations landed in Tasks 3–9. Keyboard shortcuts (Cmd+Alt+G /
Cmd+Shift+Alt+G) keep fingers on the keyboard. Esc now pops one
drill-in scope level before clearing selection, matching Google
Slides. The Arrange dropdown exposes both actions with disabled-
state predicates that mirror the context-menu guards.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
v1 design § 7 rule: a connector that crosses the group boundary (one
endpoint references an element outside the selection) cannot follow the
group — doing so would leave its external endpoint referencing an element
that is no longer a sibling. These connectors must stay at the parent.

MemSlidesStore.group() now partitions candidates into "internal" (both
endpoints in the candidate set or free) and "excluded" cross-boundary
connectors before computing the AABB and normalizing frames. Free
endpoints on internal connectors are converted from parent-local space
to the new group-local space using the ancestor + group transforms.
If exclusion leaves fewer than 2 candidates the call throws.

The editor surfaces the count via an onToast callback so the frontend
(sonner toast.info) can inform the user without the slides package
depending on any UI library.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
MemSlidesStore and Yorkie-backed store must produce identical results
when grouping and ungrouping elements. Without group/ungroup the Yorkie
store fails the SlidesStore interface contract. Nested mutations
(addElement, removeElement, updateElementFrame, updateElementData,
reorderElement, withTextElement, updateConnectorEndpoint/Arrowheads/Stroke)
now walk the Yorkie proxy tree via yorkieFindElementPath so they work on
both slide-root and group-nested elements. Empty groups self-flatten the
same way as MemSlidesStore via pruneEmptyYorkieAncestorGroups. Cross-group
connectors are excluded with the same partition logic and the same
excludedConnectorIds return value. The applyGroupTransformMatrix and
applyGroupTransformToPoint helpers are exported from @wafflebase/slides
so the frontend store can compose ancestor transforms without deep imports.
YorkieGroupElement is added to the type union in slides-document.ts.
read() recurses into group children via readElement() so groups survive
round-trips through the CRDT.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Each group/ungroup mutation must converge under concurrent edits.
Without these tests a silent state divergence between MemSlidesStore
and Yorkie peers could go undetected — the four scenarios (overlapping
group(), ungroup vs drag, concurrent insert, reorder vs delete) cover
the realistic conflict patterns that arise during collaborative editing.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Previously, every <p:grpSp> was flattened: child frames were composed
with a cumulative matrix and emitted directly into slide.elements, losing
all group structure.

Now each <p:grpSp> produces a GroupElement whose frame is the group's
own <a:xfrm> in the parent's coordinate space. Children are stored in
group-local (0..w × 0..h) coords by: (1) composing world transforms
through composeGroupTransform so each child gets a world frame, then
(2) inverting back via applyInverseMatrix / applyInversePoint so frames
and connector free-endpoints land in the enclosing group's local space.

Nested groups recurse naturally — inner groups repeat the same pattern
relative to their own parent group. The bbox-equivalence invariant is
verified by collectLeafWorldFrames tests: leaf world frames computed by
chaining applyGroupTransform up the ancestor path match the values the
old flat-import path produced, within sub-pixel tolerance.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
slides.md removes the group/ungroup entry from "Deferred to v2" and
adds it to a new "Shipped after v1" subsection with a forward link to
slides-group.md; the same entry in "Future parity → Tracked for v2"
is replaced by a "Shipped (post-v1)" bullet.

slides-group.md gains a "Known Limitations / Follow-ups" section
documenting: PDF export not yet implemented (recursive emitter pattern
ready to copy); context-menu Ungroup visibility bug (fix is to route
right-click selection through SelectionController.click); dead
selectAt helper to remove in a follow-up.

Lessons file captures: canonical location for inverse-matrix math,
the applyGroupTransform name collision resolved as
applyGroupTransformMatrix, connector free-endpoint transform gotcha,
PPTX forward/inverse direction convention, and the selectAt dead-code
note.

Task archived via pnpm tasks:archive; tasks index refreshed.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
The previous approach set group.frame to the group's <a:off>/<a:ext>
and inverted via groupToTransform() to recover group-local child coords.
This broke in two ways:

1. When <a:chOff> != <a:off>, the inverse only undid <a:off>+rotation
   but not the chOff translation, so children's world frames started
   outside the handle parallelogram (visible content rendering left of
   the selection handle).

2. <a:ext> is the container bbox, not the visual bbox — it can include
   dead padding around the actual content.

Fix: after recursing children into world space, compute the
rotation-aware AABB via combinedBoundingBox (already used by
MemSlidesStore.group() and YorkieSlidesStore.group()). Set
group.frame to that AABB (rotation=0, tightly bounds visible
content). Convert children to group-local by subtracting the AABB
origin — no matrix inverse needed.

bbox-equivalence invariant (leaf world frames = flat-import world
frames) continues to hold because children's world frames are
unchanged; only the group frame representation differs.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
hackerwins and others added 12 commits May 19, 2026 02:12
Previously groupToTransform produced only a translate+rotation matrix
(scale=1), so dragging a group's corner handle grew the selection box
while children stayed visually fixed. Design doc §1 promises that
children scale proportionally with the group.

Introduce GroupElement.data.refSize (optional, backward-compatible)
mirroring OOXML <a:chExt>/<a:ext> semantics: children are stored in
(0..refSize.w × 0..refSize.h) space, and groupToTransform now emits
a scale(w/refSize.w, h/refSize.h) component so any change to frame
dimensions is reflected in children's world frames automatically.

• model/element.ts – add optional refSize field to GroupElement
• model/group.ts – groupToTransform includes scale; fallback to
  scale=1 when refSize is absent for backward compat
• store/memory.ts – MemSlidesStore.group() sets refSize at creation
• yorkie-slides-store.ts – YorkieSlidesStore.group() sets refSize
• import/pptx/shape.ts – parseGrpSp sets refSize from AABB at import
• types/slides-document.ts – YorkieGroupElement.data.refSize optional
• test/model/group.test.ts – new suites verify scale=1 fallback,
  proportional scaling, rotation-only identity, and resize semantics

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Groups created before the refSize field was introduced have
undefined refSize, which makes the proportional-child-scaling
logic in updateElementFrame a no-op (it skips scaling when
refSize is absent). By capturing the pre-resize dimensions as
the reference inside the startResize onUp batch, the very first
drag on such a group permanently migrates it — subsequent resizes
then scale children correctly without any manual data repair.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
The group-to-transform scale wasn't reaching the canvas: drawElement
used direct ctx.translate/rotate calls that didn't account for the
group's refSize, so children kept their literal frame.w/h positions
instead of scaling proportionally when the group was resized.

Fix: derive scaleX/scaleY from frame.w/refSize.w (and h), emit
ctx.scale(scaleX, scaleY) in the centre-relative path, and use
-refW/2,-refH/2 for the final back-translate. Legacy groups without
refSize default to scale = 1 (no behaviour change).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Re-assigning data as a whole breaks for groups: data.children is a
nested Yorkie.Array proxy, and copying it via object spread exposes
the proxy methods to the next `set`, which Yorkie rejects with
"Unsupported type of value: function". This first showed up when the
resize commit path tried to set refSize on a group via
updateElementData.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Writing refSize at mouseup meant the live preview during the first
resize of a pre-refSize group still saw refSize === undefined, so the
group's children stayed static while the selection box grew —
exactly the "group box grows, children don't" report. Capturing the
migration at mousedown means paintLiveScoped sees refSize during the
drag too, so children scale in real time.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
YorkieSlidesStore.readElement constructed group `data` as
`{ children }` only, dropping any sibling `refSize` field. Every
consumer read through this path, so the renderer never saw refSize
even after migrate-on-resize-start wrote it to Yorkie successfully —
the next read came back undefined and scale fell to 1, leaving
children static while the selection box grew.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
onContextMenu used to set selection to the raw hitTestSlide leaf id,
bypassing the same drill-in state machine that single-click goes
through. Two consequences: (1) right-clicking inside a group landed
selection on the leaf child, whose frame is in group-local coords,
so the overlay rendered handles in the wrong screen position; and
(2) the Ungroup context-menu item was permanently disabled because
the selection never resolved to a GroupElement.

Route the right-click through `selection.click(hitResult, {})` —
matches onPointerDown's path — so the outermost group is picked at
scope=[] and the Ungroup predicate sees the group selection.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
collectSelectedFrames only iterated slide.elements (the top-level
array), so toolbar align / distribute and rotateBy silently no-op'd
on selections inside a drill-in group — the design doc said they
should work on the scope's siblings. Walk via findElementPath and
lift each selected element's frame to world space via toWorldFrame
so the existing align / distribute math (which assumes one consistent
coordinate system) keeps working unchanged. applyFrameUpdates
converts each world frame back through fromWorldFrame before
committing, so the store stores parent-local coordinates.

Also drop the stale "Context menu Ungroup visibility" entry from the
known-limitations section — it was already fixed in the
right-click-through-drill-in commit (18f21eb).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
After rebasing onto main (which migrated input handling to Pointer
Events), the slides-group test still dispatched MouseEvent('mousedown')
and stopped firing — the editor only listens to pointerdown now. Also
set the design doc's target-version to 0.4.2 to reflect the current
release line.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
M1: ungroup() now bakes connector free endpoints from group-local to
parent space (both MemSlidesStore and YorkieSlidesStore). Previously
only the cached `frame` was transformed; `start`/`end` x/y stayed in
group-local coordinates, making connector geometry wrong after ungroup.

M2: applyInverseMatrix/applyInversePoint now throw on a singular
(|det| < 1e-9) group transform instead of silently returning
Infinity/NaN. MemSlidesStore.group() and YorkieSlidesStore.group()
clamp the AABB dimensions to at least 1px so a degenerate group can
never be created in the first place.

M3: scopeAncestorTransform validates the entire scope chain (all
ancestor ids) against the actual path returned by findElementPath. A
stale intermediate scope id from a remote deletion now throws rather
than silently mapping to a wrong ancestry chain.

M4: connector bookkeeping helpers (elementsLookup, detachConnectors
Targeting, updateElementFrame dependent refresh, addElement frame
recompute) now use flattenElements to walk the full element tree,
so connectors and their targets inside groups are handled correctly.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
m5: Add ```text language tags to three fenced code blocks in
docs/design/slides/slides-group.md (schema tree, paintElement pseudo-
code, hitTest pseudo-code) to fix markdownlint MD040 warnings.

m6: Update the stale SlidesStore.group() JSDoc that said
excludedConnectorIds is "always [] until Task 11" — Task 11 has
landed; replace with a description of the actual partition semantics.

m7: Replace the tautological assert.equal(groupId, groupId) in
yorkie-slides-store.test.ts with assert.ok(groupId.length > 0).

m8: Reverse the ancestor-application order in collectLeafWorldFrames
(group-preserving.test.ts) from outer→inner to inner→outer, matching
applyGroupTransform semantics. Pure-translation fixtures happened to
pass either way, but rotated/scaled nested groups would have silently
miscomputed world frames.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
`pnpm tasks:index` regenerates `docs/tasks/README.md` and
`docs/tasks/archive/README.md` deterministically from the directory
tree. Rebasing onto main #264 (which archived 7 more tasks) left
those two files showing stale counts; re-running the script picks
up the new archived tasks while keeping the slides-group archive
entry intact.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@hackerwins
hackerwins force-pushed the slides-group-design branch from d8f57da to 27869ba Compare May 18, 2026 17:15
@hackerwins hackerwins changed the title Slides group / ungroup with nested element tree Provide group / ungroup with nested element tree May 18, 2026
@hackerwins
hackerwins merged commit 4e9c310 into main May 18, 2026
1 check passed
@hackerwins
hackerwins deleted the slides-group-design branch May 18, 2026 17:19
@github-actions

github-actions Bot commented May 18, 2026

Copy link
Copy Markdown
Contributor

Verification: verify:self

Result: ✅ PASS in 207.7s

Lane Status Duration
sheets:build ✅ pass 12.6s
docs:build ✅ pass 12.0s
slides:build ✅ pass 13.2s
verify:fast ✅ pass 123.0s
frontend:build ✅ pass 18.6s
verify:frontend:chunks ✅ pass 0.3s
backend:build ✅ pass 4.8s
cli:build ✅ pass 2.1s
verify:entropy ✅ pass 21.2s

Verification: verify:integration

Result: ✅ PASS

@codecov

codecov Bot commented May 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

hackerwins added a commit that referenced this pull request May 18, 2026
Rebase folded in main's depth-aware `hitTestSlide` (added with the
group / ungroup work in #263 and reused by the connector endpoint
drag fix in #265). The integration commit reroutes that function
through the new precise `hitTestElement`, which two existing tests
now need:

- `hit-test-elements.test.ts` — pass a `ctx` option to every
  `hitTestSlide` call, and give the shape factory a `fill` so the
  rect has a visible body for `isPointInPath` to land on. (The
  previous fixture relied on bbox-only hit-test which the precise
  test rejects for empty shapes.)
- `editor.test.ts` body-drag-of-half-attached-connector — the click
  used to land in empty bbox space; move it onto the line's midpoint
  with the same drag delta so the connector still gets selected.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@hackerwins hackerwins mentioned this pull request May 24, 2026
2 tasks
hackerwins added a commit that referenced this pull request Jun 11, 2026
Three slides-editor bugs surfaced while testing groups; they travel
together because the same `/shared/<id>` deck exercises them in
sequence.

1. Lines drift after grouping their endpoints (the reported bug).
   `MemSlidesStore.group()` keeps a connector as a group child when
   both endpoints reference candidates, but `drawElement`'s connector
   branch skipped the per-element frame transform — `drawConnector`
   expects world-coord endpoints and `buildElementWorldLookup` lifts
   grouped targets to world, so the group transform compounded on top
   of already-world endpoints and the line jumped by the group's
   offset.

   `drawElement` now threads a cumulative `parentTransform`; on a
   connector child it inverts the parent transform onto the ctx and
   hands `drawConnector` the lookup's view of the connector — free
   endpoints are world-coord via `walkWorld`, attached endpoints
   already resolve through the lookup, so both kinds agree. The
   "v1 group() never includes connectors" NOTE in
   `element-renderer.ts` was stale since #263; #320 then bolted the
   world-coord lookup on top, which is exactly what broke this.

2. Rotate angle tooltip drifts by the pasteboard offset.
   `acquireRotateTooltip` appends to `overlay.parentElement` so
   `renderOverlay`'s innerHTML reset can't wipe it mid-drag, but
   `showTooltip` was computing coords against `overlay`'s rect.
   These containers shared an origin until #353 gave the overlay a
   non-zero `slideOffsetCssX/Y` inside `canvasWrap`. Measure against
   the tooltip's actual parent rect instead.

3. Rotate tooltip flickers at the previous drag's last position on
   re-acquire. `acquireRotateTooltip` flipped `display: block`
   immediately while `transform` still held the previous drag's
   terminal coords. Acquire now returns the cached element hidden;
   `startRotate` calls `showTooltip(clientX, clientY, 0)` once
   immediately so `transform` and `display: block` land in the same
   paint frame. Bonus: a 0° readout pops at the click position with
   no input latency.

Pre-merge code review surfaced one blocking concern and seven
non-blocking follow-ups (tracked in the task doc).

Fixed before merge: `invertGroupTransform` now returns `null`
instead of throwing on a singular matrix, and the connector branch
skips the paint. A degenerate group (frame.w = 0 with refSize > 0,
reachable via PPTX import) used to throw, escape `drawElement`'s
try/finally, and blank every later element on the slide.

Deferred follow-ups: flip blindness in `groupToTransform`;
reference-equality identity gate; ghost connector inside a group
ghost; `tooltipContainer` cached once at `startRotate`;
`overlay.parentElement` null fallback inside `acquireRotateTooltip`;
three copies of the 6-coef affine inverse math
(`element-renderer.ts` + `model/group.ts`); acquire-then-show
invariant naming.

Regression coverage:
- `slide-renderer.test.ts` — attached connector: `ctx.transform`
  runs before `moveTo` with the expected inverse coefficients; free
  connector: pre/post-group `moveTo`/`lineTo` args match (the lookup
  swap); singular parent transform: skips that connector but keeps
  painting the rest of the slide.
- `editor.test.ts` — pasteboard layout: tooltip transform encodes
  parent-frame coords, not overlay-relative; two-cycle drag: second
  pointerdown's transform reflects the new click position, not the
  previous drag's last move.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant