Skip to content

test(live): add Docs structure-preservation regressions#841

Merged
steipete merged 3 commits into
openclaw:mainfrom
sebsnyk:qa/structure-preservation-harness
Jun 18, 2026
Merged

test(live): add Docs structure-preservation regressions#841
steipete merged 3 commits into
openclaw:mainfrom
sebsnyk:qa/structure-preservation-harness

Conversation

@sebsnyk

@sebsnyk sebsnyk commented Jun 18, 2026 •
edited by steipete
Loading

Copy link
Copy Markdown
Contributor

Summary

  • add a focused Docs structure-preservation live suite with its own structure skip key
  • fingerprint every non-target paragraph, list identity, paragraph style, text-run style, and document order before mutations
  • hard-check inline and block Markdown replacements for list/heading preservation
  • hard-check non-destructive image insertion before and after a literal anchor
  • keep --fast free of Docs structure mutations

Fixes #840.

Testing

  • bash -n scripts/live-test.sh scripts/live-tests/structure.sh
  • go test ./scripts -count=1
  • make ci
  • autoreview: clean after fixing fast-mode integration
  • live Google Docs proof: all target invariants and canonical untargeted fingerprints passed; scratch document trashed

@sebsnyk @claude
test(live): skeleton structure-preservation regression suite
8349058 
Add scripts/live-tests/structure.sh (skip key: structure), wired into
live-test.sh. Seeds a rich docs fixture (headings, bulleted/numbered
lists, bold/italic/link runs) and asserts structural invariants survive
mutating ops via the existing raw --json + python-assert idiom.

Wired asserts: inline markdown find-replace preserves bullet + heading
(the openclaw#740-guarded safe path). Known-bug op openclaw#838 (block/multi-paragraph
markdown replace drops the bullet) is an xfail run_optional that flips to
a hard assert once fixed. Slides and sheets sections plus the
person-chip / inline-image (openclaw#839) / table / adjacent-order invariants are
stubbed with TODO markers for follow-up.

Covers the structure-loss class behind openclaw#838 and openclaw#839. Design + full
coverage matrix in the tracking issue.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
@clawsweeper clawsweeper Bot mentioned this pull request Jun 18, 2026
Closed
@steipete steipete changed the title test(live): skeleton structure-preservation regression suite (draft, re #840) test(live): add Docs structure-preservation regressions Jun 18, 2026
@steipete
steipete marked this pull request as ready for review June 18, 2026 11:00
@clawsweeper

clawsweeper Bot commented Jun 18, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 18, 2026, 7:06 AM ET / 11:06 UTC.

Summary
The PR adds a Docs structure-preservation live regression suite and wires a new structure skip key into the live test runner.

Reproducibility: not applicable. as a bug reproduction for this PR. Source inspection confirms current main lacks the live structure suite, and the PR body reports a live Google Docs run of the added assertions.

Review metrics: 2 noteworthy metrics.

  • Diff scope: 2 files changed; 197 additions, 4 deletions. The PR is narrow and limited to shell live-test harness coverage.
  • Live suite added: 1 Docs structure suite. Maintainers should notice that this increases full live-test coverage while --fast skips it.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #840
Summary: This PR is the candidate implementation for the open QA issue; the closed Docs product bugs are related instances that the new suite guards.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • none.

Risk before merge

  • [P1] I did not rerun live Google Docs mutations in this read-only review; confidence comes from the PR body's live proof plus source inspection of the harness and current Docs fixes.

Maintainer options:

  1. Decide the mitigation before merge
    Merge the focused live-suite addition after normal checks, keeping it aligned with the merged Docs structure and image-anchor fixes.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • No ClawSweeper repair is needed; the remaining action is normal maintainer review and merge handling.

Security
Cleared: The diff only adds shell live-test coverage and does not introduce dependency, credential, permission, or supply-chain changes.

Review details

Best possible solution:

Merge the focused live-suite addition after normal checks, keeping it aligned with the merged Docs structure and image-anchor fixes.

Do we have a high-confidence way to reproduce the issue?

Not applicable as a bug reproduction for this PR. Source inspection confirms current main lacks the live structure suite, and the PR body reports a live Google Docs run of the added assertions.

Is this the best way to solve the issue?

Yes. A separate live suite with its own skip key matches the existing live-test organization and exercises current Docs fixes through raw JSON invariants without changing product code.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 83f51c60f183.

Label changes

Label changes:

  • add P3: This is low-risk QA coverage for an already-fixed Docs data-loss class rather than an urgent runtime defect.
  • add proof: sufficient: Contributor real behavior proof is sufficient. The PR body reports a live Google Docs proof where the new structure invariants passed and the scratch document was trashed.
  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body reports a live Google Docs proof where the new structure invariants passed and the scratch document was trashed.

Label justifications:

  • P3: This is low-risk QA coverage for an already-fixed Docs data-loss class rather than an urgent runtime defect.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body reports a live Google Docs proof where the new structure invariants passed and the scratch document was trashed.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body reports a live Google Docs proof where the new structure invariants passed and the scratch document was trashed.
Evidence reviewed

What I checked:

  • Repository policy: AGENTS.md was read fully and directs PR reviews to use gh pr view/gh pr diff without changing branches or code. (AGENTS.md:37, 83f51c60f183)
  • Current main live harness: Current main sources and runs docs, sheets, and slides live tests, but has no structure.sh source or run_structure_tests call. (scripts/live-test.sh:151, 83f51c60f183)
  • PR adds focused live suite: The PR head adds run_structure_tests, creates a scratch Docs fixture, fingerprints untargeted paragraphs, and hard-checks inline replacement, block replacement, heading replacement, and non-destructive image anchors. (scripts/live-tests/structure.sh:121, 4ed4110e0ee3)
  • Live-test helper contract: The new suite uses the existing skip, run_required, gog, and Drive cleanup helpers, matching the current live-test harness shape. (scripts/live-tests/common.sh:18, 83f51c60f183)
  • Prerequisite block replacement fix: Current main already preserves the first replacement paragraph's existing structure by starting paragraph resets after that first inserted paragraph. (internal/cmd/docs_mutation.go:176, c02fdf7bc1a1)
  • Prerequisite image-anchor fix: Current main exposes --before and --after image anchors and resolves them without returning a placeholder range to delete. (internal/cmd/docs_insert_image.go:27, 83f51c60f183)

Likely related people:

  • steipete: Git history shows this person authored the current live-test harness snapshot and the merged Docs block-replacement and image-anchor fixes that this suite exercises. (role: feature-history owner and recent area contributor; confidence: high; commits: 6bc67d7dd770, c02fdf7bc1a1, 83f51c60f183; files: scripts/live-test.sh, scripts/live-tests/docs.sh, internal/cmd/docs_mutation.go)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. labels Jun 18, 2026
@steipete
steipete merged commit c172741 into openclaw:main Jun 18, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

QA: no structure-preservation regression suite for mutating commands (seed-fixture + raw-diff invariants); covers the #838/#839 class

2 participants

@sebsnyk @steipete