Skip to content

improve(ci): remove remaining warm-cache critical paths#109593

Merged
steipete merged 1 commit into
mainfrom
codex/ci-cache-followups
Jul 17, 2026
Merged

improve(ci): remove remaining warm-cache critical paths#109593
steipete merged 1 commit into
mainfrom
codex/ci-cache-followups

Conversation

@steipete

@steipete steipete commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Related: #109425

What Problem This Solves

Warm transform caches exposed two remaining CI critical-path costs: the broad core shard started late in the full matrix, one registration test spent minutes evaluating the complete command-handler graph for two order assertions, and self-built plugin SDK declarations were regenerated on every build-artifact job.

Why This Change Was Made

Admit the broad core-fast cache writer in the first full-matrix wave without changing shard layout. Make command precedence a typed, data-only contract so the order test avoids importing every handler while the runtime still proves complete handler coverage. Persist self-built declaration outputs behind a content fingerprint of the repository-owned TypeScript/JSON graph, excluding installed and generated directories; rotate the outer CI cache key to match.

The declaration cache is job-local while writing and immutable between jobs. Stale coarse restores remain safe because build-all verifies its own versioned input fingerprint before restoring outputs. Cache size is bounded by the existing public declaration graph ratchet; the measured entry is 7 MB.

User Impact

No product behavior changes. Maintainers should see the first reusable transform seed start earlier, the former core-fast quiet tail disappear, and warm build-artifact jobs skip roughly 24 seconds of declaration generation even after partial non-source changes.

Evidence

  • Command registration test before/after on the same checkout: 264.84s import / 269.36s wrapper wall time to 13ms import / 3.56s wrapper wall time. Final Vitest duration: 99ms; 2 tests passed.
  • Blacksmith cold/warm build:ci-artifacts: declaration phase 23.8s to 157ms; total build 41.2s to 7.94s. The 7 MB cache restored flat declarations after tsdown cleared dist, and the plugin SDK export graph gate passed both runs.
  • Exact-head hosted CI run 29555352240 completed successfully. One tooling shard was killed once by SIGKILL; its exact rerun passed 88 files / 1,457 tests in 66.59s, matching two earlier green runs (66.09s and 64.92s) with the same Vitest/cache configuration. No assertion, timeout, cache error, ENOTEMPTY, EEXIST, EBUSY, ENOSPC, or cancellation signature appeared.
  • Earlier same-SHA warm proof restored the exact build-all-v4 key: build-artifacts fell from 152s to 76s, while representative checks-node-compact-large-1 fell from 169s to 116s. Across all 26 paired Linux Node jobs, summed job duration fell from 3301s to 2954s; 20 improved, with a median 8.5s reduction. The registration test completed in 0ms inside the shard.
  • A cold GitHub-hosted control run reduced the core-fast test phase from the prior 232.55s baseline to 173.32s (25.5%). Its cache writer saved a 55 MB reusable transform seed.
  • Focused suites: 141 tests passed, 1 skipped across build cache, shard planner, workflow guards, and registration order.
  • Exact-head changed gate: all conflict, formatting, dependency, SDK API, boundary, core/core-test/root-test typecheck, lint, database-first, import-cycle, webhook, and pairing guards passed. Blacksmith had already passed the changed surfaces and typechecks; the final rerun used the trusted local fallback after a new Testbox could not complete targeted or full sync.
  • Autoreview clean after replacing an implicit Set matcher with an explicit size comparison.

AI-assisted: yes. I reviewed the cache invalidation/output contracts, handler ordering, admission behavior, and validation results.

@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation scripts Repository scripts size: M maintainer Maintainer-authored PR labels Jul 17, 2026
@steipete
steipete force-pushed the codex/ci-cache-followups branch from 63714a3 to ec87bd9 Compare July 17, 2026 03:21

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 21e1c11cc5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/ci.yml
with:
path: .artifacts/build-all-cache
key: ${{ runner.os }}-build-all-v3-${{ hashFiles('package.json', 'pnpm-lock.yaml', 'npm-shrinkwrap.json', 'packages/plugin-sdk/package.json', 'packages/llm-core/package.json', 'packages/model-catalog-core/package.json', 'packages/memory-host-sdk/package.json', 'scripts/build-all.mjs', 'scripts/write-plugin-sdk-entry-dts.ts', 'scripts/lib/plugin-sdk-entries.mjs', 'tsconfig.json', 'tsconfig.plugin-sdk.dts.json', 'src/plugin-sdk/**', 'packages/llm-core/src/**', 'packages/model-catalog-core/src/**', 'packages/memory-host-sdk/src/**', 'src/types/**', 'src/video-generation/dashscope-compatible.ts', 'src/video-generation/types.ts', 'scripts/copy-export-html-templates.ts', 'scripts/lib/copy-assets.ts', 'src/auto-reply/reply/export-html/**') }}
key: ${{ runner.os }}-build-all-v4-${{ hashFiles('package.json', 'pnpm-lock.yaml', 'npm-shrinkwrap.json', 'scripts/build-all.mjs', 'scripts/write-plugin-sdk-entry-dts.ts', 'scripts/lib/plugin-sdk-entries.mjs', 'scripts/lib/plugin-sdk-entrypoints.json', 'scripts/lib/plugin-sdk-private-local-only-subpaths.json', 'scripts/lib/plugin-sdk-deprecated-public-subpaths.json', 'scripts/lib/plugin-sdk-deprecated-barrel-subpaths.json', 'scripts/copy-export-html-templates.ts', 'scripts/lib/copy-assets.ts', 'tsconfig.json', 'tsconfig.plugin-sdk.dts.json', 'src/**', 'packages/**', '!src/**/dist/**', '!src/**/node_modules/**', '!packages/**/dist/**', '!packages/**/node_modules/**') }}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Add the SDK JSONs to build-artifact gating

When a PR changes one of the newly hashed plugin SDK JSON files here (for example scripts/lib/plugin-sdk-entrypoints.json), this cache key now correctly treats it as a build input, but the preflight predicate that decides whether build-artifacts runs still skips all scripts/lib/*.json: BUILD_INPUT_RE in scripts/lib/ci-changed-node-test-plan.mjs:42-43 only allows copy-assets.ts and plugin-sdk-entries.mjs. On canonical PRs the manifest sets run_build_artifacts from that predicate, so SDK export/deprecation-list changes can avoid pnpm build:ci-artifacts and the plugin SDK export gate entirely; please add these JSON inputs to the build-impact regex and its tests.

Useful? React with 👍 / 👎.

@steipete
steipete force-pushed the codex/ci-cache-followups branch from 21e1c11 to b3d2954 Compare July 17, 2026 03:45

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b3d2954418

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/build-all.mjs
Comment on lines +42 to +43
extensions: [".ts", ".tsx", ".mts", ".cts", ".json"],
excludeDirectories: ["dist", "node_modules"],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include .d.ts inputs in the SDK declaration cache

When a PR changes ambient declaration inputs such as src/types/*.d.ts, build-artifacts restores the broad build-all-v4- cache via the coarse restore key, but the new internal fingerprint for the self-built plugin SDK declarations only hashes .ts/.tsx/.mts/.cts/.json files under src and packages. tsconfig.plugin-sdk.dts.json explicitly includes src/types/**/*.d.ts, so those changes can leave write-plugin-sdk-entry-dts marked fresh and restore stale flat/package-bridge declarations before check-plugin-sdk-exports runs. Please include .d.ts in both broad source input lists so declaration-only type changes invalidate this cache.

Useful? React with 👍 / 👎.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. labels Jul 17, 2026
@clawsweeper

clawsweeper Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 17, 2026, 12:43 AM ET / 04:43 UTC.

Summary
This PR prioritizes the broad core CI shard, replaces runtime-heavy command-order tests with a typed order contract, and caches generated plugin SDK declarations using repository-source fingerprints.

PR surface: Source +49, Tests +80, Docs +1, Config 0, Other +39. Total +169 across 10 files.

Reproducibility: yes. from source: a change limited to one of the four newly fingerprinted SDK JSON manifests is not matched by the unchanged build-impact predicate, so build-artifacts can be omitted.

Review metrics: 1 noteworthy metric.

  • SDK manifest gate coverage: 4 fingerprinted, 0 added to build-impact detection. A manifest-only change can invalidate the declaration cache contract without scheduling the build job that verifies it.

Stored data model
Persistent data-model change detected: persistent cache schema: test/scripts/build-all.test.ts, persistent cache schema: test/scripts/ci-workflow-guards.test.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🦐 gold shrimp
Proof: 🦐 gold shrimp
Patch quality: 🦐 gold shrimp
Result: blocked until stronger real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Add all four SDK JSON manifests to build-impact detection with a manifest-only regression test.
  • Post current-head cold/warm workflow logs showing build-artifacts runs, validates exports, and restores the declaration cache.

Proof guidance:

  • [P1] Needs stronger real behavior proof before merge: The PR contains strong cold/warm logs and timing evidence, but those runs precede the latest force-pushed head; provide current-head artifact-gate and cache-restore output with private endpoints, credentials, and other sensitive details redacted. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] A pull request changing only one of the newly fingerprinted plugin SDK JSON manifests can avoid build-artifacts entirely, bypassing the declaration build and export-graph validation this cache is intended to protect.
  • [P1] The current branch head lacks demonstrated after-fix exact-head warm-cache proof because the cited successful runs and proof label precede the latest force-push.

Maintainer options:

  1. Repair gating and refresh proof (recommended)
    Add the four SDK manifest paths to build-artifact impact detection with regression coverage, then attach current-head cold/warm workflow evidence before merge.

Next step before merge

  • [P1] The protected maintainer label keeps this human-routed; the author should repair the remaining changed-path defect and refresh exact-head proof.

Security
Cleared: The diff changes cache and CI execution behavior but adds no third-party action, permission, secret access, dependency source, executable download, or package lifecycle hook.

Review findings

  • [P2] Gate builds on the newly fingerprinted SDK manifests — .github/workflows/ci.yml:1023
Review details

Best possible solution:

Align build-impact detection with all four SDK manifest inputs, add a manifest-only changed-path regression test, and provide a cold/warm exact-head run showing the artifact gate executes and restores the corrected cache safely.

Do we have a high-confidence way to reproduce the issue?

Yes from source: a change limited to one of the four newly fingerprinted SDK JSON manifests is not matched by the unchanged build-impact predicate, so build-artifacts can be omitted.

Is this the best way to solve the issue?

No, not yet; the cache design is coherent, but its changed-path gate must recognize every newly declared input so validation cannot be skipped.

Full review comments:

  • [P2] Gate builds on the newly fingerprinted SDK manifests — .github/workflows/ci.yml:1023
    The prior finding remains on this head: the cache key now depends on four plugin SDK JSON manifests, but BUILD_INPUT_RE still does not schedule build-artifacts for manifest-only changes. Such a PR can skip declaration generation and the plugin SDK export gate entirely; add these paths to changed-impact detection and cover that case in its tests.
    Confidence: 0.98

Overall correctness: patch is incorrect
Overall confidence: 0.97

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 7641aa80c23b.

Label changes

Label changes:

  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: The PR contains strong cold/warm logs and timing evidence, but those runs precede the latest force-pushed head; provide current-head artifact-gate and cache-restore output with private endpoints, credentials, and other sensitive details redacted. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
  • remove proof: sufficient: Current real behavior proof status is insufficient, not sufficient.
  • remove status: ⏳ waiting on author: Current PR status label is status: 📣 needs proof.

Label justifications:

  • P2: The defect is a bounded CI validation gap with limited blast radius, but it should be corrected before merging the optimization.
  • merge-risk: 🚨 automation: Merging as written could cause changed-path automation to skip build-artifact and plugin SDK export validation for four newly significant manifest inputs.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🦐 gold shrimp.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: The PR contains strong cold/warm logs and timing evidence, but those runs precede the latest force-pushed head; provide current-head artifact-gate and cache-restore output with private endpoints, credentials, and other sensitive details redacted. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +49, Tests +80, Docs +1, Config 0, Other +39. Total +169 across 10 files.

View PR surface stats
Area Files Added Removed Net
Source 2 93 44 +49
Tests 4 98 18 +80
Docs 1 2 1 +1
Config 1 2 2 0
Generated 0 0 0 0
Other 2 45 6 +39
Total 10 240 71 +169

What I checked:

  • Prior blocker remains: The current workflow cache key fingerprints four plugin SDK JSON manifests, but the PR does not update BUILD_INPUT_RE, so changes limited to those manifests can still skip build-artifacts and its plugin SDK export checks. (scripts/lib/ci-changed-node-test-plan.mjs:42, d4c7873e4529)
  • Changed cache surface: The build-all-v4 cache key adds plugin-sdk-entrypoints.json and three deprecated/private subpath manifests as inputs, making the unchanged build-impact predicate observably incomplete. (.github/workflows/ci.yml:1023, d4c7873e4529)
  • Review continuity: The earlier ClawSweeper cycle raised the same build-artifact gating defect, and the latest ten-file diff still contains no change to ci-changed-node-test-plan.mjs. (.github/workflows/ci.yml:1023, d4c7873e4529)
  • Typed runtime contract: The command runtime now constructs handlers from the typed commandHandlerOrder list and a keyed handler map, preserving a compile-time coverage boundary without importing the runtime graph in the order test. (src/auto-reply/reply/commands-handlers.runtime.ts:48, d4c7873e4529)
  • Proof chronology: The proof:sufficient label was applied at 2026-07-17T04:36:24Z, before the branch was force-pushed to d4c7873 at 2026-07-17T04:38:35Z; current-head checks remain incomplete. (d4c7873e4529)
  • Feature provenance: Merged pull request improve(ci): persist warm Vitest and Node caches #109425 introduced the warm Vitest and Node cache foundation; this PR is a follow-up optimization rather than a duplicate or already-implemented change. (.github/workflows/ci.yml, b78cf56349ad)

Likely related people:

  • steipete: Authored the merged warm-cache foundation in b78cf56 and has the clearest recent history across the affected workflow, build-cache, and shard-planning surfaces. (role: feature owner and recent area contributor; confidence: high; commits: b78cf56349ad; files: .github/workflows/ci.yml, scripts/build-all.mjs, scripts/lib/ci-node-test-plan.mjs)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (1 earlier review cycle)
  • reviewed 2026-07-17T04:33:37.290Z sha b3d2954 :: found issues before merge. :: [P2] Include declaration files in the SDK cache fingerprint | [P2] Gate build artifacts on the newly hashed SDK JSON inputs

@steipete
steipete force-pushed the codex/ci-cache-followups branch from b3d2954 to d4c7873 Compare July 17, 2026 04:38
@clawsweeper clawsweeper Bot added status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed proof: sufficient ClawSweeper judged the real behavior proof convincing. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Jul 17, 2026
@steipete
steipete merged commit e7c389b into main Jul 17, 2026
210 of 215 checks passed
@steipete
steipete deleted the codex/ci-cache-followups branch July 17, 2026 04:51
@steipete

Copy link
Copy Markdown
Contributor Author

Merged via squash.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Improvements or additions to documentation maintainer Maintainer-authored PR merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. P2 Normal backlog priority with limited blast radius. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. scripts Repository scripts size: M status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant