Skip to content

fix(lock): expire stale owners before pid liveness#86892

Merged
steipete merged 5 commits into
openclaw:mainfrom
Alix-007:issue-86814-stale-lock-created-at
May 26, 2026
Merged

fix(lock): expire stale owners before pid liveness#86892
steipete merged 5 commits into
openclaw:mainfrom
Alix-007:issue-86814-stale-lock-created-at

Conversation

@Alix-007

@Alix-007 Alix-007 commented May 26, 2026

Copy link
Copy Markdown
Contributor

Fixes #86814.

Summary

  • make shared stale-lock recovery honor invalid/expired createdAt before trusting PID liveness
  • keep non-expired live-PID locks protected from permission-denied/alive process probes
  • update plugin-sdk file-lock coverage for the reused-PID stale lock case

Real behavior proof

  • Behavior or issue addressed: Expired OpenClaw .lock files should be reclaimed even when the lock payload contains a PID that is currently alive or has been reused; non-expired live-PID locks should still be preserved.
  • Real environment tested: Local OpenClaw checkout at PR head 6e5c406d18, Linux, Node v22.22.0, runtime path using real acquireFileLock() from src/plugin-sdk/file-lock.ts.
  • Exact steps or command run after this patch: Ran a live Node/tsx script that created an auth-profiles.json.lock sidecar with pid: process.pid and an expired createdAt, then called acquireFileLock(filePath, { stale: 10, retries: 0 }).
  • Evidence after fix: Terminal output from the real runtime command after this patch:
environment=node v22.22.0 platform=linux
tempDir=/media/vdc/0668001470/tmp/openclaw-pr-86892-proof-Ee9yPc
beforeLock={"pid":203424,"createdAt":"2026-05-26T14:06:10.291Z"}
acquiredLockPath=/media/vdc/0668001470/tmp/openclaw-pr-86892-proof-Ee9yPc/auth-profiles.json.lock
afterLock={
  "pid": 203424,
  "createdAt": "2026-05-26T14:07:10.299Z"
}

result=reclaimed_expired_lock_with_live_pid
  • Observed result after fix: The lock was acquired successfully and the sidecar lock payload was rewritten with a fresh createdAt, proving the expired lock was reclaimed even though the recorded PID was the currently running process.
  • What was not tested: No cross-process PID reuse race was forced at the OS scheduler level; the live-PID stale-lock condition was reproduced by using the current process PID in the stale lock payload, which exercises the same liveness branch deterministically.

Local verification

  • git diff --check origin/main...HEAD
  • OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS=30000 OPENCLAW_VITEST_MAX_WORKERS=1 timeout 240s node scripts/test-projects.mjs src/infra/stale-lock-file.test.ts src/plugin-sdk/file-lock.test.ts
  • node scripts/run-oxlint.mjs --tsconfig config/tsconfig/oxlint.core.json src/agents/tools/image-tool.ts
  • pnpm check:changed

@clawsweeper re-review

@Alix-007

Copy link
Copy Markdown
Contributor Author

Current head is 51255e6c71. Local verification completed successfully:

  • git diff --check origin/main...HEAD
  • OPENCLAW_VITEST_MAX_WORKERS=1 node scripts/run-vitest.mjs src/infra/stale-lock-file.test.ts src/plugin-sdk/file-lock.test.ts --pool forks --maxWorkers=1 --reporter=tap
  • pnpm check:changed

The initial GitHub Actions failures on this head appear to be runner/platform setup failures rather than code failures. Examples from the failed logs:

  • actions/upload-artifact@v7 / pnpm/action-setup / github/codeql-action archive downloads failed from codeload.github.com during job setup.
  • Real behavior proof failed in checkout with Your account is suspended / HTTP 403 while fetching openclaw/openclaw, even though the PR author account is not suspended and git ls-remote https://github.com/openclaw/openclaw.git HEAD works locally.

I cannot rerun failed workflows from this account because GitHub requires repository admin rights for gh run rerun. @clawsweeper re-review

@clawsweeper

clawsweeper Bot commented May 26, 2026

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Command router queued. I will update this comment with the next step.

@Alix-007
Alix-007 force-pushed the issue-86814-stale-lock-created-at branch from 51255e6 to 6e5c406 Compare May 26, 2026 14:02
@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: S labels May 26, 2026
@Alix-007

Copy link
Copy Markdown
Contributor Author

Rebased this PR onto current main after the upstream CI workflow/node fallback fixes landed, and pushed a fresh head to replace the earlier platform-failed run.

Current head: 6e5c406d18

I also included the same tiny lint-only cleanup for two redundant image-tool.ts registry casts that now block pnpm check:changed on the latest base; this does not change runtime behavior.

Current local verification:

  • git diff --check origin/main...HEAD
  • OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS=30000 OPENCLAW_VITEST_MAX_WORKERS=1 timeout 240s node scripts/test-projects.mjs src/infra/stale-lock-file.test.ts src/plugin-sdk/file-lock.test.ts
  • node scripts/run-oxlint.mjs --tsconfig config/tsconfig/oxlint.core.json src/agents/tools/image-tool.ts
  • pnpm check:changed

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented May 26, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@openclaw-barnacle openclaw-barnacle Bot added the triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. label May 26, 2026
@clawsweeper

clawsweeper Bot commented May 26, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed May 26, 2026, 10:31 AM ET / 14:31 UTC.

Summary
The PR reorders stale file-lock reclamation, updates lock tests, removes redundant image-tool registry casts, and adds transcripts tool-display metadata/snapshot coverage.

PR surface: Source +17, Tests +7, Other +15. Total +39 across 6 files.

Reproducibility: yes. Current main source has a deterministic helper path where a payload with both pid and expired createdAt returns the live-PID result before considering expiry.

Review metrics: none identified.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Next step before merge
No repair lane is needed because the PR already makes the narrow code and regression-test changes; remaining work is normal CI and maintainer merge review.

Security
Cleared: No concrete security or supply-chain issue was found; the diff changes local lock reclamation logic and generated tool metadata without new dependencies, permissions, workflows, or secret exposure.

Review details

Best possible solution:

Land the small shared helper fix after required CI and maintainer checks so expired locks are reclaimed before PID liveness while non-expired live locks remain protected.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main source has a deterministic helper path where a payload with both pid and expired createdAt returns the live-PID result before considering expiry.

Is this the best way to solve the issue?

Yes. The PR fixes the shared stale-lock helper used by both core and plugin-sdk lock callers and keeps the non-expired live-PID protection intact.

AGENTS.md: found and applied where relevant.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 35310dce8c31.

Label changes

Label changes:

  • add proof: sufficient: Contributor real behavior proof is sufficient. The PR body supplies terminal proof from a real acquireFileLock() runtime path reclaiming the expired live-PID lock; contributors should keep private paths or secrets redacted in future proof.
  • add rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • remove rating: 🐚 platinum hermit: Current PR rating is rating: 🦞 diamond lobster, so this older rating label is no longer current.

Label justifications:

  • P1: The linked bug affects real auth-profile lock recovery and can delay active channel replies by the file-lock retry tail.
  • rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body supplies terminal proof from a real acquireFileLock() runtime path reclaiming the expired live-PID lock; contributors should keep private paths or secrets redacted in future proof.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body supplies terminal proof from a real acquireFileLock() runtime path reclaiming the expired live-PID lock; contributors should keep private paths or secrets redacted in future proof.
Evidence reviewed

PR surface:

Source +17, Tests +7, Other +15. Total +39 across 6 files.

View PR surface stats
Area Files Added Removed Net
Source 3 23 6 +17
Tests 2 26 19 +7
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 1 15 0 +15
Total 6 64 25 +39

What I checked:

  • Current main bug path: shouldRemoveDeadOwnerOrExpiredLock() currently returns the PID liveness result before evaluating createdAt, so an expired lock with a reused live PID is preserved. (src/infra/stale-lock-file.ts:27, 35310dce8c31)
  • Auth profile impact path: markAuthProfileSuccess() updates auth-profiles.json through updateAuthProfileStoreWithLock(), which uses the shared withFileLock() wrapper affected by this stale-lock decision. (src/agents/auth-profiles/store.ts:705, 35310dce8c31)
  • Existing lock stale windows: Auth store locks use a 30s stale window and OAuth refresh locks have an explicit timeout-below-stale invariant, so the PR changes the existing stale-expiry rule rather than adding a new config/default surface. (src/agents/auth-profiles/constants.ts:18, 35310dce8c31)
  • PR fix shape: The PR diff changes shouldRemoveDeadOwnerOrExpiredLock() to return true for invalid/expired createdAt before consulting PID liveness, and keeps PID liveness for non-expired locks. (src/infra/stale-lock-file.ts:27, 0797500f7e97)
  • Regression coverage: The PR adds source-level coverage for expired locks with a live PID and preserves non-expired live-PID behavior, plus plugin-sdk coverage for the reused PID path. (src/infra/stale-lock-file.test.ts:5, 0797500f7e97)
  • Real behavior proof: The PR body includes terminal output from a real Linux Node runtime using acquireFileLock() that reclaimed an expired .lock file whose payload PID was the current process. (src/plugin-sdk/file-lock.ts:78, 0797500f7e97)

Likely related people:

  • vincentkoc: Git blame and log show the current stale-lock helper, plugin-sdk lock wrapper, and initial tests were introduced in dcf0941cd6b3bde338a2d047cb9ae085c042a339. (role: introduced lock wrapper behavior; confidence: high; commits: dcf0941cd6b3; files: src/infra/stale-lock-file.ts, src/plugin-sdk/file-lock.ts, src/plugin-sdk/file-lock.test.ts)
  • steipete: Recent main commits moved transcripts into core and changed image-tool registry typing, which are the adjacent surfaces touched by this PR’s rebase cleanup. (role: recent adjacent area contributor; confidence: medium; commits: cac0b2db18ea, 30e59b409015; files: src/agents/openclaw-tools.ts, src/agents/tools/transcripts-tool.ts, src/agents/tools/image-tool.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@Alix-007

Copy link
Copy Markdown
Contributor Author

Updated the PR body with structured real behavior proof fields required by the proof gate. The after-fix evidence now includes terminal output from a real acquireFileLock() runtime command that reclaimed an expired auth-profiles.json.lock payload whose PID was the currently running process.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented May 26, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@openclaw-barnacle openclaw-barnacle Bot added proof: supplied External PR includes structured after-fix real behavior proof. and removed triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 26, 2026
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P1 High-priority user-facing bug, regression, or broken workflow. labels May 26, 2026
@clawsweeper

clawsweeper Bot commented May 26, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper PR egg

✨ Hatched: 🥚 common Velvet Merge Sprite

Hatch command

Comment @clawsweeper hatch when this PR is hatchable.

Hatchability rules:

  • Merged PRs are hatchable.
  • Open PRs are hatchable when they are status: 👀 ready for maintainer look, status: 🚀 automerge armed, or labeled clawsweeper:automerge.
  • Closed unmerged PRs are hatchable only when one of those hatchable labels is still present in the durable record.

Rarity: 🥚 common.
Trait: sparkles near resolved comments.
Image traits: location branch lighthouse; accessory miniature diff map; palette charcoal, cyan, and signal green; mood patient; pose holding its accessory up for inspection; shell starlit enamel shell; lighting soft studio lighting; background tiny shells and proof notes.
Share on X: post this hatch
Copy: My PR egg hatched a 🥚 common Velvet Merge Sprite in ClawSweeper.

What is this egg doing here?
  • Eggs appear after the PR passes real-behavior proof. It is here for vibes, not verdicts: it does not change labels, ratings, merge decisions, or automation.
  • The shell reacts to review momentum: open follow-up work warms it up, re-review makes it wobble, and a clean final review lets it hatch.
  • Hatchability usually comes from sufficient real-behavior proof, no blocking P0/P1/P2 findings, no security attention needed, and clean correctness. A merged PR is already final, so merge makes the egg hatchable independently.
  • The hatch is seeded from this repository and PR number, so the same PR keeps the same creature; the reviewed head SHA can only change safe visual details.
  • Rarity is just collectible sparkle: 🥚 common, 🌱 uncommon, 💎 rare, ✨ glimmer, and 🌈 legendary.

@Alix-007
Alix-007 force-pushed the issue-86814-stale-lock-created-at branch from a8be0a9 to 0797500 Compare May 26, 2026 14:24
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 26, 2026
@Alix-007

Copy link
Copy Markdown
Contributor Author

Follow-up pushed at 0797500f7e after the latest main moved the transcripts runtime tool into core.

This fixes the check-guards failure from tool-display:check by adding display metadata for the transcripts runtime tool and refreshing the generated tool-display.json snapshot.

Current focused verification after the rebase:

  • node --import tsx scripts/tool-display.ts --check
  • git diff --check origin/main...HEAD
  • OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS=30000 OPENCLAW_VITEST_MAX_WORKERS=1 timeout 240s node scripts/test-projects.mjs src/infra/stale-lock-file.test.ts src/plugin-sdk/file-lock.test.ts

The previous Real behavior proof gate is passing on the structured proof now. @clawsweeper re-review

@clawsweeper

clawsweeper Bot commented May 26, 2026

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Command router queued. I will update this comment with the next step.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels May 26, 2026
@steipete steipete self-assigned this May 26, 2026
@steipete
steipete force-pushed the issue-86814-stale-lock-created-at branch from 0797500 to b9102b0 Compare May 26, 2026 20:22
@openclaw-barnacle openclaw-barnacle Bot removed the agents Agent runtime and tooling label May 26, 2026
@steipete

Copy link
Copy Markdown
Contributor

Maintainer fixup pushed for the merge blocker.

Behavior addressed: stale plugin lock recovery now requires owner identity proof before removing a PID-owned sidecar. Timestamp expiry alone no longer steals a live owner's lock; future plugin lock payloads include Linux starttime when available so PID reuse can be detected safely.
Real environment tested: local macOS checkout, Node/pnpm repo test harness.
Exact steps or command run after this patch: pnpm test src/infra/stale-lock-file.test.ts src/plugin-sdk/file-lock.test.ts; /Users/steipete/Projects/agent-skills/skills/autoreview/scripts/autoreview --mode branch --base origin/main; git diff --check; duplicate display key check with rg for transcripts in the TS and Swift resource display configs.
Evidence after fix: both Vitest shards passed (2 files, 10 tests); autoreview clean with no accepted/actionable findings; only one transcripts display entry remains in each display config.
Observed result after fix: recycled PID with mismatched starttime is reclaimable, dead PID is reclaimable, live/unknown PID owners stay fail-closed, and pidless expired locks still expire.
What was not tested: full CI/Testbox; GitHub checks are running on the pushed PR SHA.

@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P1 High-priority user-facing bug, regression, or broken workflow. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stale lock recovery ignores createdAt when recorded PID is reused

2 participants