Skip to content

fix(gmail-watcher): catch renewal interval errors#100342

Merged
vincentkoc merged 6 commits into
openclaw:mainfrom
cxbAsDev:fix/gmail-watcher-renewal-catch
Jul 6, 2026
Merged

fix(gmail-watcher): catch renewal interval errors#100342
vincentkoc merged 6 commits into
openclaw:mainfrom
cxbAsDev:fix/gmail-watcher-renewal-catch

Conversation

@cxbAsDev

@cxbAsDev cxbAsDev commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

The Gmail watcher sets up a setInterval that calls startGmailWatch(runtimeConfig) as a fire-and-forget void expression in two places:

  1. The gateway watcher (src/hooks/gmail-watcher.ts).
  2. The foreground openclaw webhooks gmail run path (src/hooks/gmail-ops.ts).

In the foreground path, the local startGmailWatch helper does not internally catch runCommandWithTimeout failures. If the periodic Gmail API watch registration rejects, the rejection becomes an unhandled rejection and can crash the foreground process hours after startup.

Why This Change Was Made

Added a .catch to both renewal startGmailWatch calls. On rejection, a warning/error is logged and the watcher keeps running, so the next renewal cycle can retry. The initial startup path is unchanged.

User Impact

Long-running gateway and foreground Gmail watchers no longer crash on transient Gmail API renewal failures.

Evidence

Real behavior proof builds fake gog executables on disk, prepends them to PATH, and exercises the actual watcher code. The fake gog succeeds on the initial watch start, runs serve, then deletes itself so every subsequent renewal watch start fails with ENOENT.

Command:

node_modules/.bin/tsx scripts/proof/gmail-watcher-renewal-catch.mts

After the fix both paths pass. The proof now also terminates the fake gog watch serve child before the foreground process exits and fails if spawnSync itself errors:

=== Results ===
Gateway watcher: PASS
Foreground service: PASS

PASS: both gateway and foreground watchers survive a renewal failure.

Before the fix the foreground path leaks unhandled rejections and the proof fails:

=== Foreground service full output ===
...
UNHANDLED_REJECTION: Error: spawn gog ENOENT
UNHANDLED_REJECTION: Error: spawn gog ENOENT
...
FOREGROUND_UNHANDLED_COUNT: 20

=== Results ===
Gateway watcher: PASS
Foreground service: FAIL

FAIL: one or more proof steps did not pass.

Regression tests still pass:

node scripts/run-vitest.mjs src/hooks/gmail-watcher.test.ts src/hooks/gmail-ops.test.ts
Test Files  2 passed (2)
     Tests  10 passed (10)

@openclaw-barnacle openclaw-barnacle Bot added scripts Repository scripts size: S labels Jul 5, 2026
@cxbAsDev

cxbAsDev commented Jul 5, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@clawsweeper

clawsweeper Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed July 6, 2026, 1:21 AM ET / 05:21 UTC.

Summary
The branch adds a rejection handler to the foreground Gmail watch renewal interval and adds a focused runGmailService regression test.

PR surface: Source +2, Tests +118. Total +120 across 2 files.

Reproducibility: yes. Source inspection shows current main discards a renewal promise whose callee can reject, and the PR’s focused test/proof path simulates that rejected renewal.

Review metrics: none identified.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • none.

Next step before merge

  • No ClawSweeper repair lane is needed; the PR has no blocking findings and should proceed through ordinary maintainer review and exact-head merge gates.

Security
Cleared: Cleared: the final diff only changes Gmail renewal error handling and a focused test, with no dependency, workflow, package, secret-storage, or downloaded-code changes.

Review details

Best possible solution:

Merge this narrow foreground renewal catch after normal maintainer review, while leaving broader invalid-OAuth terminal watcher behavior to the existing Gmail watcher health work.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows current main discards a renewal promise whose callee can reject, and the PR’s focused test/proof path simulates that rejected renewal.

Is this the best way to solve the issue?

Yes. Catching only the renewal call preserves initial startup failure behavior while preventing later transient renewal errors from becoming unhandled rejections; broader terminal OAuth classification belongs to the related watcher-health PR.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 06fbd84d2b1a.

Label changes

Label justifications:

  • P2: The PR fixes a bounded Gmail foreground watcher availability bug without changing config, persisted state, or broader provider routing.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): Sufficient: the PR discussion includes before/after terminal output from a fake-gog real-code proof, and the latest head keeps the foreground runtime catch that proof exercises.
  • proof: sufficient: Contributor real behavior proof is sufficient. Sufficient: the PR discussion includes before/after terminal output from a fake-gog real-code proof, and the latest head keeps the foreground runtime catch that proof exercises.
Evidence reviewed

PR surface:

Source +2, Tests +118. Total +120 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 3 1 +2
Tests 1 118 0 +118
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 121 1 +120

What I checked:

  • Repository policy read: Root AGENTS.md was read fully enough to apply OpenClaw PR review, proof, and compatibility guidance; no narrower src/hooks AGENTS.md exists for the touched files. (AGENTS.md:1, 06fbd84d2b1a)
  • Current main missing foreground renewal catch: runGmailService schedules void startGmailWatch(runtimeConfig) in the renewal interval on current main, so a rejected renewal promise has no handler. (src/hooks/gmail-ops.ts:323, 06fbd84d2b1a)
  • Process helper can reject on spawn errors: runCommandWithTimeout rejects from the child error event, which is the failure mode this PR catches for renewal calls. (src/process/exec.ts:643, 06fbd84d2b1a)
  • PR head catches rejected renewal: At PR head, the foreground renewal interval catches rejected startGmailWatch calls and logs the failure without changing the initial startup path. (src/hooks/gmail-ops.ts:325, b7f04da8eecd)
  • Focused regression coverage: The added test makes initial watch registration succeed, makes later renewal calls reject, advances the renewal timer, and asserts the logged renewal failure. (src/hooks/gmail-ops.test.ts:88, b7f04da8eecd)
  • Aggregate diff checked: The final PR diff against current main is limited to src/hooks/gmail-ops.ts and a new src/hooks/gmail-ops.test.ts; earlier proof-script and gateway-watcher churn is not in the final aggregate diff. (b7f04da8eecd)

Likely related people:

  • steipete: Git shortlog shows the most history on the Gmail hooks files, and commit metadata ties this handle to the original Gmail hooks wizard plus later option refactors. (role: area contributor; confidence: high; commits: 523d9ec3c2e4, 77db65d6692b, 1fe9f648b1f6; files: src/hooks/gmail-ops.ts, src/hooks/gmail.ts, src/cli/webhooks-cli.ts)
  • jverdi: GitHub commit metadata shows this handle authored the original gateway Gmail watcher surface that shares the same watch-renew invariant. (role: introduced adjacent watcher surface; confidence: medium; commits: ca9b0dbc887d; files: src/hooks/gmail-watcher.ts)
  • vincentkoc: The live PR is assigned to this handle, and the latest PR head commit narrows the branch to the foreground runtime catch and focused test. (role: recent reviewer and branch maintainer; confidence: high; commits: b7f04da8eecd; files: src/hooks/gmail-ops.ts, src/hooks/gmail-ops.test.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (7 earlier review cycles)
  • reviewed 2026-07-05T14:28:41.371Z sha 3539757 :: needs real behavior proof before merge. :: [P2] Cover the foreground renewal path too | [P2] Make the regression fail before the fix
  • reviewed 2026-07-05T15:20:15.557Z sha bdb58ce :: needs real behavior proof before merge. :: none
  • reviewed 2026-07-05T16:13:20.097Z sha 31ee824 :: needs real behavior proof before merge. :: [P2] Stop the foreground proof child
  • reviewed 2026-07-05T16:18:45.278Z sha 31ee824 :: needs real behavior proof before merge. :: [P2] Stop the foreground proof child
  • reviewed 2026-07-05T18:12:01.040Z sha 1f04f50 :: needs real behavior proof before merge. :: [P2] Stop the fake gog child before exiting proof
  • reviewed 2026-07-06T02:13:04.029Z sha f601a63 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-06T05:13:48.979Z sha 45ba0db :: needs maintainer review before merge. :: none

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. labels Jul 5, 2026
@cxbAsDev

cxbAsDev commented Jul 5, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Addressed the review findings:

  1. Covered the foreground renewal path — added the same .catch() to the openclaw webhooks gmail run interval in src/hooks/gmail-ops.ts:325-327.
  2. Added a regression test that fails against current main — new src/hooks/gmail-ops.test.ts verifies the foreground interval catches runCommandWithTimeout rejections instead of leaking an unhandled rejection. Verified it fails on current main and passes with the fix.
  3. Updated proof scriptscripts/proof/gmail-watcher-renewal-catch.mts now exercises both gateway and foreground regression suites.

Local verification:

node_modules/.bin/tsx scripts/proof/gmail-watcher-renewal-catch.mts
node scripts/run-vitest.mjs src/hooks/gmail-watcher.test.ts src/hooks/gmail-ops.test.ts
node_modules/.bin/oxlint src/hooks/gmail-ops.ts src/hooks/gmail-ops.test.ts scripts/proof/gmail-watcher-renewal-catch.mts

All clean.

@clawsweeper

clawsweeper Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@cxbAsDev
cxbAsDev force-pushed the fix/gmail-watcher-renewal-catch branch from bdb58ce to 31ee824 Compare July 5, 2026 16:08
@cxbAsDev

cxbAsDev commented Jul 5, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Replaced the Vitest-only proof with a real behavior proof that builds fake gog executables on disk and exercises the actual production code:

  • scripts/proof/gmail-watcher-renewal-catch.mts now runs both the gateway watcher (startGmailWatcher) and the foreground service (runGmailService) against a fake gog that succeeds once and then fails every renewal.
  • Verified before/after:
    • With the .catch() fix reverted, the foreground path emits UNHANDLED_REJECTION: Error: spawn gog ENOENT and the proof FAILs.
    • With the fix, both paths log the renewal failure and survive; proof PASSes.
  • oxlint clean and regression tests still pass (10/10).

PR body updated with the new proof output.

@clawsweeper

clawsweeper Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@cxbAsDev

cxbAsDev commented Jul 5, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Fixed the foreground proof child to exit reliably instead of relying on the spawnSync timeout:

  • The child now stores the runGmailService promise and attaches a rejection handler, so an async startup failure is still observable.
  • After the 3-second observation window it explicitly calls process.exit(0) so the spawned gog serve child does not keep the proof alive until the outer timeout.
  • Re-ran the proof locally; both gateway and foreground paths PASS.
node_modules/.bin/tsx scripts/proof/gmail-watcher-renewal-catch.mts

Result:

Gateway watcher: PASS
Foreground service: PASS

PASS: both gateway and foreground watchers survive a renewal failure.

Regression tests and lint still clean:

node scripts/run-vitest.mjs src/hooks/gmail-watcher.test.ts src/hooks/gmail-ops.test.ts
npx oxlint --config .oxlintrc.json src/hooks/gmail-ops.ts src/hooks/gmail-ops.test.ts scripts/proof/gmail-watcher-renewal-catch.mts src/hooks/gmail-watcher.ts src/hooks/gmail-watcher.test.ts

@clawsweeper

clawsweeper Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@cxbAsDev

cxbAsDev commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Fixed the foreground proof per the latest review:

  • The fake gog watch serve child now writes its PID to a file and the foreground proof child sends SIGTERM to it before exiting, so the proof no longer relies on the spawnSync timeout to clean up.
  • runForegroundProof now fails when spawnSync returns result.error.
  • Re-ran the proof locally; both gateway and foreground paths PASS.
  • Regression tests still pass (10/10).

Branch pushed: f601a63ffe.

@clawsweeper

clawsweeper Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jul 6, 2026
@vincentkoc
vincentkoc force-pushed the fix/gmail-watcher-renewal-catch branch from f601a63 to f9d6b99 Compare July 6, 2026 04:50
@openclaw-barnacle openclaw-barnacle Bot added size: S and removed scripts Repository scripts size: M labels Jul 6, 2026
@vincentkoc vincentkoc self-assigned this Jul 6, 2026
@vincentkoc
vincentkoc force-pushed the fix/gmail-watcher-renewal-catch branch from f9d6b99 to 45ba0db Compare July 6, 2026 05:03
@clawsweeper clawsweeper Bot removed proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jul 6, 2026
@vincentkoc
vincentkoc force-pushed the fix/gmail-watcher-renewal-catch branch from 45ba0db to b7f04da Compare July 6, 2026 05:14
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jul 6, 2026
@vincentkoc

Copy link
Copy Markdown
Member

Land-ready verification completed on exact head b7f04da8eecd1b1df6ff740828c53697269e8ff3.

The two-file runtime/test repair is ready for the native prepare/merge gates.

@vincentkoc
vincentkoc merged commit b6e4cd6 into openclaw:main Jul 6, 2026
113 of 120 checks passed
@vincentkoc

Copy link
Copy Markdown
Member

Merged via squash.

github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 6, 2026
* fix(gmail-watcher): catch renewal interval errors

* fix(gmail-ops): catch foreground renewal interval errors; add failing regression test

* proof(gmail-watcher): rewrite proof with real fake-gog subprocess for before/after

* fix(proof): make gmail-watcher foreground proof child exit reliably

* fix(proof): kill fake gog serve child and check spawnSync error in renewal proof

* test(gmail): narrow renewal rejection coverage

---------

Co-authored-by: Vincent Koc <[email protected]>
giodl73-repo pushed a commit to giodl73-repo/openclaw that referenced this pull request Jul 8, 2026
* fix(gmail-watcher): catch renewal interval errors

* fix(gmail-ops): catch foreground renewal interval errors; add failing regression test

* proof(gmail-watcher): rewrite proof with real fake-gog subprocess for before/after

* fix(proof): make gmail-watcher foreground proof child exit reliably

* fix(proof): kill fake gog serve child and check spawnSync error in renewal proof

* test(gmail): narrow renewal rejection coverage

---------

Co-authored-by: Vincent Koc <[email protected]>
@cxbAsDev
cxbAsDev deleted the fix/gmail-watcher-renewal-catch branch July 15, 2026 03:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants