Skip to content

fix: retry startup resume repairs

MacroscopeApp / Macroscope - Effect Service Conventions failed Aug 28, 2026 in 2m 23s

Effect Service Conventions: 1 issue found

1 finding

  • apps/server/src/orchestration/Layers/ProviderCommandReactor.ts (lines 1242-1264): retryUsageLimitResumeDispatch uses an unbounded exponential retry schedule with no Schedule.upTo bound and no retryability predicate, so a persistent/deterministic dispatch failure is never isolated. This is reachable today: archiving a thread cancels neither the in-memory timer nor the durable usageLimitResume, so the timer fires, thread.usage-limit-resume.attempt fails requireThreadNotArchived with OrchestrationCommandInvariantError, and the effect retries forever, logging a warning every 30s. The repository's own precedent (apps/server/src/server.ts:639) uses the identical exponential+cap schedule but adds Schedule.upTo({ duration: "10 minutes" }) plus a while predicate excluding non-retryable errors.

The prior run's finding on the startup-repair path (no retry, interruption reduced to a warning) has been addressed \u2014 it now uses Effect.retry({ times: 1 }), re-fails on interrupt, and has tests for both the retry-success and persistent-failure-continuation paths.

Details

Note

Your check run agent prompt is: .macroscope/check-run-agents/effect-service-conventions.md
More information about how Check Run Agents work can be found in our Help Center.

Scope examined

All changed apps/**/*.ts(x) and packages/**/*.ts(x) files, focusing on newly introduced Effect service code and directly affected call sites:

  • apps/server/src/provider/usageLimits.ts (new pure helper module)
  • apps/server/src/orchestration/Layers/ProviderCommandReactor.ts
  • apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts
  • apps/server/src/orchestration/Layers/ProjectionPipeline.ts, ProjectionSnapshotQuery.ts
  • apps/server/src/orchestration/decider.ts, projector.ts, Schemas.ts
  • apps/server/src/persistence/** (services, layers, migration 044)
  • apps/server/src/provider/Layers/{Claude,Codex,Cursor,OpenCode}Adapter.ts
  • packages/contracts/src/orchestration.ts, providerRuntime.ts
  • packages/client-runtime/src/{operations/commands.ts,state/*}
  • apps/web/src/components/{ChatView.tsx,chat/ThreadErrorBanner.tsx}

Conventions verified as satisfied

  • Imports: all new Effect imports use subpath namespaces (effect/DateTime, effect/Fiber, effect/Schedule, effect/unstable/sql/SqlClient); no consolidated named imports from "effect". @t3tools/contracts keeps named imports as expected. usageLimits.ts is a pure-helper module, so named imports of providerUsageLimitFromError / nextUsageLimitRetryAt are correct and no namespace import is warranted.
  • Service definition / layout: no new Context.Service tags, make/layer exports, or service module moves. No new FooShape types were introduced; the pre-existing ProviderCommandReactorShape and OpenCodeAdapterShape are untouched legacy and out of scope.
  • Dependency acquisition / runtime boundaries: no new ManagedRuntime.make, Layer.succeed, or service-instance parameters. Effect.runPromise / Effect.runSync occurrences are confined to test harnesses; the PR actually replaces several bare Effect.runPromise calls with harness.runEffect.
  • Errors and predicates: no new error classes, Schema.Defect, catchTag, or catchIf-with-schema-predicate usage. Interruption is consistently preserved via Cause.hasInterruptsOnly checks before downgrading to Effect.logWarning.
  • Change discipline: new tests cover genuine behavior changes (decider lifecycle, projector, reactor recovery, adapter classification), not a mechanical refactor.

Methodology

Read the full diff, then opened ProviderCommandReactor.ts at head to inspect the new timer/dispatch helpers and the start recovery path in context, cross-checked the archive/unarchive decider branches to confirm reachability of the unbounded-retry loop, and grepped for existing Schedule.exponential usage to establish the repository's own retry precedent.


🔍 Agent activity — 7 tool calls

💭 I see the previous startup repair issue was resolved with a retry and interrupt-aware error handling. Now I'm checking the rest of the diff for new Effect service convention violations, starting with the new pure helper module for usage limits. I notice ProviderCommandReactor.ts changed Effect.failCause(cause) to Effect.interrupt in a catchCause handler, so I want to pull up the file to see the surrounding context. I'm also concerned about the usage limit resume dispatch retry using an unbounded exponential schedule that could retry forever, which conflicts with the convention that transient per-entity repairs should retry briefly before isolating persistent failures rather than retrying indefinitely.

1. browse_code — apps/server/src/orchestration/Layers/ProviderCommandReactor.ts

2. browse_code — apps/server/src/orchestration/Layers/ProviderCommandReactor.ts

💭 The switch to Effect.interrupt at line 485 still preserves interruption behavior, so that's fine. Now I'm moving on to examine the new usage-limit code section spanning lines 1242-1440.

3. browse_code — apps/server/src/orchestration/Layers/ProviderCommandReactor.ts

4. browse_code — apps/server/src/orchestration/Layers/ProviderCommandReactor.ts

💭 With that fixed, I'm scanning for other convention violations, like whether error objects are being logged raw instead of using safer formatting elsewhere in the file. I'm also checking imports across the changed files to make sure they're all consolidated from "effect" properly—going through usageLimits.ts, migration files, test files, and adapter files one by one, and so far they all look fine. Then I move on to checking change discipline: no unnecessary large tests, no new error classes introduced, and the test files' use of runPromise/runSync are acceptable within test boundaries. I also note the usageLimitResumeFibers is just local mutable state within a closure, similar to other acceptable patterns like handledTurnStartKeys, so nothing concerning there either. I do spot a minor leak — replaceUsageLimitResumeSchedule never cleans up the fiber map entry when a fiber completes, only cancelUsageLimitResumeSchedule removes it, but that's a minor bug rather than a convention violation. Checking the pre-existing shape types like ProviderCommandReactorShape and OpenCodeAdapterShape, these predate the PR and aren't touched, so they're out of scope even though they violate the inline-interface convention. Then I look at the new pure helper functions added to ClaudeAdapter for API error handling, which seem fine. Now I'm scanning the CursorAdapter's import of providerUsageLimitFromError and its new event-draining call — this is a behavioral change, not a convention issue. I also check import ordering in ProviderRuntimeIngestion.ts and the contracts orchestration schema, neither of which raises convention concerns. I look for any newly added Schema.is predicates or catchTag/catchIf usage across the diff, but find none — only Effect.catchCause appears, which isn't a flagged convention pattern here. The interrupt handling logic in processUsageLimitResumeAttempted checks whether the failure cause is interrupts-only before deciding to fail or continue, which seems consistent. I'm now weighing whether the infinite retry schedule in retryUsageLimitResumeDispatch aligns with the stated conventions for startup reconciliation processes. The startup path itself uses a bounded retry of one attempt, matching the pattern in serverRuntimeStartup.ts, and the resumeAt-based scheduling path doesn't retry since it can't fail meaningfully—it just forks. I'm now checking whether the unbounded exponential retry in retryUsageLimitResumeDispatch could be problematic given the reactor's scope. If the reactor shuts down, the retrying fiber gets interrupted and terminates cleanly, but a persistent failure like an invariant error would loop forever, logging a warning roughly every 30 seconds. That's specifically concerning here: if a thread gets archived while a usage-limit-resume timer is still pending, the timer fires, dispatch fails with an OrchestrationCommandInvariantError from the decider's archived-thread check, and the retry logic spins indefinitely at that 30-second cap—producing an infinite warning loop rather than a clean, isolated failure. The test confirming archived threads fail with an invariant error validates that this retry loop is a genuine defect. Now I'm weighing whether this actually falls within the scope of the "Effect Service Conventions" guidance, particularly the bullet about preserving isolation when startup reconciliation repairs multiple independent entities.

5. git_tools — thread.archive

💭 Archiving doesn't cancel the pending resume timer, and unarchiving re-arms it, so archived threads still carry the durable schedule. Since the reactor's in-memory timer has no handler for thread.archived, it still fires while archived, triggering thread.usage-limit-resume.attempt, which fails requireThreadNotArchived with an OrchestrationCommandInvariantError — and that causes retryUsageLimitResumeDispatch to retry indefinitely on an unbounded exponential backoff capped at 30s. This is a real bug: the retry-and-isolate guidance calls for bounding retries and isolating persistent failures, but here it never isolates this deterministic domain error. I want to suggest bounding the retry (e.g. Schedule.recurs intersected with the backoff, or a while guard on transient errors) and giving up gracefully on persistent ones. Before finalizing, I should verify Schedule.modifyDelay is valid in this Effect version by checking how Schedule is used elsewhere in the codebase.

6. git_tools — Schedule.(exponential|recurs|modifyDelay|intersect)

💭 The repo's own precedent for this schedule pattern bounds the retry duration and excludes non-retryable errors, but the new code copies the exponential backoff without either safeguard—that's a solid, concrete finding tied to existing convention. I'll leave an inline comment on ProviderCommandReactor.ts around lines 1257-1263 suggesting the retry be bounded with a duration cap, matching the established pattern. Adding a Schedule.upTo bound... Though after retries give up, I'm considering whether the resulting error properly propagates and gets isolated in the forked fiber without becoming an unhandled failure — it's acceptable but worth noting. I'll keep my suggestion focused on just adding the upper bound to the schedule, then move on to checking the web ChatView changes, which aren't related to the Effect service. Scanning the other DateTime usages, they're pure helpers with default parameters, so nothing to flag there. I'll settle on this single finding — given the existing convention in server.ts using upTo with a while loop, the explicit guidance about isolating persistent failures, and a demonstrable infinite-loop scenario, I'm confident enough to mark this a failure state and post the review.

7. modify_pr — Reviewed the new usage-limit resume services against the Effect service conventions. One finding: the shared usage-limit dispatch retry is unbounded and does not isolate persistent failures.


Agent Credits: 120 credits

Token usage: 12 uncached input · 484,026 cache-read input · 107,243 cache-write input · 9,010 output
Agent Credits may also include non-token charges from external tools such as web research.