Skip to content

test(orchestration): provide replay recovery dependencies

MacroscopeApp / Macroscope - Effect Service Conventions failed Aug 30, 2026 in 3m 19s

Effect Service Conventions: 1 issue found

apps/server/src/orchestration-v2/Orchestrator.ts (recoverDeferredOrganization, lines 7245-7292) — startup reconciliation over independent threads swallows interruption via per-thread Effect.catchCause and performs no transient retry before readiness. A shutdown mid-recovery is logged as a warning and the sweep continues, and a one-off projection/dispatch failure drops that thread's intent repair until the next restart. Suggested fix posted inline: retry briefly, then re-fail interrupt-only causes before falling back to the warning, plus a retry-success test alongside the existing persistent-failure continuation coverage.

The per-thread failure isolation flagged in earlier runs has been addressed in this revision.

Details

Note

Your check run agent prompt is: .macroscope/check-run-agents/effect-service-conventions.md
More information about how Check Run Agents work can be found in our Help Center.

Reviewed against the Effect service conventions:

  • Imports/namespaces — new import * as Orchestrator from "./orchestration-v2/Orchestrator.ts" in serverRuntimeStartup.ts uses the module namespace at the service boundary; contracts remain named imports. No violations.
  • Service definition — OrchestratorV2Shape / OrchestratorMcpServiceShape are pre-existing standalone shapes; this PR only appends a member, so no new violation is introduced.
  • Dependency acquisition — runV2RecoveryPhase accepts two Effect values (composition, not service-instance injection); no new ManagedRuntime.make, runPromise, or runtime-backed Layer.succeed. layerUnavailable follows the existing failing-stub pattern.
  • Errors — new failures reuse OrchestratorDispatchError / OrchestratorProjectionError (Schema.TaggedErrorClass with attribute-derived messages); ThreadId.make("thread:shell") for shell-snapshot failures matches existing usage. MCP-side failure(...) construction matches the surrounding handlers.
  • Contracts/schemas — OrchestrationV2DeferredOrganization, command variants, and the MCP input/result schemas follow existing structure and JSON-variant conventions.
  • Startup reconciliation — the one finding above.

Files examined: apps/server/src/orchestration-v2/Orchestrator.ts, apps/server/src/orchestration-v2/ProjectionStore.ts, apps/server/src/serverRuntimeStartup.ts, apps/server/src/mcp/OrchestratorMcpService.ts, apps/server/src/mcp/toolkits/orchestrator/{tools.ts,handlers.ts}, packages/contracts/src/{orchestrationV2.ts,orchestratorMcp.ts}, packages/shared/src/t3McpToolPresentation.ts, plus the added/updated test and testkit files.


🔍 Agent activity — 15 tool calls

💭 I need to check whether the previously flagged issue about startup recovery failure isolation still holds at the current head commit. Looking at the diff again, the error handling with catchCause does appear to be applied per-thread inside the forEach callback rather than globally, which would change my earlier assessment. But the catchCause here swallows interruption causes entirely into a warning log, with no retry logic for transient failures before marking the service ready — that conflicts with the convention of preserving interruption semantics and retrying before isolating persistent failures, so this still looks like a valid remaining concern. Also the new test only covers the persistent-failure continuation path (unreadable thread with corrupted payload_json still lets recovery proceed), not the retry-success path. I want to verify this catchCause placement moved from outer to per-thread in direct response to my earlier comment at 271e82b, which specifically flagged the outer catchCause reducing per-thread failures including interruption to a single warning.

1. git_tools — recoverDeferredOrganization

2. git_tools — Failed to recover deferred thread organization

💭 Neither search turned up results, which seems strange—maybe the results got truncated. I should grep Orchestrator.ts directly instead.

3. git_tools — const recoverDeferredOrganization

💭 Getting no output there, so maybe the grep tool defaults to a different revision than what's in the URL. Instead of fighting with the rev parameter, I'll just view the file region directly.

4. git_tools — recoverDeferredOrganization

5. browse_code — apps/server/src/orchestration-v2/Orchestrator.ts

💭 The per-thread isolation issue is addressed now, but I still need to check whether the code satisfies the reconciliation rule: transient failures should retry before readiness, persistent failures on one entity should be isolated so they don't abort global startup or block repair of other entities, and both paths need test coverage. Looking at the actual catch, Effect.catchCause swallows interruption entirely rather than preserving it, and neighboring loops like resumeQueuedRuns follow the same legacy pattern — so I'm weighing whether this counts as a genuine convention violation worth flagging versus something already covered by my earlier comment. My prior comment focused mainly on per-thread isolation, which is now fixed, so the remaining gaps are: interruption being swallowed instead of preserved, no retry logic before readiness, and no test for the retry-success path.

6. browse_code — apps/server/src/orchestration-v2/Orchestrator.ts

💭 This new code matches the existing recovery pattern used elsewhere in the repo, so flagging the interruption-swallowing here would mean flagging a repo-wide convention rather than something specific to this change, which the guidance discourages. But there's a real tension since the stated convention explicitly calls for preserving interruption and retrying only transient errors, so I'm weighing consistency with the codebase against correctness of the actual rule. I already raised this in a prior comment and the author adjusted the code to match the neighboring pattern, so raising it again with a stricter demand risks feeling repetitive. I'll move on to check other parts of the diff for genuine violations, starting with import ordering in the server runtime startup file (which doesn't map to a stated convention) and the service shape interfaces, which are pre-existing legacy structures that the PR merely extends rather than introduces. Adding a member to an existing shape isn't the same as creating a new standalone shape, so I probably shouldn't flag it. I'm also noticing a non-null assertion on input.action in the MCP service—likely safe given the schema filter, but more of a type-safety nit than an Effect convention issue, so I'll leave it aside. More interesting is a mapError call that constructs an OrchestratorProjectionError using a fabricated thread ID like "thread:shell" for what's actually a shell-snapshot failure—that feels like stuffing a fake entity identifier into structured error context, so I want to check if there's precedent for this pattern elsewhere in the codebase before flagging it. Similarly, in layerUnavailable, the deferred-organization failure path passes a plain string as the cause field of an OrchestratorDispatchError, so I want to compare that against how other members of layerUnavailable construct their errors to see if this is consistent or an outlier. I also want to verify whether the eager Effect.fail construction inside the Layer.succeed matches other patterns in the file, and look closer at how the failure helper is used inside Effect.mapError—since if failure itself returns an Effect rather than a plain error value, wrapping it in mapError could be structurally wrong.

7. git_tools — const failure =

8. git_tools — thread:shell

💭 "thread:shell" already follows an existing convention, that's fine. Now I'm looking at the failure helper in OrchestratorMcpService.

9. git_tools — function failure|failure =

💭 I'm checking whether failure returns a plain value or something yieldable — since OrchestratorMcpFailure likely extends a Schema tagged error class, it may itself be yieldable as a failing Effect, which would explain the yield* failure(...) pattern. Confirming that Data/Schema.TaggedError instances are yieldable, so both yield* failure(...) and Effect.mapError returning the failure value work consistently with the rest of the codebase, including the non-null assertion and mapError-to-failure pattern in the MCP handlers. Now I'm moving on to check the contracts schema additions and the recovery phase logic in serverRuntimeStartup.ts. I'm noting that runV2RecoveryPhase is a small composition helper generic over five type params using Effect.tap, which is a minor stylistic nit but borderline acceptable since it's not purely an error factory. Now I'm looking at the dispatch options and the new dispatchDeferredOrganization in Orchestrator.ts, comparing how it constructs OrchestratorDispatchError with a cause field versus how dispatchWithReceiptEffect uses a detail field, so I need to check the actual error type definition to see if this is inconsistent.

10. git_tools — class OrchestratorDispatchError|class OrchestratorProjectionError

💭 Everything looks consistent with the existing patterns so far. Checking whether the new layerUnavailable entry for the deferred organization recovery case follows the same error-construction convention as the other entries.

11. browse_code — apps/server/src/orchestration-v2/Orchestrator.ts

💭 I'm looking at the startup reconciliation retry/interruption handling again — the convention document calls this out explicitly, and even though the surrounding code follows the same pattern, the instructions say to treat the documented convention as authoritative over existing code. I recall flagging this same loop in a previous review, and the author only partially addressed it by moving the catchCause inline, so I'm weighing whether to raise it again. Given that my earlier suggestion is what produced this exact code, re-flagging it now would be duplicative and unhelpful, even though the interruption concern technically persists. I decide to move on rather than re-raise my own suggestion as a new issue. I go through the remaining changes — the test harness layer composition, the persistence layer merge in the runtime test, the schema filter check in the orchestrator contracts, and the handlers addition — and each looks fine on inspection. Checking the MCP service's non-null assertion, the deferThreadOrganization shape usage, and import ordering in the server runtime startup file, none of these rise to the level of a convention violation worth flagging. I also verify that passing Effects as parameters to the recovery phase function is just value composition rather than a dependency-hiding issue, so that's fine too, and the namespace import pattern for Orchestrator matches the existing convention used for ProviderRuntimeRecovery. With no clear violations remaining, I need to reconcile the instruction to output exactly "All clear" with the requirement that my entire response be valid JSON — I'll resolve this by producing JSON with a success state and "All clear" as the title/summary. I'm also parsing whether "defaults to failure" refers to the check's overall conclusion when violations exist, which seems the most sensible reading given there are no findings here. But I want to double-check the retry convention once more — the PR only tests the persistent-failure path, not a retry path, and since no retry logic exists at all, the rule about retrying a transient per-entity repair before readiness could genuinely be violated here, distinct from anything I've already flagged. I'm weighing whether this is worth raising: a transient DB failure during startup would just leave the intent logged and persisted, relying on the next restart to pick it back up. I'm inclined to post one concise comment covering both retry-before-readiness and interruption preservation, since the code did change in direct response to my earlier isolation comment, making a follow-up fair game. But I'm still second-guessing whether interruption preservation is actually the right call here given how the catch logic handles the cause. Now I'm sketching out the actual fix: adding a retry with exponential backoff before falling into the catch, and checking if the cause is purely an interruption to re-fail rather than log a warning and continue. That code needs Schedule and Cause imports though, which complicates whether this can be a simple inline suggestion versus needing a unified diff — I should check whether those are already imported in Orchestrator.ts before deciding how to post this. I'm also second-guessing whether catchCause actually intercepts interrupts in this version of Effect, which matters for whether the fix even works as intended.

12. browse_code — apps/server/src/orchestration-v2/Orchestrator.ts

💭 I need to note that Cause and Schedule aren't imported, so I'll flag that the fix requires adding imports and present it as a unified diff. I'm drafting a single review comment for Orchestrator.ts around the catchCause pipe, explaining that startup reconciliation across independent threads should retry transient per-thread repairs before readiness and preserve interruption rather than just logging it. Writing the diff...

13. git_tools — Effect.retry(

14. git_tools — isInterrupted|Cause.

💭 Confirming Cause.hasInterruptsOnly is the right idiom here for effect v4. Since the instructions treat touched code as authoritative even though other recovery loops lack the same interrupt handling, I'll post one focused follow-up comment covering both preserving interruption and retrying transient per-entity repairs, along with a suggested test for retry-success, keeping it concise. I'm pinning down the exact diff line for the inline comment—checking that lines 7274-7281 in the head file correspond to the .pipe(Effect.catchCause(...)) block being flagged, and drafting the comment explaining that startup reconciliation should preserve interruption and retry transient failures rather than swallowing interrupt causes and dropping repairs until the next restart. I'm checking whether the retry options object syntax is valid in this version of effect, settling on the times/schedule form as an illustrative fix rather than something auto-applied. I also want to verify that re-failing with the original cause on interrupt preserves the right error type so it still flows correctly through the outer mapError into an OrchestratorProjectionError. Given the instructions explicitly call out preserving interruption instead of downgrading it to a warning, and retrying only transient per-entity repairs, I'm confident this is a clear, deliberate convention stated for this exact PR family — so I'll mark the review as a failure with that single finding and post it.

15. modify_pr — One finding on the new deferred-organization startup recovery loop in apps/server/src/orchestration-v2/Orchestrator.ts. Per-thread isolation is now in place; the remaining gap is interruption handling and transient retry before readiness.


Agent Credits: 129 credits

Token usage: 26 uncached input · 733,921 cache-read input · 84,799 cache-write input · 13,043 output
Agent Credits may also include non-token charges from external tools such as web research.