Skip to content

OIC durable job worker: retries, idempotent commits, restart recovery #226

Open

Description

@exactml

Context

Investigations run in the background on a remote host. Queued and running work must survive restarts, and retries must never write the same claim twice. Depends on #225.

Behavior to implement

  • A database-backed job table (per Design: Organizational Intelligence Core module boundary, stack, and packaging #219): an investigation is persisted before dispatch, workers claim jobs with a lease, and expired leases are recovered on startup.
  • Retry policy with backoff. A failed job records its reason.
  • Idempotency keys on canonical commits, so a retried job that already committed its proposals does nothing the second time.
  • Limits: concurrent agent runs, per-investigation timeout, one agent per investigation, cancel and pause.

Acceptance criteria

Tests:

  • Worker restart while a job is queued, and while one is running.
  • Duplicate job delivery produces one set of canonical writes.
  • Cancellation stops the run and records it.
  • A timeout marks the run failed and keeps the checkpoint.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    backendbackend devfeatureoicOrganizational Intelligence Core

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions