Skip to content

feat(status): surface task run pressure#57350

Merged
vincentkoc merged 3 commits intomainfrom
feat/task-run-followups
Mar 30, 2026
Merged

feat(status): surface task run pressure#57350
vincentkoc merged 3 commits intomainfrom
feat/task-run-followups

Conversation

@vincentkoc
Copy link
Copy Markdown
Member

Summary

AI-assisted: yes. Tested with scoped task/status tests plus pnpm build.

  • Problem: the shared task-run ledger exists, but openclaw status and gateway restart deferral still mostly ignore it.
  • Why it matters: operators cannot see queued or running task pressure in the main status surface, and the gateway can decide it is idle while shared task runs are still active.
  • What changed: added a pure task summary helper, surfaced task pressure in status --json, status overview text, and openclaw tasks, and included active task runs in gateway restart deferral counts.
  • What did NOT change (scope boundary): no task schema expansion, no persistence backend work, no hook/event bus work, no richer workflow orchestration.

Change Type (select all)

  • Feature
  • Refactor required for the fix
  • Bug fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

  • Closes #
  • Related #
  • This PR fixes a bug or regression

Root Cause / Regression History (if applicable)

  • Root cause: N/A
  • Missing detection / guardrail: N/A
  • Prior context (git blame, prior PR, issue, or refactor if known): follow-up to the shared task-run registry merge
  • Why this regressed now: N/A
  • If unknown, what was ruled out: N/A

Regression Test Plan (if applicable)

N/A

User-visible / Behavior Changes

  • openclaw status --json now includes task-run summary counts.
  • openclaw status overview now shows task pressure: queued, running, issues, tracked total.
  • openclaw tasks prints a compact task-pressure summary line before the table.
  • Gateway restart deferral now waits for active shared task runs too, not just command queue, pending replies, and embedded runs.

Diagram (if applicable)

Before:
[shared task runs active] -> [status mostly silent]
[gateway reload] -> [idle check ignores task runs] -> [restart can proceed]

After:
[shared task runs active] -> [status shows queued/running pressure]
[gateway reload] -> [idle check counts active task runs] -> [restart waits]

Security Impact (required)

  • New permissions/capabilities? (Yes/No) No
  • Secrets/tokens handling changed? (Yes/No) No
  • New/changed network calls? (Yes/No) No
  • Command/tool execution surface changed? (Yes/No) No
  • Data access scope changed? (Yes/No) No
  • If any Yes, explain risk + mitigation:

Repro + Verification

Environment

  • OS: macOS
  • Runtime/container: local Node 24.13.0 / pnpm 10.32.1
  • Model/provider: N/A
  • Integration/channel (if any): N/A
  • Relevant config (redacted): default local dev config

Steps

  1. Create or inspect shared task runs from task registry producers.
  2. Run openclaw status or openclaw status --json.
  3. Trigger a gateway restart path while shared task runs are active.

Expected

  • Status surfaces show queued/running task pressure.
  • Restart deferral waits until active task runs settle.

Actual

  • This PR implements the expected behavior.

Evidence

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

Human Verification (required)

  • Verified scenarios: scoped task/status test suite and full pnpm build
  • Edge cases checked: cold-start status --json shape, redacted status summary shape, task summary counting, task CLI summary text
  • What you did not verify: repo-wide pnpm check is still red on current main because of unrelated preexisting failures in src/agents/cli-runner.test-support.ts and src/config/io.ts

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

  • Backward compatible? (Yes/No) Yes
  • Config/env changes? (Yes/No) No
  • Migration needed? (Yes/No) No
  • If yes, exact upgrade steps:

Risks and Mitigations

  • Risk: status JSON gains a new tasks object and downstream consumers may assume a fixed shape.
    • Mitigation: additive only; existing fields are unchanged.
  • Risk: restart deferral could wait longer if stale active task records linger.
    • Mitigation: operator-facing counts use inspectable task summaries, so stale active runs can project to lost after the existing grace period.

@vincentkoc vincentkoc self-assigned this Mar 29, 2026
@openclaw-barnacle openclaw-barnacle bot added gateway Gateway runtime commands Command implementations size: M maintainer Maintainer-authored PR labels Mar 29, 2026
@vincentkoc vincentkoc marked this pull request as ready for review March 30, 2026 00:04
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 30, 2026

Greptile Summary

This PR surfaces task-run pressure that was previously invisible to operators: openclaw status and openclaw status --json now include a tasks breakdown (queued, running, failures, total), openclaw tasks prints a compact "Task pressure" summary line, and the gateway's restart deferral logic now also waits for active shared task runs before proceeding.

Key changes:

  • New src/tasks/task-registry.summary.ts provides a clean, single-pass summarizeTaskRecords helper and createEmptyTaskRegistrySummary zero-value factory.
  • TaskRegistrySummary, TaskStatusCounts, TaskRuntimeCounts types are added in task-registry.types.ts; using Record<TaskStatus, …> / Record<TaskRuntime, …> ensures compile-time exhaustiveness.
  • getInspectableTaskRegistrySummary() (maintenance module) and getTaskRegistrySummary() (registry module) are correctly separated: the reconciled inspectable variant is used wherever operator-visible state is needed (status, restart deferral), and the raw variant is used in tests.
  • The fix to the status.summary.test.ts mock path (../config/config.js../config/io.js) corrects a pre-existing test setup bug.
  • formatTaskListSummary in tasks.ts re-implements the same status-bucketing logic already present in summarizeTaskRecords — a minor duplication opportunity (see inline comment).
  • In the restart-deferral polling path, getInspectableTaskRegistrySummary() may perform per-task disk I/O (session store reads for subagent/cli tasks); using it here is semantically correct (dead-session tasks project to lost and stop blocking restart), but the trade-off is worth noting.

Confidence Score: 5/5

Safe to merge — all changes are additive, existing fields and behavior are untouched, and the new task counts are correctly wired into both the status surface and the gateway restart deferral.

No P0 or P1 issues found. The two findings are both P2: a minor code duplication in formatTaskListSummary vs summarizeTaskRecords, and a note on per-task disk I/O inside the restart-deferral polling loop (semantically correct trade-off, not a bug). Test coverage spans the new summary logic, the status JSON shape, and the tasks CLI output.

src/commands/tasks.ts has a minor duplication in formatTaskListSummary; src/gateway/server-reload-handlers.ts has per-task I/O in the deferral polling path (noted but not blocking).

Important Files Changed

Filename Overview
src/tasks/task-registry.summary.ts New pure helper module; summarizeTaskRecords correctly counts totals, active vs terminal split, failures, and per-status/per-runtime breakdowns in a single pass. No issues found.
src/tasks/task-registry.types.ts Adds TaskStatusCounts, TaskRuntimeCounts, and TaskRegistrySummary types using Record<TaskStatus, number> / Record<TaskRuntime, number> for compile-time exhaustiveness.
src/tasks/task-registry.maintenance.ts Adds getInspectableTaskRegistrySummary using the existing reconcileInspectableTasks path, correctly delegating to summarizeTaskRecords.
src/tasks/task-registry.ts Adds getTaskRegistrySummary that reads directly from the in-memory map without reconciliation. Correctly calls ensureTaskRegistryReady(). Used in tests.
src/commands/tasks.ts Adds formatTaskListSummary and a 'Task pressure' log line. The function duplicates status-bucketing logic already present in summarizeTaskRecords (see inline comment).
src/commands/status.command.ts Adds a 'Tasks' row to the status overview table. Formatting is clear and consistent with other multi-part values. Shows 'none' when total is 0.
src/commands/status.types.ts Adds tasks: TaskRegistrySummary to StatusSummary. Additive change; existing fields are untouched.
src/commands/status.summary.ts Follows the established lazy-import pattern for the new task-registry maintenance module. Task summary is fetched via the reconciled path and included in the returned summary object.
src/gateway/server-reload-handlers.ts Active task count correctly added to getActiveCounts and formatActiveDetails. Deferral correctly waits for task runs. Using reconciled summary means tasks with dead backing sessions don't unnecessarily block restart.
src/gateway/server.impl.ts SIGUSR1 restart deferral check now includes active task runs. Consistent with the reload-handlers path.
src/commands/status.summary.test.ts Mock corrected from ../config/config.js to ../config/io.js (fixing a pre-existing test bug). loadStatusSummaryForTest with vi.resetModules() correctly resets module-level lazy singletons between tests.
src/tasks/task-registry.test.ts New test exercises the summary with queued/running/timed_out tasks across acp/cron/subagent runtimes, asserting all counter fields including byStatus and byRuntime breakdowns.

Comments Outside Diff (1)

  1. src/gateway/server-reload-handlers.ts, line 167-178 (link)

    P2 getInspectableTaskRegistrySummary performs disk I/O inside the deferral poll loop

    getInspectableTaskRegistrySummary() calls reconcileInspectableTasks(), which in turn calls hasBackingSession() for every active task that has a childSessionKey. For subagent/cli runtimes that means loading the session store from disk; for acp it reads ACP session metadata. This function is invoked on every tick of the deferGatewayRestartUntilIdle polling loop.

    Semantically, using the inspectable (reconciled) version is the right choice here — it lets the deferral end immediately once tasks whose backing sessions are already gone project to lost, rather than waiting up to the full 5-minute grace period. But it's worth being aware that at high active-task counts each poll cycle carries per-task file reads.

    No action required unless this proves problematic in practice; just worth documenting the trade-off or adding a note near this call site.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: src/gateway/server-reload-handlers.ts
    Line: 167-178
    
    Comment:
    **`getInspectableTaskRegistrySummary` performs disk I/O inside the deferral poll loop**
    
    `getInspectableTaskRegistrySummary()` calls `reconcileInspectableTasks()`, which in turn calls `hasBackingSession()` for every active task that has a `childSessionKey`. For `subagent`/`cli` runtimes that means loading the session store from disk; for `acp` it reads ACP session metadata. This function is invoked on every tick of the `deferGatewayRestartUntilIdle` polling loop.
    
    Semantically, using the inspectable (reconciled) version is the right choice here — it lets the deferral end immediately once tasks whose backing sessions are already gone project to `lost`, rather than waiting up to the full 5-minute grace period. But it's worth being aware that at high active-task counts each poll cycle carries per-task file reads.
    
    No action required unless this proves problematic in practice; just worth documenting the trade-off or adding a note near this call site.
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/commands/tasks.ts
Line: 86-93

Comment:
**`formatTaskListSummary` duplicates `summarizeTaskRecords` logic**

The three separate `.filter()` passes here replicate the exact same status-bucketing logic already present in `summarizeTaskRecords` (added in `task-registry.summary.ts`). The `reconcileInspectableTasks()` call on the line above already returns the reconciled records — you could call `summarizeTaskRecords(tasks)` and format from its result instead of re-iterating the array three times.

```suggestion
function formatTaskListSummary(tasks: TaskRecord[]) {
  const summary = summarizeTaskRecords(tasks);
  return `${summary.byStatus.queued} queued · ${summary.byStatus.running} running · ${summary.failures} issues`;
}
```

(Requires importing `summarizeTaskRecords` from `../tasks/task-registry.summary.js`.)

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/gateway/server-reload-handlers.ts
Line: 167-178

Comment:
**`getInspectableTaskRegistrySummary` performs disk I/O inside the deferral poll loop**

`getInspectableTaskRegistrySummary()` calls `reconcileInspectableTasks()`, which in turn calls `hasBackingSession()` for every active task that has a `childSessionKey`. For `subagent`/`cli` runtimes that means loading the session store from disk; for `acp` it reads ACP session metadata. This function is invoked on every tick of the `deferGatewayRestartUntilIdle` polling loop.

Semantically, using the inspectable (reconciled) version is the right choice here — it lets the deferral end immediately once tasks whose backing sessions are already gone project to `lost`, rather than waiting up to the full 5-minute grace period. But it's worth being aware that at high active-task counts each poll cycle carries per-task file reads.

No action required unless this proves problematic in practice; just worth documenting the trade-off or adding a note near this call site.

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "Merge branch 'main' into feat/task-run-f..." | Re-trigger Greptile

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
@vincentkoc vincentkoc merged commit e6445c2 into main Mar 30, 2026
7 checks passed
@vincentkoc vincentkoc deleted the feat/task-run-followups branch March 30, 2026 00:09
@aisle-research-bot
Copy link
Copy Markdown

aisle-research-bot bot commented Mar 30, 2026

🔒 Aisle Security Analysis

We found 3 potential security issue(s) in this PR:

# Severity Title
1 🟡 Medium Task registry operational metadata exposed to non-admin clients via gateway status method
2 🟡 Medium Prototype pollution / DoS via unvalidated task.status and task.runtime used as object keys in summarizeTaskRecords
3 🟡 Medium Potential DoS: restart deferral polling performs expensive full task reconciliation and disk I/O
Vulnerabilities

1. 🟡 Task registry operational metadata exposed to non-admin clients via gateway status method

Property Value
Severity Medium
CWE CWE-200
Location src/commands/status.summary.ts:85-103

Description

The gateway status request returns getStatusSummary({ includeSensitive: scopes.includes("operator.admin") }), intending to hide sensitive fields from non-admin clients. However, getStatusSummary() now always includes a tasks summary and redactSensitiveStatusSummary() does not remove or coarsen it.

As a result, any client authorized for the status method (e.g., operator.read) can learn internal operational metadata such as:

  • total number of tracked tasks
  • counts of queued/running tasks
  • counts of failures/timeouts/lost tasks
  • runtime breakdown (acp/cron/subagent/cli)

This can leak service health and activity patterns to parties that are intentionally not granted admin visibility.

Vulnerable code paths:

  • status handler only gates sensitive view by admin scope, but still returns tasks
  • redactSensitiveStatusSummary() does not redact tasks

Vulnerable code:

// src/gateway/server-methods/health.ts
const status = await getStatusSummary({
  includeSensitive: scopes.includes(ADMIN_SCOPE),
});
respond(true, status, undefined);
// src/commands/status.summary.ts
const tasks = (await loadTaskRegistryMaintenanceModule()).getInspectableTaskRegistrySummary();
...
return includeSensitive ? summary : redactSensitiveStatusSummary(summary);
// src/commands/status.summary.ts
export function redactSensitiveStatusSummary(summary: StatusSummary): StatusSummary {
  return {
    ...summary,
    sessions: { ... },// tasks is not removed/redacted
  };
}

Recommendation

Decide whether tasks is admin-only data.

If it should be admin-only, redact it when includeSensitive is false:

// src/commands/status.summary.ts
import { createEmptyTaskRegistrySummary } from "../tasks/task-registry.summary.js";

export function redactSensitiveStatusSummary(summary: StatusSummary): StatusSummary {
  return {
    ...summary,
    tasks: createEmptyTaskRegistrySummary(), // or omit tasks entirely
    sessions: {
      ...summary.sessions,
      paths: [],
      defaults: { model: null, contextTokens: null },
      recent: [],
      byAgent: summary.sessions.byAgent.map((entry) => ({ ...entry, path: "[redacted]", recent: [] })),
    },
  };
}

Alternatively, gate inclusion at the source:

const tasks = includeSensitive
  ? (await loadTaskRegistryMaintenanceModule()).getInspectableTaskRegistrySummary()
  : createEmptyTaskRegistrySummary();

Add a test asserting tasks is redacted/empty for non-admin callers of the gateway status method.


2. 🟡 Prototype pollution / DoS via unvalidated task.status and task.runtime used as object keys in summarizeTaskRecords

Property Value
Severity Medium
CWE CWE-1321
Location src/tasks/task-registry.summary.ts:40-46

Description

summarizeTaskRecords() aggregates counts by writing to plain JavaScript objects using dynamic keys derived from TaskRecord fields:

  • summary.byStatus[task.status] += 1
  • summary.byRuntime[task.runtime] += 1

Although TypeScript types restrict status/runtime, persisted task records are loaded from JSON without runtime validation. loadTaskRegistrySnapshotFromJson() accepts arbitrary objects from runs.json and restores them directly into the in-memory registry (restoreTaskRegistryOnce()), meaning a tampered/corrupted state file can inject unexpected strings like "__proto__", "constructor", or "toString" into task.status / task.runtime.

Impact:

  • Availability (DoS): += 1 on inherited/accessor properties (e.g., __proto__) can yield unexpected coercions and may throw in some environments or break later logic by overwriting methods (e.g., toString).
  • Prototype pollution class risk: assigning to __proto__ can invoke the special setter, potentially mutating the prototype chain (depending on the value type and engine semantics), which can cause unpredictable behavior.

Vulnerable code:

summary.byStatus[task.status] += 1;
summary.byRuntime[task.runtime] += 1;

Because task records are restored from disk with minimal checks (only taskId is validated), this is reachable when an attacker (or another local process) can modify the state directory (OPENCLAW_STATE_DIR/.../tasks/runs.json).

Recommendation

Defend in depth by (1) validating/normalizing restored records and/or (2) making the aggregation maps safe against special keys.

Option A (preferred): Validate on restore (or before summarizing)

  • Accept only known TaskStatus/TaskRuntime values; otherwise coerce to a safe default or skip.
const STATUS: ReadonlySet<string> = new Set([
  "queued","running","succeeded","failed","timed_out","cancelled","lost",
]);
const RUNTIME: ReadonlySet<string> = new Set(["subagent","acp","cli","cron"]);

function safeStatus(value: unknown): TaskStatus {
  return typeof value === "string" && STATUS.has(value) ? (value as TaskStatus) : "queued";
}
function safeRuntime(value: unknown): TaskRuntime {
  return typeof value === "string" && RUNTIME.has(value) ? (value as TaskRuntime) : "cli";
}// when restoring or summarizing:
const status = safeStatus(task.status);
const runtime = safeRuntime(task.runtime);
summary.byStatus[status] += 1;
summary.byRuntime[runtime] += 1;

Option B: Use null-prototype objects or Map

const byStatus = Object.create(null) as Record<string, number>;// initialize known keys to 0, then increment

Also consider tightening loadTaskRegistrySnapshotFromJson() to validate required fields (including status and runtime) instead of restoring arbitrary objects.


3. 🟡 Potential DoS: restart deferral polling performs expensive full task reconciliation and disk I/O

Property Value
Severity Medium
CWE CWE-400
Location src/tasks/task-registry.maintenance.ts:123-130

Description

getInspectableTaskRegistrySummary() summarizes tasks by calling reconcileInspectableTasks(), which:

  • Calls listTaskRecords() (builds an array of all tasks and sorts by createdAt => O(n log n) + allocations)
  • Maps every task through reconcileTaskRecordForOperatorInspection()
  • For some tasks, reconciliation can perform filesystem reads via loadSessionStore() / readAcpSessionEntry() when determining whether to mark tasks as lost

This expensive summary is now invoked inside restart deferral checks that can be polled every 500ms (see deferGatewayRestartUntilIdle default) via setPreRestartDeferralCheck / scheduleGatewaySigusr1Restart. If the task registry grows large, or if many tasks trigger hasBackingSession() checks, a restart deferral window can cause repeated heavy CPU + disk I/O, potentially allowing resource exhaustion (DoS) by repeatedly triggering restarts while many tasks exist.

Vulnerable code:

export function getInspectableTaskRegistrySummary(): TaskRegistrySummary {
  return summarizeTaskRecords(reconcileInspectableTasks());
}

and used in a hot polling path:

setPreRestartDeferralCheck(() => ... + getInspectableTaskRegistrySummary().active)

and deferGatewayRestartUntilIdle polls getPendingCount() every 500ms by default.

Recommendation

Make restart/health checks use a lightweight active-task counter that does not sort/reconcile every task and never performs disk I/O.

Suggested changes:

  1. Add an O(1) / O(n) in-memory counter (no sorting) for active tasks, e.g. by iterating tasks.values() without listTaskRecords() sorting, and do not call hasBackingSession() in this path.
  2. Cache/throttle getInspectableTaskRegistrySummary() (e.g., memoize for 1–5s) if it must be used in frequently-called paths.
  3. Keep the expensive reconciliation (that may touch disk) for explicit operator/diagnostic commands only.

Example (lightweight active count):

// task-registry.ts
export function countActiveTasks(): number {
  ensureTaskRegistryReady();
  let active = 0;
  for (const task of tasks.values()) {
    if (task.status === "queued" || task.status === "running") active++;
  }
  return active;
}// server.impl.ts
setPreRestartDeferralCheck(() =>
  getTotalQueueSize() +
  getTotalPendingReplies() +
  getActiveEmbeddedRunCount() +
  countActiveTasks(),
);

Analyzed PR: #57350 at commit 61c6ab9

Last updated on: 2026-03-30T09:58:38Z

pritchie pushed a commit to pritchie/openclaw that referenced this pull request Mar 30, 2026
* feat(status): surface task run pressure

* Update src/commands/tasks.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
alexjiang1 pushed a commit to alexjiang1/openclaw that referenced this pull request Mar 31, 2026
* feat(status): surface task run pressure

* Update src/commands/tasks.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
pgondhi987 pushed a commit to pgondhi987/openclaw that referenced this pull request Mar 31, 2026
* feat(status): surface task run pressure

* Update src/commands/tasks.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

commands Command implementations gateway Gateway runtime maintainer Maintainer-authored PR size: M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant