refactor(qa): add shared QA channel contract and harden worker startup#64562
Conversation
🔒 Aisle Security AnalysisWe found 3 potential security issue(s) in this PR:
1. 🟠 Symlink TOCTOU allows arbitrary directory clearing outside repo in QA artifact preservation
Description
Because the filesystem can change between the validation and use (TOCTOU), an attacker with write access to the repo filesystem can:
Vulnerable code: const preserveToDir = params.repoRoot
? await ensureRepoBoundDirectory(params.repoRoot, params.preserveToDir, "QA gateway artifact directory", { mode: 0o700 })
: params.preserveToDir;
await fs.mkdir(preserveToDir, { recursive: true, mode: 0o700 });
await clearQaGatewayArtifactDir(preserveToDir);While this primarily affects local/CI threat models, it is still an arbitrary delete/write primitive if untrusted code can manipulate the workspace concurrently (e.g., malicious PR code, compromised dependency running tests, or a multi-tenant runner). RecommendationAvoid validating a path and then later using it by name for destructive operations. Mitigations (prefer combining multiple):
import { constants as FS_CONST } from "node:fs";
async function assertSafeDir(dir: string, repoRoot: string) {
// Ensure dir itself is not a symlink
const st = await fs.lstat(dir);
if (st.isSymbolicLink()) throw new Error("preserveToDir must not be a symlink");
// Ensure realpath is within repo at time-of-use
const repoReal = await fs.realpath(repoRoot);
const dirReal = await fs.realpath(dir);
const rel = path.relative(repoReal, dirReal);
if (rel.startsWith("..") || path.isAbsolute(rel)) throw new Error("preserveToDir must stay within repo");
}
await assertSafeDir(preserveToDir, params.repoRoot);
await clearQaGatewayArtifactDir(preserveToDir);
2. 🟡 Channel stop timeout leaves channel task running and unmanaged after abort
DescriptionIn Impact:
Vulnerable code: const stoppedCleanly = await waitForChannelStopGracefully(task, CHANNEL_STOP_ABORT_TIMEOUT_MS);
if (!stoppedCleanly) {
setRuntime(channelId, id, { running: true, lastError: `channel stop timed out...` });
return; // leaves store.aborts/store.tasks entries intact
}
store.aborts.delete(id);
store.tasks.delete(id);This is a security-relevant availability/integrity issue when “stop” is expected to prevent further processing (e.g., disabling an integration) and/or when shutdown expects tasks to terminate. RecommendationEnsure stop operations leave the system in a consistent and enforceable state even when graceful shutdown times out. Recommended fixes (choose one based on desired semantics):
Example (bookkeeping cleanup + explicit stopping state): if (!stoppedCleanly) {
store.aborts.delete(id);
store.tasks.delete(id);
setRuntime(channelId, id, {
accountId: id,
running: false,
restartPending: false,
lastError: `channel stop timed out after ${CHANNEL_STOP_ABORT_TIMEOUT_MS}ms`,
// optionally add: stoppingFailed: true
});
return;
}If the task can still run after cleanup, additionally attach a 3. 🟡 Sensitive data exposure via unredacted failure reply text in QA suite reports
DescriptionThe QA suite transport wait helper throws an This can leak sensitive information into test artifacts (e.g.,
Vulnerable code: throw new Error(extractQaFailureReplyText(failureMessage.text) ?? failureMessage.text);Even though a log redaction helper exists ( RecommendationRedact or avoid persisting raw message text in thrown errors and reports. Options:
import { redactQaGatewayDebugText } from "./gateway-log-redaction.js";
const failureText = extractQaFailureReplyText(failureMessage.text) ?? failureMessage.text;
throw new Error(redactQaGatewayDebugText(failureText));
Analyzed PR: #64562 at commit Last updated on: 2026-04-12T17:46:49Z |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d1780efb25
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Greptile SummaryThis PR introduces a shared All 37 existing QA scenarios and 63 unit tests pass. Confidence Score: 5/5Safe to merge; all findings are P2 style/hardening suggestions that do not affect correctness under normal operation. The refactor is well-structured, boundary-clean, and thoroughly validated (63 unit tests + 37 E2E scenarios pass). The two flagged issues are both in unreachable or edge-case paths: the extra config build with a throwaway port is a code-smell but idempotent in practice, and the rpcClient cleanup gap in the outer catch only matters if the post-loop defensive check throws, which cannot happen when the loop exits via break. All remaining feedback is P2. extensions/qa-lab/src/gateway-child.ts — two minor cleanup/style items in the retry loop and outer catch.
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2215302eb4
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 741041bd0b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
741041b to
48e4f65
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 48e4f656dc
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ffb37830da
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2d1cd92fc5
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 77b0a3e876
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f9687012ae
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b13234c7fa
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
979d2f0 to
a23e259
Compare
|
Follow-up on the updated Aisle summary: Addressed in
Reviewed and not taking in this PR:
So, for the current Aisle summary:
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a23e259fad
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
openclaw#64562) * refactor(qa): add shared transport contract and suite migration * refactor(qa): harden worker gateway startup * fix(qa): scope waits and sanitize shutdown artifacts * fix(qa): confine artifacts and redact preserved logs * fix(qa): block symlink escapes in artifact paths * fix(gateway): clear shutdown race timers * fix(qa): harden shutdown cleanup paths * fix(qa): sanitize gateway logs in thrown errors * fix(qa): harden suite startup and artifact paths * fix(qa): stage bundled plugins from mutated config * fix(qa): broaden gateway log bearer redaction * fix(qa-channel): restore runtime export * fix(qa): stop failed gateway startups as a process tree * fix(qa-channel): load runtime hook from api surface
openclaw#64562) * refactor(qa): add shared transport contract and suite migration * refactor(qa): harden worker gateway startup * fix(qa): scope waits and sanitize shutdown artifacts * fix(qa): confine artifacts and redact preserved logs * fix(qa): block symlink escapes in artifact paths * fix(gateway): clear shutdown race timers * fix(qa): harden shutdown cleanup paths * fix(qa): sanitize gateway logs in thrown errors * fix(qa): harden suite startup and artifact paths * fix(qa): stage bundled plugins from mutated config * fix(qa): broaden gateway log bearer redaction * fix(qa-channel): restore runtime export * fix(qa): stop failed gateway startups as a process tree * fix(qa-channel): load runtime hook from api surface
openclaw#64562) * refactor(qa): add shared transport contract and suite migration * refactor(qa): harden worker gateway startup * fix(qa): scope waits and sanitize shutdown artifacts * fix(qa): confine artifacts and redact preserved logs * fix(qa): block symlink escapes in artifact paths * fix(gateway): clear shutdown race timers * fix(qa): harden shutdown cleanup paths * fix(qa): sanitize gateway logs in thrown errors * fix(qa): harden suite startup and artifact paths * fix(qa): stage bundled plugins from mutated config * fix(qa): broaden gateway log bearer redaction * fix(qa-channel): restore runtime export * fix(qa): stop failed gateway startups as a process tree * fix(qa-channel): load runtime hook from api surface
openclaw#64562) * refactor(qa): add shared transport contract and suite migration * refactor(qa): harden worker gateway startup * fix(qa): scope waits and sanitize shutdown artifacts * fix(qa): confine artifacts and redact preserved logs * fix(qa): block symlink escapes in artifact paths * fix(gateway): clear shutdown race timers * fix(qa): harden shutdown cleanup paths * fix(qa): sanitize gateway logs in thrown errors * fix(qa): harden suite startup and artifact paths * fix(qa): stage bundled plugins from mutated config * fix(qa): broaden gateway log bearer redaction * fix(qa-channel): restore runtime export * fix(qa): stop failed gateway startups as a process tree * fix(qa-channel): load runtime hook from api surface
openclaw#64562) * refactor(qa): add shared transport contract and suite migration * refactor(qa): harden worker gateway startup * fix(qa): scope waits and sanitize shutdown artifacts * fix(qa): confine artifacts and redact preserved logs * fix(qa): block symlink escapes in artifact paths * fix(gateway): clear shutdown race timers * fix(qa): harden shutdown cleanup paths * fix(qa): sanitize gateway logs in thrown errors * fix(qa): harden suite startup and artifact paths * fix(qa): stage bundled plugins from mutated config * fix(qa): broaden gateway log bearer redaction * fix(qa-channel): restore runtime export * fix(qa): stop failed gateway startups as a process tree * fix(qa-channel): load runtime hook from api surface
openclaw#64562) * refactor(qa): add shared transport contract and suite migration * refactor(qa): harden worker gateway startup * fix(qa): scope waits and sanitize shutdown artifacts * fix(qa): confine artifacts and redact preserved logs * fix(qa): block symlink escapes in artifact paths * fix(gateway): clear shutdown race timers * fix(qa): harden shutdown cleanup paths * fix(qa): sanitize gateway logs in thrown errors * fix(qa): harden suite startup and artifact paths * fix(qa): stage bundled plugins from mutated config * fix(qa): broaden gateway log bearer redaction * fix(qa-channel): restore runtime export * fix(qa): stop failed gateway startups as a process tree * fix(qa-channel): load runtime hook from api surface
Problem
The shared E2E QA runner in
qa-labwas directly coupled toqa-channel.Files like
extensions/qa-lab/src/suite.tsandextensions/qa-lab/src/qa-gateway-config.tswere assumingqa-channelbehavior inline, including:That coupling makes the shared runner own behavior that belongs to a QA channel implementation. If another QA channel implementation is added later, the shared runner would need more transport-specific branches, or a separate runner would need to duplicate orchestration, reporting, worker lifecycle, and scenario execution.
While validating this refactor, the full suite also exposed shared worker lifecycle issues:
What this PR changes
1. Introduces a shared QA channel contract
This PR adds an explicit shared QA channel contract in:
extensions/qa-lab/src/qa-transport.tsextensions/qa-lab/src/qa-transport-registry.tsIt moves the current
qa-channelbehavior into its own implementation in:extensions/qa-lab/src/qa-channel-transport.tsIt rewires the shared suite to depend on that contract instead of directly assuming
qa-channelbehavior in:extensions/qa-lab/src/suite.tsextensions/qa-lab/src/qa-gateway-config.ts2. Extracts common QA channel capabilities
The common QA channel capabilities now live in the shared base and are inherited by implementations.
That shared capability surface covers:
qa-channelnow uses that shared capability base as the first implementation.3. Hardens shared worker startup and restart behavior
This PR also hardens:
extensions/qa-lab/src/gateway-child.tsextensions/qa-lab/src/gateway-rpc-client.tssrc/gateway/server-close.tssrc/gateway/server-channels.tsThe hardening includes:
Why this structure
This PR makes the shared runner own shared orchestration and makes the QA channel implementation own its transport-specific behavior.
After this change:
qa-labowns suite orchestrationqa-channelis an implementation of the shared QA channel contractqa-channelinternalsThat gives the shared QA layer a clean boundary and removes transport-specific behavior from the shared runner.
What did not change
qa-channelscenarios still workqa-whatsapp,qa-telegram, or any other new QA channel yetValidation
This refactor was validated against the current shared QA system.
Focused tests
Ran:
pnpm test extensions/qa-lab/src/qa-channel-transport.test.ts extensions/qa-lab/src/qa-gateway-config.test.ts extensions/qa-lab/src/gateway-child.test.ts extensions/qa-lab/src/gateway-rpc-client.test.ts extensions/qa-lab/src/suite.test.ts extensions/qa-lab/src/live-transports/shared/live-gateway.runtime.test.tsResult:
63/63tests passedBuild
Ran:
pnpm buildResult:
Full existing QA suite
Ran:
pnpm openclaw qa suite --output-dir .artifacts/qa-e2e/transport-hardening-prResult:
37/37scenarios passed