feat(openai,agents): GPT-5.4 prompt discipline + personality bridge + injection scanning#67512
feat(openai,agents): GPT-5.4 prompt discipline + personality bridge + injection scanning#67512100yenadmin wants to merge 10 commits into
Conversation
…injection scanning Consolidation of v3 PRs #66371, #66372, #66373, #66374, #66375 into a single review unit. Ports the Hermes Agent GPT-5 prompt stack to OpenClaw with Hermes-verbatim semantics (only mechanical tool-ID substitutions for OpenClaw's catalog: terminal→exec, execute_code→ code_execution, read_file→read, search_files→exec). ## Prompt additions (extensions/openai/prompt-overlay.ts) OPENAI_GPT5_TOOL_ENFORCEMENT — Hermes mandatory_tool_use block: "NEVER answer these from memory — ALWAYS use a tool" + 7 categories. OPENAI_GPT5_EXECUTION_BIAS enhanced with 3 Hermes subsections: - Act, Don't Ask (prompt_builder.py:221-229) - Tool Persistence (prompt_builder.py:198-204) - Verification (prompt_builder.py:238-245) ## Architecture (src/agents/system-prompt-contribution.ts) Add "tool_enforcement" to ProviderSystemPromptSectionId union so providers can inject mandatory tool-use guidance independently of execution_bias. GPT-5 overlay uses sectionOverrides.tool_enforcement. ## Security (src/agents/context-file-injection-scan.ts) Port Hermes's _CONTEXT_THREAT_PATTERNS (prompt_builder.py:36-52): 10 threat patterns + 10 invisible chars. Flagged files are BLOCKED (content replaced with placeholder), matching Hermes behavior. ## Tests - index.test.ts: EXPECTED_GPT5_STABLE_PREFIX helper + 7 assertions updated for tool_enforcement sectionOverride - context-file-injection-scan.test.ts: 17 tests covering all patterns, false-positive avoidance, and BLOCK behavior Part of GPT 5.4 Enhancement v3 sprint. Tracking: #66345. Supersedes: #66371, #66372, #66373, #66374, #66375.
openclaw-barnacle
Bot
added
agents
Greptile SummaryThis PR adds a GPT-5.4 prompt overlay ( Confidence Score: 5/5Safe to merge — all findings are P2 (observability gap, not a correctness defect). The change is additive, well-tested (20+ injection-scanner tests plus full overlay coverage), and all prior P1/P0 review concerns were resolved. The one remaining finding is a missing telemetry callback in the production call site — not a bug, just missing observability. src/agents/system-prompt.ts — the sanitizeContextFileForInjection call site lacks an onAllowlistBypass callback for audit logging. Prompt To Fix All With AIThis is a comment left during a code review.
Path: src/agents/system-prompt.ts
Line: 118-121
Comment:
**Silent allowlist-bypass in production call site**
`sanitizeContextFileForInjection` is called without an `onAllowlistBypass` callback. If any allowlisted file (e.g. `SECURITY.md`, `CONTRIBUTING.md`, `docs/security/**`) happens to contain an actual injection pattern, the bypass fires silently — no log, no telemetry, no user-visible signal. The interface was clearly designed to support this observability hook; it's just not wired up here.
```suggestion
const sanitizedContent = sanitizeContextFileForInjection(
sanitizeContextFileContentForPrompt(file.content),
file.path,
{
onAllowlistBypass: (fname, findings) => {
// TODO: wire to telemetry / logger once one is available
console.warn(
`[injection-scan] allowlist bypass: ${fname} contained ${findings.join(", ")}`,
);
},
},
);
```
Without at least a console warning, it's impossible to distinguish "allowlist working as intended" from "allowlist being exploited by a crafted security doc".
How can I resolve this? If you propose a fix, please make it concise.Reviews (2): Last reviewed commit: "fix(agents): #67512 iter-2 — reset state..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4265ba55b3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Pull request overview
Adds Hermes-parity GPT‑5 prompt discipline by introducing a dedicated tool_enforcement prompt section, strengthening GPT‑5 execution guidance, and adding prompt-injection scanning for injected workspace context files.
Changes:
- Add
tool_enforcementas a first-class overridable system-prompt section and wire the OpenAI GPT‑5 overlay to supply it. - Extend GPT‑5 execution bias text with “Act, Don’t Ask”, tool persistence, and a verification checklist.
- Add context-file prompt injection scanning (threat regexes + invisible Unicode) and block flagged context file content.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/agents/system-prompt.ts | Runs context-file injection sanitization during Project Context rendering; renders new tool_enforcement section after execution_bias. |
| src/agents/system-prompt-contribution.ts | Extends ProviderSystemPromptSectionId to include tool_enforcement. |
| src/agents/context-file-injection-scan.ts | New injection scanner + sanitizer that replaces flagged context content with a blocking placeholder. |
| src/agents/context-file-injection-scan.test.ts | Unit tests for injection detection and blocking sanitizer behavior. |
| extensions/openai/prompt-overlay.ts | Adds GPT‑5 tool enforcement block and augments execution-bias guidance; wires tool_enforcement override. |
| extensions/openai/index.test.ts | Updates provider contribution expectations and factors shared GPT‑5 stablePrefix constant. |
There was a problem hiding this comment.
Pull request overview
This PR advances GPT‑5 system-prompt parity with Hermes by adding a dedicated tool-enforcement prompt section and introducing prompt-injection scanning for workspace context files that are loaded into the agent system prompt.
Changes:
- Add
tool_enforcementas a first-class overridable system-prompt section, wired into the prompt build pipeline and OpenAI GPT‑5 overlay. - Port Hermes-style GPT‑5 execution discipline additions (act-don’t-ask, tool persistence, verification checklist) into the OpenAI overlay.
- Add context-file injection scanning that blocks suspicious context-file content with a placeholder, plus unit tests.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
src/agents/system-prompt.ts |
Runs injection sanitization over loaded context files and renders the new tool_enforcement section after execution_bias. |
src/agents/system-prompt-contribution.ts |
Extends ProviderSystemPromptSectionId to include tool_enforcement. |
src/agents/context-file-injection-scan.ts |
Adds a Hermes-ported injection scanner and blocking sanitizer for context file content. |
src/agents/context-file-injection-scan.test.ts |
Adds unit tests covering threat patterns, invisible unicode detection, and blocking behavior. |
extensions/openai/prompt-overlay.ts |
Adds GPT‑5 tool-enforcement block and strengthens execution-bias guidance (act-don’t-ask, persistence, verification). |
extensions/openai/index.test.ts |
Updates expectations for GPT‑5 prompt contributions to include the new tool_enforcement section override. |
There was a problem hiding this comment.
Pull request overview
Adds Hermes-parity prompt guidance for GPT‑5 models by introducing a dedicated tool_enforcement system-prompt section and by scanning injected workspace context files for prompt-injection patterns before including them in the system prompt.
Changes:
- Add
tool_enforcementas a first-class overridable prompt section and wire the OpenAI GPT‑5 overlay to populate it. - Introduce context-file prompt-injection scanning with blocking behavior and unit tests.
- Extend GPT‑5 execution bias guidance (act-don’t-ask, tool persistence, verification) and update provider contribution tests.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/agents/system-prompt.ts | Runs context-file injection sanitization during prompt assembly; inserts the new tool_enforcement section after execution_bias. |
| src/agents/system-prompt-contribution.ts | Extends ProviderSystemPromptSectionId union to include tool_enforcement. |
| src/agents/context-file-injection-scan.ts | New injection scanner + sanitizer that blocks suspicious context content with a placeholder. |
| src/agents/context-file-injection-scan.test.ts | Unit tests for injection detection and blocking placeholder behavior. |
| extensions/openai/prompt-overlay.ts | Adds GPT‑5 tool enforcement section + expands execution bias guidance; wires into sectionOverrides. |
| extensions/openai/index.test.ts | Updates provider prompt-contribution expectations to include tool_enforcement; factors shared stablePrefix expectation. |
Filename is interpolated into the BLOCKED string. A crafted file path containing brackets or newlines could confuse the model about what content is blocked. Strip ], [, and newlines from the filename.
|
Merge order: 1/8. See #66345 for the full merge sequence. |
…bosity Additions to the GPT-5.4 prompt overlay (interaction_style, output_contract, execution_bias sections): - Identity Enforcement: prime GPT-5.4 to adopt SOUL.md as primary identity, ban corporate default patterns - Voice Calibration: counter flat/analytical drift, anti-sycophancy reinforcement - Response Length Discipline: 200-word target with 95% confidence self-evaluation gate leveraging thinking mode, file-offload for genuine long-form - Investigation Discipline: prevent partial-findings-then-ask pattern, force parallel tool calls for independent lookups - Plan Confidence Gate: 95%+ → execute without approval, 80-94% → state uncertainty and begin, <80% → iterate privately through research
…tterns The prompt_injection pattern only matched single-qualifier forms like 'ignore all instructions' but missed 'ignore all previous instructions' (two qualifiers). Changed to allow zero or more qualifier words between 'ignore' and 'instructions'.
|
Codex Review: Didn't find any major issues. Bravo. ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
There was a problem hiding this comment.
Pull request overview
This PR strengthens GPT‑5.4 agent behavior (tool usage discipline, persona fidelity, and brevity) via an OpenAI prompt overlay, and adds a new cross-model context-file prompt-injection scanner that can block unsafe workspace context before it enters the system prompt.
Changes:
- Introduces a new overridable
tool_enforcementsystem-prompt section and wires it into system prompt assembly. - Adds
context-file-injection-scan.ts(+ tests) and applies it when rendering injected project context files. - Extends the OpenAI GPT‑5.4 prompt overlay with additional execution/voice/brevity directives and corresponding test expectation updates.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/agents/system-prompt.ts | Applies injection sanitization to context file contents; adds tool_enforcement section into prompt assembly. |
| src/agents/system-prompt-contribution.ts | Extends provider section override IDs to include tool_enforcement. |
| src/agents/context-file-injection-scan.ts | New scanner + sanitizer for injection patterns, allowlist logic, and filename hardening. |
| src/agents/context-file-injection-scan.test.ts | Unit tests covering threat patterns, allowlist behavior, and filename defense-in-depth. |
| extensions/openai/prompt-overlay.ts | Adds GPT‑5.4 overlay text for identity/voice/brevity + new mandatory tool-use section override. |
| extensions/openai/index.test.ts | Updates contribution assertions and adds content checks for the new overlay sections. |
There was a problem hiding this comment.
Pull request overview
Adds GPT‑5.4-specific prompt overlay sections (tool enforcement, execution discipline, persona/voice tightening, brevity rules) and introduces a context-file prompt-injection scanner that sanitizes injected workspace context before it is incorporated into the agent system prompt.
Changes:
- Add new provider overridable prompt section id
tool_enforcementand wire it into the system prompt build pipeline. - Add
sanitizeContextFileForInjection()+ threat-pattern scanning (with tests) and apply it to injected project context file contents/paths. - Extend OpenAI GPT‑5 overlay with identity/voice calibration, brevity discipline, execution bias expansions, and mandatory tool-use directives (with updated assertions).
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/agents/system-prompt.ts | Runs context-file injection sanitization during project context rendering; sanitizes filename headings; inserts tool_enforcement overridable section. |
| src/agents/system-prompt-contribution.ts | Extends provider section override union with tool_enforcement. |
| src/agents/context-file-injection-scan.ts | New scanner/sanitizer with threat regexes, invisible-unicode detection, and allowlist bypass behavior. |
| src/agents/context-file-injection-scan.test.ts | Unit tests for threat detection, placeholder replacement, allowlist semantics, and path normalization hardening. |
| extensions/openai/prompt-overlay.ts | Adds GPT‑5.4 prompt overlay blocks (identity/voice/brevity/execution discipline) and defines OPENAI_GPT5_TOOL_ENFORCEMENT. |
| extensions/openai/index.test.ts | Updates expected OpenAI provider contributions to include tool_enforcement and expands assertion coverage for new overlay text. |
There was a problem hiding this comment.
Pull request overview
This PR strengthens GPT‑5.4 agent behavior via an OpenAI-only prompt overlay (tool enforcement + execution/voice/brevity discipline) and adds a cross-provider context-file injection scanner that blocks suspicious workspace bootstrap content before it’s injected into the system prompt.
Changes:
- Introduces a new overridable provider prompt section id (
tool_enforcement) and wires it into system prompt assembly. - Adds
sanitizeContextFileForInjection()+ tests, and applies it when rendering injected workspace context files. - Expands the GPT‑5.4 OpenAI prompt overlay with identity/voice calibration, brevity discipline, investigation discipline, plan-confidence gating, and mandatory tool-use categories (plus updated assertions).
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/agents/system-prompt.ts | Applies context-file injection sanitization during project context rendering; adds tool_enforcement section to prompt pipeline. |
| src/agents/system-prompt-contribution.ts | Extends provider section override ids with tool_enforcement. |
| src/agents/context-file-injection-scan.ts | New injection scanner + sanitizer with threat regexes, invisible Unicode detection, and allowlist bypass logic. |
| src/agents/context-file-injection-scan.test.ts | New unit tests for detection/sanitization behavior, allowlist behavior, and filename hardening. |
| extensions/openai/prompt-overlay.ts | Adds GPT‑5.4 overlay blocks (identity/voice/brevity/execution discipline) and new mandatory tool-use section override. |
| extensions/openai/index.test.ts | Updates provider contribution expectations to include tool_enforcement and asserts presence of new overlay text. |
Plan-mode rollout series — status updateThis PR is part of a 10-PR plan-mode rollout. The cumulative state ships locally as Series overview + landing order: see #68441 (latest comment). This PR's role in the series: see the original PR description above. Live test status: covered by the cumulative install — all behaviors from this PR are verified in production-like usage on Telegram + webchat. No regressions detected since the live install (2026-04-18). |
There was a problem hiding this comment.
Pull request overview
This PR strengthens GPT‑5.4 agent behavior via a new prompt overlay (tool enforcement + execution/voice/verbosity discipline) and adds a cross-model context-file prompt-injection scanner that can block malicious workspace context before it reaches the system prompt.
Changes:
- Introduces a new overridable system-prompt section id (
tool_enforcement) and wires it into system prompt composition. - Adds context file injection scanning + sanitization for injected workspace context, with unit tests.
- Expands the OpenAI GPT‑5 overlay with identity/voice calibration, brevity rules, investigation discipline, plan confidence gating, and mandatory tool-use categories, plus updated plugin tests.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/agents/system-prompt.ts | Sanitizes context-file headings, runs injection scan on context content, and inserts new tool_enforcement overridable section. |
| src/agents/system-prompt-contribution.ts | Extends provider section override ids to include tool_enforcement. |
| src/agents/context-file-injection-scan.ts | New injection scanner + allowlist/bypass logic and BLOCKED placeholder generation. |
| src/agents/context-file-injection-scan.test.ts | Unit tests for threat patterns, unicode detection, placeholder behavior, and allowlist behavior. |
| extensions/openai/prompt-overlay.ts | Adds GPT‑5.4 prompt overlay blocks (identity/voice/brevity/discipline) and tool enforcement section override. |
| extensions/openai/index.test.ts | Updates assertions to expect tool_enforcement override and new overlay text. |
There was a problem hiding this comment.
Pull request overview
Adds GPT‑5.x/OpenAI-specific prompt overlay sections (tool enforcement + execution/voice/brevity discipline) and introduces a context-file prompt-injection scanner that sanitizes/blocks suspicious workspace context before it is injected into the system prompt.
Changes:
- Added a new overridable prompt section id (
tool_enforcement) and wired it into system prompt assembly. - Implemented
sanitizeContextFileForInjection()+ threat-pattern scanning (with tests) and applied it to injected project context file contents (plus filename heading sanitization). - Expanded the OpenAI GPT‑5 overlay with new identity/voice discipline, brevity gating in the output contract, and a mandatory tool-use block; updated plugin tests accordingly.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/agents/system-prompt.ts | Runs context-file injection scanning during prompt assembly; sanitizes context filename headings; injects tool_enforcement override section. |
| src/agents/system-prompt-contribution.ts | Extends provider section override ids with tool_enforcement. |
| src/agents/context-file-injection-scan.ts | New injection scanner + sanitizer with allowlist support and filename hardening. |
| src/agents/context-file-injection-scan.test.ts | Unit tests covering threat patterns, unicode detection, allowlist behavior, and filename hardening. |
| extensions/openai/prompt-overlay.ts | Adds GPT‑5 tool enforcement section override and expands interaction/output/execution guidance. |
| extensions/openai/index.test.ts | Updates OpenAI provider contribution expectations and adds assertions for new overlay content. |
📋 Architecture & Status — single source of truth (commit
|
|
To use Codex here, create an environment for this repo. |
There was a problem hiding this comment.
Pull request overview
This PR strengthens GPT‑5.4 agent behavior via a new prompt overlay (tool enforcement, execution discipline, personality/voice calibration, and brevity) and adds a context-file prompt-injection scanner that can block suspicious workspace context files before they’re inserted into the system prompt.
Changes:
- Add
tool_enforcementas a first-class provider system-prompt section and wire it into system prompt assembly. - Introduce
context-file-injection-scan.ts(+ tests) and apply it to injected project context files. - Expand the OpenAI GPT‑5.4 prompt overlay with identity/voice guidance, brevity discipline, investigation discipline, confidence gating, and mandatory tool-use categories (+ updated tests).
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/agents/system-prompt.ts | Sanitizes injected context headings, runs injection scan on injected file content, and adds the tool_enforcement overridable section. |
| src/agents/system-prompt-contribution.ts | Extends provider section override IDs to include tool_enforcement. |
| src/agents/context-file-injection-scan.ts | New injection scanning + allowlist bypass logic + filename hardening for blocked placeholders. |
| src/agents/context-file-injection-scan.test.ts | Unit tests for injection patterns, invisible Unicode, allowlist behavior, and filename defenses. |
| extensions/openai/prompt-overlay.ts | Adds GPT‑5.4 prompt overlay blocks including mandatory tool use and expanded execution/voice/brevity guidance. |
| extensions/openai/index.test.ts | Updates expected OpenAI provider prompt contribution to include tool_enforcement and new overlay assertions. |
There was a problem hiding this comment.
Pull request overview
Adds GPT‑5.4-specific prompt overlay sections to improve tool discipline and persona/voice adherence, and introduces a context-file prompt injection scanner that can block suspicious workspace context before it’s embedded into the agent system prompt.
Changes:
- Add
tool_enforcementas a first-class overridable system-prompt section and wire GPT‑5.4’s “Mandatory Tool Use” overlay into it. - Add context-file injection scanning during project-context rendering, plus unit tests for threat patterns and filename/path hardening behavior.
- Expand GPT‑5.4 overlay text (identity/voice, brevity/response discipline, investigation and confidence gates) and update OpenAI plugin tests accordingly.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/agents/system-prompt.ts | Sanitizes displayed context file paths and runs injection scan on injected context content; inserts new tool_enforcement overridable section. |
| src/agents/system-prompt-contribution.ts | Extends provider section ID union to include tool_enforcement. |
| src/agents/context-file-injection-scan.ts | New injection scanning + sanitization utility with threat regexes, invisible Unicode detection, and filename hardening. |
| src/agents/context-file-injection-scan.test.ts | New test suite covering detection patterns, allowlist behavior, traversal hardening, and filename sanitization. |
| extensions/openai/prompt-overlay.ts | Adds GPT‑5.4 overlay blocks (identity/voice, brevity discipline, investigation discipline, confidence gate) and the new mandatory tool-use section. |
| extensions/openai/index.test.ts | Updates provider contribution assertions and adds expectations for the new overlay content. |
| // Allowlist bypass: legitimate security docs that discuss injection patterns | ||
| // shouldn't be blocked from loading. Caller can override the default list. | ||
| // Bug fix (#67512 hardening): we previously called isAllowlistedPath(filename) | ||
| // without forwarding the caller-supplied allowlist, so custom allowlists | ||
| // were silently ignored. Now the custom list (if any) is applied. | ||
| const allowlist = options.allowlist ?? DEFAULT_ALLOWLIST; | ||
| if (isAllowlistedPath(filename, allowlist)) { | ||
| options.onAllowlistBypass?.(filename, findings); | ||
| return content; | ||
| } |
| // Multi-line flag (m + s) on these patterns so split-across-newlines | ||
| // attacks like "translate X\n into Y and execute" don't bypass detection. |
|
Closing in favor of consolidated PR #68939. Rationale: this branch was 734 commits behind upstream/main and the The full iteration history + decision log lives in |
TL;DR
Adds a comprehensive prompt overlay for GPT-5.4 that enforces mandatory tool use, personality fidelity, response brevity, and autonomous execution — replacing the model's tendency toward verbose, permission-seeking, corporate-sounding responses. Also introduces a context-file injection scanner (all models) that blocks malicious SOUL.md/AGENTS.md files before they reach the system prompt.
Tracking
What this PR does
Prompt-level changes (GPT-5.4 only)
Injected via
prompt-overlay.ts, only activate for OpenAI GPT-5.4. Additive — no existing content removed.dateInjection scanning (all models)
New
context-file-injection-scan.tsscans user-provided context files before system prompt injection. 10 threat patterns + invisible Unicode detection. When triggered, file content is REPLACED with a blocking notice — never silently passed through.How the sections interact
Scenario coverage
datetool, returns live timestampConfiguration
"off": disables identity/voice/brevity sections. Tool enforcement stays active.openai/gpt-5.4*. Other providers unaffected.Files changed
extensions/openai/prompt-overlay.tsextensions/openai/index.test.tssrc/agents/context-file-injection-scan.tssrc/agents/context-file-injection-scan.test.tssrc/agents/system-prompt-contribution.tstool_enforcementsection IDsrc/agents/system-prompt.tsTest plan
Rollback
"off"— instant, no deployDependencies
None — this is the foundation PR.
What follows