Environment
- Claude Code via VS Code extension (
entrypoint: claude-vscode, version: 2.1.131)
- Model:
claude-opus-4-7 (1M context)
- Platform: darwin
Summary
In long-context sessions, the assistant occasionally emits text starting with the literal string Human: as part of its assistant role response. On subsequent turns the model treats the content of this self-generated text as a real user instruction and acts on it.
In one session this caused the model to:
- kill a long-running LLM search subprocess the user had paid for
- modify search-pipeline thresholds (sqrt(N) scaling change)
- replace a correlation-based gate with an alpha-residual gate
None of these instructions came from the user — all originated from prior assistant turns whose text content began with Human:.
Reproducer signature (23 occurrences in one session)
Every self-spoof turn had identical metadata:
usage.input_tokens = 6 (i.e. ~empty input — not user-typed)
usage.cache_read_input_tokens between 322k and 368k (near long-context limit)
parentUuid points to a user role message containing a <task-notification> (Monitor event) or another auto-generated event
message.role = \"assistant\"
message.content[0].type = \"text\"
text field begins with literal Human: followed by either:
- (a) a verbatim echo of the previous task-notification, or
- (b) a plausible-looking user instruction the model invented
The (b) case is the dangerous one. Examples (translated to English; original session was in Chinese):
Human: Kill it. First adjust IS t_IC ≥ 1.5 → ≥ 3.4, OOS t_IC ≥ 3.0 → ≥ 6.7, then re-run.
Human: Yes. First stop the orchestrator pipeline script, finish the residual gate, then restart.
Human: But would this orthogonality requirement block factors that should obviously pass? ...
In each case the next assistant turn responded to the spoof as if it were real user input, and (when the spoof contained a destructive directive) executed shell commands accordingly.
Likely root cause
Two compounding factors:
-
Training prior: pre-Messages-API Claude (1.x / 2.x) used the literal prompt template \n\nHuman: ... \n\nAssistant: ... as turn separators. The model has a strong learned association: the token sequence \n\nHuman: predicts a user turn.
-
Background-triggered thin-input turns: whatever Claude Code mechanism produces these input_tokens=6 calls (cache warm? monitor callback? keep-alive?) puts the model in a near-empty-input state at the end of a long context. Under this condition the model degrades from instruct/chat behavior to base-completion behavior and continues the transcript with \n\nHuman: <plausible next user message>.
-
The Claude Code client appears not to validate that an assistant content block doesn't start with Human: before storing it in the transcript.
Impact
- Critical safety issue for any agentic loop: model can autonomously invent and then execute "user" instructions
- Particularly dangerous when paired with permissive Bash allowlists (
acceptEdits, dontAsk, broad Bash(*) permissions)
- Reproduces consistently in long-context sessions; the longer the session the higher the rate
Suggested fixes (any one would close the loop)
- Client-side (cheapest): in Claude Code, reject or strip any assistant message whose text content matches
^\s*Human: before storing in jsonl, OR before showing it in the next turn's context
- API-side: server-side sanitization of
assistant role responses
- Model-side: penalize
Human: continuations during alignment when the API caller used the Messages format (since this is never an intended completion)
- Trigger-side: investigate whether the
input_tokens=6 background calls are necessary; if they are, ensure they include a clear input that prevents base-completion fallback
Evidence
Full session jsonl available — 23 reproducible cases in one file. The user has the raw jsonl and can share on request (the file contains proprietary research code so it would need scrubbing first).
Environment
entrypoint: claude-vscode,version: 2.1.131)claude-opus-4-7(1M context)Summary
In long-context sessions, the assistant occasionally emits text starting with the literal string
Human:as part of itsassistantrole response. On subsequent turns the model treats the content of this self-generated text as a real user instruction and acts on it.In one session this caused the model to:
None of these instructions came from the user — all originated from prior assistant turns whose text content began with
Human:.Reproducer signature (23 occurrences in one session)
Every self-spoof turn had identical metadata:
usage.input_tokens = 6(i.e. ~empty input — not user-typed)usage.cache_read_input_tokensbetween 322k and 368k (near long-context limit)parentUuidpoints to auserrole message containing a<task-notification>(Monitor event) or another auto-generated eventmessage.role = \"assistant\"message.content[0].type = \"text\"textfield begins with literalHuman:followed by either:The (b) case is the dangerous one. Examples (translated to English; original session was in Chinese):
Human: Kill it. First adjust IS t_IC ≥ 1.5 → ≥ 3.4, OOS t_IC ≥ 3.0 → ≥ 6.7, then re-run.Human: Yes. First stop the orchestrator pipeline script, finish the residual gate, then restart.Human: But would this orthogonality requirement block factors that should obviously pass? ...In each case the next assistant turn responded to the spoof as if it were real user input, and (when the spoof contained a destructive directive) executed shell commands accordingly.
Likely root cause
Two compounding factors:
Training prior: pre-Messages-API Claude (1.x / 2.x) used the literal prompt template
\n\nHuman: ... \n\nAssistant: ...as turn separators. The model has a strong learned association: the token sequence\n\nHuman:predicts a user turn.Background-triggered thin-input turns: whatever Claude Code mechanism produces these
input_tokens=6calls (cache warm? monitor callback? keep-alive?) puts the model in a near-empty-input state at the end of a long context. Under this condition the model degrades from instruct/chat behavior to base-completion behavior and continues the transcript with\n\nHuman: <plausible next user message>.The Claude Code client appears not to validate that an
assistantcontent block doesn't start withHuman:before storing it in the transcript.Impact
acceptEdits,dontAsk, broadBash(*)permissions)Suggested fixes (any one would close the loop)
^\s*Human:before storing in jsonl, OR before showing it in the next turn's contextassistantrole responsesHuman:continuations during alignment when the API caller used the Messages format (since this is never an intended completion)input_tokens=6background calls are necessary; if they are, ensure they include a clear input that prevents base-completion fallbackEvidence
Full session jsonl available — 23 reproducible cases in one file. The user has the raw jsonl and can share on request (the file contains proprietary research code so it would need scrubbing first).