Skip to content

fix(codebuddy-cn): make the system-prompt length gate tunable and loud - #3342

Open
ntdatt812 wants to merge 1 commit into
decolua:masterfrom
ntdatt812:fix/codebuddy-system-prompt-len-3339
Open

ntdatt812 wants to merge 1 commit into
decolua:masterfrom
ntdatt812:fix/codebuddy-system-prompt-len-3339

Conversation

@ntdatt812

Copy link
Copy Markdown
Contributor

Closes #3339.

CodeBuddyExecutor.transformRequest swaps a system prompt for a neutral one when it looks like a coding-agent identity, because Tencent's content filter rejects those outright (#2071). The detector is an identity regex OR text.length > 2000, and the length half is the problem:

if (text.length > 2000 || AGENT_PATTERN.test(text)) { /* replace */ }

2000 characters is not much. A hand-written project system prompt — coding conventions, architecture notes, a CLAUDE.md pasted into the client — clears it easily and gets replaced with no error, no log, nothing. What the user sees is a model that quietly ignores its instructions, with no thread to pull. The report is accurate; I reproduced the path in open-sse/executors/codebuddy-cn.js:41 on master.

What this does not change

The length catch-all stays on, at 2000, by default. It is not decoration: the regex enumerates specific agents (you are claude code, cursor|windsurf|cline|aider|…, <agent-identity>, …) and cannot know the next one. The length rule is what stops an unrecognised agent CLI from tripping the filter and failing the whole request. Removing it — the issue's second option — trades a silent degradation for a hard 400 on clients that work today, so I did not take it.

The identity regex is also untouched and still evaluated independently of the limit. Raising or disabling the length rule cannot let a known agent prompt through; there is a test that pins exactly that.

What changes

CODEBUDDY_SYSTEM_PROMPT_MAX_LEN — raise it when a long legitimate prompt is being eaten, or set 0 to drop the length rule and let the identity markers decide alone. Unset keeps today's behaviour exactly. Junk (abc, 1.5) and negative values fall back to the default rather than silently disabling the guard, which is the failure mode you want from a safety heuristic. Documented in .env.example alongside the other optional knobs.

The replacement is no longer silent. It logs which rule fired:

[codebuddy-cn] system prompt replaced: agent identity marker
[codebuddy-cn] system prompt replaced: 3162 chars > CODEBUDDY_SYSTEM_PROMPT_MAX_LEN (2000)

The second line names both the measurement and the knob, so a user hitting the false positive can see the cause and the fix in one line. This is arguably the more important half of the fix — the issue title says "silently", and that is the part that made it hard to diagnose.

Something related I noticed

open-sse/executors/codebuddy-intl.js:31 does the same thing far more bluntly, and unconditionally:

transformed.messages = [{ role: "system", content: "You are CodeBuddy Code." }];
for (const message of source) {
  if (!message || typeof message !== "object" || ["system", "developer"].includes(message.role)) continue;
  ...
}

Every system and developer message is dropped on codebuddy-intl, at any length, agent prompt or not. The comment explains the gateway needs a leading system prompt — which does not by itself require discarding the caller's. I have left it alone: I have no codebuddy-intl account to check how strict that gateway is, and guessing there would risk a regression of the same class as #2071. Happy to follow up if a maintainer confirms the intent, or to file it separately.

Verification

9 tests in unit/codebuddy-system-prompt-len-3339.test.js: an agent prompt is still replaced; a short prompt is untouched and logs nothing; the default is still 2000 when the var is unset; a raised limit lets a long project prompt survive; 0 means identity-markers-only and still catches an agent prompt; junk/negative/empty values fall back to the default; the warning names the rule, the length and the env var; typed-block content ([{type:"text",text}]) keeps its shape on replacement; user and assistant messages are never touched.

Mutation-checked so the tests are known to bite:

mutation result
restore the hard-coded text.length > 2000 the raised-limit and 0 tests fail
delete the console.warn the "names the rule that fired" test fails

The pre-existing unit/codebuddy-reasoning-optin.test.js (#2071) still passes. Full suite (npx vitest run unit translator): 1817 passed / 93 failed, failing set byte-identical to master's, comm empty in both directions. npx eslint reports no issues on the changed files.

Tencent's filter rejects coding-agent identity prompts, so the executor swaps
them for a neutral one. The detector is an identity regex OR a hard-coded
"longer than 2000 characters", and the length half misfires: a hand-written
project system prompt — conventions, architecture notes, a CLAUDE.md pasted in
— clears 2000 easily and gets replaced with no error, no log, nothing. The user
sees a model that ignores its instructions and has no way to find out why.

The catch-all stays on by default, unchanged at 2000, because it is what keeps
an agent CLI the regex has never seen from tripping the filter. What changes:

- CODEBUDDY_SYSTEM_PROMPT_MAX_LEN raises or lowers the limit; 0 disables the
  length rule so only the identity markers decide. Junk and negative values
  fall back to the default rather than disabling the guard by accident.
- A replacement now logs which rule fired, with the prompt length and the env
  var name when it was the length rule, so a false positive is diagnosable
  instead of silent.

The identity regex is untouched and still fires regardless of the limit, so
raising or disabling the length rule cannot let a known agent prompt through.

Reported in decolua#3339.
@ntdatt812
ntdatt812 force-pushed the fix/codebuddy-system-prompt-len-3339 branch from 39cb668 to b6297f3 Compare September 9, 2026 03:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

codebuddy-cn: hard-coded 2000-char threshold silently replaces legitimate long system prompts

1 participant