You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 83866af
Browse filesBrowse the repository at this point in the historyBrowse files
- repair after a fallback re-asks the upstream that produced the failing
response (active config) instead of always the primary
- a repaired tool-call answer is forwarded like a primary one
- validated answers drop the stale Content-Length/Content-Encoding framing
- legacy functions/function_call requests also disable repair
- compiled validators are cached by schema text (no per-request event-loop
compilation); schema budget exhaustion is its own non-repairable outcome
- repair generations default to a 1024 token bound; nested config copies are
deep-copied; shared body materialization; readability and visibility fixes
|`repairPrompt`| the operator's repair instruction; required when repair is enabled | — |
21
21
|`repairMaxAttempts`| bounded regeneration budget |`1`, hard-capped at `3`|
22
-
|`repairMaxTokens`|optional upper bound applied to `max_tokens` of a repair generation | unset|
22
+
|`repairMaxTokens`| upper bound applied to `max_tokens` of a repair generation (the provider dialect may rename the field, e.g. `max_completion_tokens`) |`1024`|
23
23
24
24
Misconfiguration fails fast (unsupported schema, repair without prompt, out-of-range
25
25
budget) — nothing silently degrades. Without a schema the response bytes are
@@ -38,6 +38,7 @@ upstream attempt.
38
38
| TOOL_CALLS_PRESENT |`finish_reason: tool_calls` or message `tool_calls`| forwarded untouched — the client's workflow decision | never |
39
39
| INVALID_JSON | content field present but not parseable | REJECT (HTTP 502, reason `CONTENT`) | yes |
| SCHEMA_BUDGET_EXCEEDED | content exceeds the bounded validator's size/complexity budget — it may be perfectly valid, just too large to validate | REJECT (HTTP 502) | no — a regeneration of a merely oversized document wastes billable work |
41
42
| MALFORMED_ENVELOPE | outer response is not a parseable chat completion | REJECT (HTTP 502) | no |
42
43
| EMPTY | content absent or blank | REJECT (HTTP 502) | no |
43
44
| TRUNCATED |`finish_reason: length`| REJECT (HTTP 502) | no |
@@ -52,8 +53,17 @@ A repairable failure regenerates by re-preparing the original request with one
52
53
appended user message: the operator's `repairPrompt` (the only instruction source),
53
54
the authoritative schema, the bounded classification evidence, and the previous
54
55
output as clearly labeled **data** — model text cannot inject repair instructions.
55
-
`stream` is forced off, model/provider references are preserved, and an optional
56
-
`repairMaxTokens` clamps `max_tokens`.
56
+
`stream` is forced off, model/provider references are preserved, and
57
+
`repairMaxTokens` clamps the generation's token bound. The repair re-asks the
58
+
upstream that produced the failing response (primary after a primary failure,
59
+
the fallback after a fallback failure). Tool-bearing traffic — the `tools` array
60
+
as well as the legacy `functions`/`function_call` shapes — never enters repair.
61
+
62
+
Refusal detection is deliberately protocol-level only (`finish_reason:
63
+
content_filter`): a refusal delivered as plain text with `finish_reason: stop`
64
+
is a semantic judgement owned by the semantic-safety boundary (#7623), not
65
+
something the gateway guesses — such traffic is classified like any other
66
+
content, never specially repaired.
57
67
58
68
Every actual regeneration is a correlated billable attempt: it has its own
59
69
`Phase.REPAIR` attempt identity (`AiAttemptEvent.Kind.REPAIR`), its own usage tap,
Copy file name to clipboardExpand all lines: shenyu-plugin/shenyu-plugin-ai/shenyu-plugin-ai-common/src/main/java/org/apache/shenyu/plugin/ai/common/config/AiCommonConfig.java
+18-1Lines changed: 18 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -109,7 +109,24 @@ public AiCommonConfig(final AiCommonConfig other) {
Copy file name to clipboardExpand all lines: shenyu-plugin/shenyu-plugin-ai/shenyu-plugin-ai-proxy/src/main/java/org/apache/shenyu/plugin/ai/proxy/enhanced/conversion/ConnectResponsesToChatConverter.java
Copy file name to clipboardExpand all lines: shenyu-plugin/shenyu-plugin-ai/shenyu-plugin-ai-proxy/src/main/java/org/apache/shenyu/plugin/ai/proxy/enhanced/model/AiContentCompliance.java
+7Lines changed: 7 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -45,6 +45,13 @@ public enum AiContentCompliance {
45
45
*/
46
46
SCHEMA_MISMATCH,
47
47
48
+
/**
49
+
* The generated content exceeded the bounded validator's size or complexity
50
+
* budget; it may be perfectly valid, just too large to validate, so it is
51
+
* rejected without a repair regeneration.
52
+
*/
53
+
SCHEMA_BUDGET_EXCEEDED,
54
+
48
55
/**
49
56
* The generated content field was absent or blank.
Copy file name to clipboardExpand all lines: shenyu-plugin/shenyu-plugin-ai/shenyu-plugin-ai-proxy/src/main/java/org/apache/shenyu/plugin/ai/proxy/enhanced/service/AiGeneratedContentInspector.java
0 commit comments