Skip to content

Fix json schema with '\' in literals#17307

Merged
CISC merged 6 commits intoggml-org:masterfrom
i-v-s:fix-json-schema-escape
Nov 29, 2025
Merged

Fix json schema with '\' in literals#17307
CISC merged 6 commits intoggml-org:masterfrom
i-v-s:fix-json-schema-escape

Conversation

@i-v-s
Copy link
Copy Markdown
Contributor

@i-v-s i-v-s commented Nov 16, 2025

@ehoogeveen-medweb
Copy link
Copy Markdown

Depending on the implementation, there might be an ordering issue here: If " is escaped to \" and then \ is escaped to \\, " might get escaped to \\". Generally \ should be escaped first to avoid such issues - but I don't know if it's a problem here.

@i-v-s
Copy link
Copy Markdown
Contributor Author

i-v-s commented Nov 16, 2025

I think that twice replacement is not possible here, because the substitution result does not participates in the matching. My test shows that all works as expected:

$ ./debug/bin/llama-cli -m gemma-3-4B-it-Q8_0.gguf --json-schema '{"properties":{"code":{"const":"auto s = \"000\";","description":"Generated code","title":"Code","type":"string"}},"required":["code"],"title":"DecoderResponse","type":"object"}'

...

== Running in interactive mode. ==
 - Press Ctrl+C to interject at any time.
 - Press Return to return control to the AI.
 - To return control without starting a new line, end your input with '/'.
 - If you want to submit another line, end your input with '\'.
 - Not using system message. To change it, set a different value via -sys PROMPT


> AA
{"code":"auto s = \"000\";"}

@CISC
Copy link
Copy Markdown
Member

CISC commented Nov 17, 2025

Please add a test to tests/test-json-schema-to-grammar.cpp.

@i-v-s i-v-s requested a review from ggerganov as a code owner November 23, 2025 15:41
@github-actions github-actions Bot added testing Everything test related examples python python script changes server labels Nov 23, 2025
@i-v-s
Copy link
Copy Markdown
Contributor Author

i-v-s commented Nov 29, 2025

Hello. @ggerganov, @CISC, could you merge my PR, please?

@CISC CISC merged commit 0874693 into ggml-org:master Nov 29, 2025
1 check passed
@CISC
Copy link
Copy Markdown
Member

CISC commented Nov 29, 2025

Hello. @ggerganov, @CISC, could you merge my PR, please?

Sure thing, sorry for the delay, thank you for the fix! :)

@i-v-s i-v-s deleted the fix-json-schema-escape branch November 29, 2025 16:21
Anico2 added a commit to Anico2/llama.cpp that referenced this pull request Jan 15, 2026
* Fix json schema with '\' in literals

* Add "literal string with escapes" test
blime4 referenced this pull request in blime4/llama.cpp Feb 5, 2026
* Fix json schema with '\' in literals

* Add "literal string with escapes" test
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
* Fix json schema with '\' in literals

* Add "literal string with escapes" test
phuongncn pushed a commit to phuongncn/llama.cpp-gx10-dgx-sparks-deepseekv4 that referenced this pull request Apr 28, 2026
… and new jinja template engine (ggml-org#1369)

---------

Co-authored-by: Piotr Wilkin <[email protected]>

common : add nemotron 3 parsing (ggml-org#18077)

common : add parser for ministral/mistral large 3/devstral 2 (ggml-org#17713)

common : default content to an empty string (ggml-org#18485)

chat: make tool description and parameters optional per OpenAI spec (ggml-org#18478)

Per the OpenAI API specification, both 'description' and 'parameters'
fields in tool function definitions are optional. Previously, the parser
would throw an exception if these fields were missing.

Attempts to fix ggml-org#17667

common : implement new jinja template engine (ggml-org#18462)
---------

Co-authored-by: Alde Rojas <[email protected]>
Co-authored-by: Sigbjørn Skjæret <[email protected]>
Co-authored-by: Georgi Gerganov <[email protected]>

jinja: correct member access rule (ggml-org#18905)

jinja : fix lexing of float literals with sign (ggml-org#18901)

jinja : add missing tojson filter for bool (ggml-org#18900)

jinja : attribute support for join, map and sort (ggml-org#18883)

jinja : fix object item order (and properly implement dictsort) (ggml-org#18904)

tests : add test-jinja -py option for cross-checking (ggml-org#18906)

Co-authored-by: Sigbjørn Skjæret <[email protected]>

---------

Co-authored-by: Sigbjørn Skjæret <[email protected]>

ci : run test-jinja -py on high perf [no ci] (ggml-org#18916)

jinja : fix undefined keys and attributes and int/float as bool (ggml-org#18924)

jinja: support none|string (ggml-org#18995)

Co-authored-by: Sigbjørn Skjæret <[email protected]>

Co-authored-by: Sigbjørn Skjæret <[email protected]>

---------

Co-authored-by: Sigbjørn Skjæret <[email protected]>

jinja : implement mixed type object keys (ggml-org#18955)

---------

Co-authored-by: Xuan Son Nguyen <[email protected]>

jinja : undefined should be treated as sequence/iterable (return string/array) by filters/tests (ggml-org#19147)

`tojson` is not a supported `undefined` filter

keep it DRY and fix some types

jinja : do not pass empty tools and add some none filters (ggml-org#19176)

jinja : add unordered_map include to value.h [no ci] (ggml-org#19205)

jinja : add missing 'in' test to template engine (ggml-org#19004) (ggml-org#19239)

The jinja template parser was missing the 'in' test from
global_builtins(), causing templates using reject("in", ...),
select("in", ...), or 'x is in(y)' to fail with
"selectattr: unknown test 'in'".

This broke tool-calling for Qwen3-Coder and any other model
whose chat template uses the 'in' test.

Added test_is_in supporting array, string, and object containment
checks, mirroring the existing 'in' operator logic in runtime.cpp.

Includes test cases for all three containment types plus
reject/select filter usage.

Co-Authored-By: Claude Opus 4.5 <[email protected]>

---------

Co-authored-by: Sid Mohan <[email protected]>
Co-authored-by: Claude Opus 4.5 <[email protected]>
Co-authored-by: Xuan Son Nguyen <[email protected]>

Add Jinja support for "indent" string filter (ggml-org#19529)

Co-authored-by: Sigbjørn Skjæret <[email protected]>

Co-authored-by: Sigbjørn Skjæret <[email protected]>

---------

Co-authored-by: Sigbjørn Skjæret <[email protected]>

add vendor

refactor chat

server : support preserving reasoning_content in assistant message (ggml-org#18994)

chat : fix translategemma crash on common_chat_format_example (ggml-org#19019)

chat: fix language input for translategemma (ggml-org#19052)

Co-authored-by: Aldehir Rojas <[email protected]>

---------

Co-authored-by: Aldehir Rojas <[email protected]>

chat: fix case where template accepts type content only (ggml-org#19419)

mtmd : chat : Fix extra \n between text and media marker (ggml-org#19595)

Thanks to @tugot17 for detecting and reporting the issue.

For vision models (e.g. LFM2.5-VL-1.6B and Qwen/Qwen3-VL-4B-Instruct) `llama-mtmd-cli` produces identical output to HF implementation.

However `llama-server` doesn't. I traced it down to extra newline
inserted after `<__media__>`.

This happens in `to_json_oaicompat`, that treats media markers as text
and joins all parts with `\n` separator.

PR introduces new type `media_marker` and uses it for media markers.
Extra logic is added to prevent insertion of newlines before and after
media markers.

With this change number of input tokens is identical to HF
implementation and as a result the output is also identical.

I explored other ways to address the issue
* remove completely `\n` between text parts in `to_json_oaicompat`
* merge text messages in server-common.cpp before sending them to `to_json_oaicompat`

Please propose alternative ways of fixing this issue.

Co-authored-by: Piotr Wilkin (ilintar) <[email protected]>

---------

Co-authored-by: Piotr Wilkin (ilintar) <[email protected]>

common : merge qwen3-coder and nemotron nano 3 parsers (ggml-org#19765)

common : fix improper trimming in XML parser on complete message (ggml-org#19805)

Co-authored-by: Jules LEIDELINGER <[email protected]>

jinja: correct stats for tojson and string filters (ggml-org#19785)

jinja : correct default size for string slices (ggml-org#19913)

common : handle unicode during partial json parsing (ggml-org#16526)

common : fix json schema with '\' in literals (ggml-org#17307)

add back qwen_coder_xml and mirothinker

Co-authored-by: Aldehir Rojas <[email protected]>
ljubomirj pushed a commit to ljubomirj/llama.cpp that referenced this pull request May 6, 2026
* Fix json schema with '\' in literals

* Add "literal string with escapes" test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

examples python python script changes server testing Everything test related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Misc. bug: Symbol '\' is not escaped in the json schema literals.

4 participants