Eval bug: Jinja Template Parsing Error in TranslateGemma #20305
Description
Name and Version
llama-cli --version
ggml_cuda_init: found 4 ROCm devices (Total VRAM: 98240 MiB):
Device 0: Radeon RX 7900 XTX, gfx1100 (0x1100), VMM: no, Wave Size: 32, VRAM: 24560 MiB (24518 MiB free)
Device 1: Radeon RX 7900 XTX, gfx1100 (0x1100), VMM: no, Wave Size: 32, VRAM: 24560 MiB (24518 MiB free)
Device 2: Radeon RX 7900 XTX, gfx1100 (0x1100), VMM: no, Wave Size: 32, VRAM: 24560 MiB (24518 MiB free)
Device 3: Radeon RX 7900 XTX, gfx1100 (0x1100), VMM: no, Wave Size: 32, VRAM: 24560 MiB (24518 MiB free)
version: 8255 (43e1cbd)
built with GNU 13.3.0 for Linux x86_64
Operating systems
Linux
GGML backends
HIP
Hardware
4x 7900XTX
Models
TranslateGemma-27b-it-Q8_0
Problem description & steps to reproduce
When I use default(--jinja), I have a error like this.
render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
srv init: init: chat template parsing error: Unable to generate parser for this template. Automatic parser generation failed: std::bad_alloc
srv init: init: please consider disabling jinja via --no-jinja, or use a custom chat template via --chat-template
srv init: init: for example: --no-jinja --chat-template chatml
srv operator(): operator(): cleaning up before exit...
main: exiting due to model loading error
And I use it as I wrote in the example(--no-jinja), this error occurs.(and same with --no-jinja, --chat-template-file)
No parser definition detected, assuming pure content parser.No parser definition detected, assuming pure content parser.No parser definition detected, assuming pure content parser.No parser definition detected, assuming pure content parser.No parser definition detected, assuming pure content parser.No parser definition detected, assuming pure content parser.
This will repeat until generation is complete. The output works fine, but it outputs as regular conversation chat, not translation.
When specifying a jinja file via --chat-template-file(not include --no-jinja), the following error occurs.
ggml_cuda_init: found 4 ROCm devices (Total VRAM: 98240 MiB):
Device 0: Radeon RX 7900 XTX, gfx1100 (0x1100), VMM: no, Wave Size: 32, VRAM: 24560 MiB (24518 MiB free)
Device 1: Radeon RX 7900 XTX, gfx1100 (0x1100), VMM: no, Wave Size: 32, VRAM: 24560 MiB (24518 MiB free)
Device 2: Radeon RX 7900 XTX, gfx1100 (0x1100), VMM: no, Wave Size: 32, VRAM: 24560 MiB (24518 MiB free)
Device 3: Radeon RX 7900 XTX, gfx1100 (0x1100), VMM: no, Wave Size: 32, VRAM: 24560 MiB (24518 MiB free)
render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
common_chat_verify_template: failed to apply template: Unable to generate parser for this template. Automatic parser generation failed: std::bad_alloc
error: the supplied chat template is not supported: {%- set languages = {
"aa": "Afar",
omitted below
First Bad Commit
b7981(#19419) is first.
b7976 is fine.
Relevant log output
Logs
render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
init: chat template, example_format: '<start_of_turn>user