Repository navigation
Commit 8d46973
authored
Python: fix(python): keep ChatMessageContent.to_prompt a str when encoding is set (#14464)
### What this fixes
`ChatMessageContent.to_prompt()` is annotated `-> str` but returns
`bytes` for any message that carries an `encoding`.
```python
root = self.to_element()
return ElementTree.tostring(root, encoding=self.encoding or "unicode", short_empty_elements=False)
```
`encoding` is a real field on the content, so the `or` never protects
it: setting `encoding="utf-8"` makes `tostring` take its bytes branch.
Measured on `ca40aa72`:
```
encoding='utf-8' -> type=bytes value=b'<message role="user" encoding="utf-8">...'
encoding='ascii' -> b'<?xml ...?>\n<message ...' # XML declaration too
```
This is user-visible rather than merely untidy:
`jinja2_system_helpers._message_to_prompt` and its handlebars twin
render a context variable via `str(context.to_prompt())`, so the prompt
the model receives literally contains `b'<message role="user" ...>'` —
bytes repr plus quote noise — whenever the message had an encoding set.
Every other `tostring` call site in the same package already hardcodes
`encoding="unicode"`: `chat_history.py:285`, `:317`, `:362`,
`chat_message_content.py:275`, `handlebars_system_helpers.py:37`, `:55`
and `jinja2_system_helpers.py:29`. This one line was the outlier, so the
change aligns it rather than introducing a new convention.
The message's own encoding is still preserved: it is an attribute on the
element and round-trips through `to_element`/`from_element`; it
describes the text content, not how we serialize a prompt string.
### Change
One line, plus a docstring note, and a parametrized regression test over
`utf-8`, `ascii` and `latin-1` asserting the prompt is a `str`, has no
XML declaration, keeps the non-ASCII text, and re-parses to the same
content.
### How this was verified
`cd /private/tmp/sk/tree/python`, `PYTHONPATH=.`, the repo's own test
file `tests/unit/contents/test_chat_message_content.py`.
Base control — `semantic_kernel/contents/chat_message_content.py`
restored from `ca40aa7226531d28a721d0ca0e451d0aaf86dafc` with `git diff
-- <that file>` printing nothing (so the run measures upstream code with
the new tests kept), sha re-echoed:
```
FAILED tests/unit/contents/test_chat_message_content.py::test_cmc_to_prompt_with_encoding[utf8]
FAILED tests/unit/contents/test_chat_message_content.py::test_cmc_to_prompt_with_encoding[ascii]
FAILED tests/unit/contents/test_chat_message_content.py::test_cmc_to_prompt_with_encoding[latin1]
3 failed, 43 passed
```
With the patch applied: `46 passed`.
Enclosing directory, both revisions, same command (`pytest
tests/unit/contents`): the `test_chat_history.py::test_to_from_file` /
`::test_chat_history_serialize` collection errors appear **identically
at base and at head** — they come from optional extras missing in my
environment, not from this diff. I did not run the integration or dotnet
suites.
`ruff check` on the two files reports only a pre-existing `D` finding at
`chat_message_content.py:201`, present at base too; `ruff format
--check` is clean. (`mypy` was not installed in this environment, so the
CI type gate is unverified here.)
Dedup: searched issues and PRs (all states) for `to_prompt`,
`message_to_prompt` and `encoding unicode prompt`; the only hits are
merged history (#13738, #5550). The nearby `data_uri` MIME-subtype
defect I found while investigating is already claimed by open #14304, so
it is deliberately not part of this PR.1 parent 07ffe35 commit 8d46973
2 files changed
Lines changed: 23 additions & 1 deletion
File tree
- python
- semantic_kernel/contents
- tests/unit/contents
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
294 | 294 | | |
295 | 295 | | |
296 | 296 | | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
297 | 301 | | |
298 | 302 | | |
299 | 303 | | |
300 | 304 | | |
301 | | - | |
| 305 | + | |
302 | 306 | | |
303 | 307 | | |
304 | 308 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
152 | 152 | | |
153 | 153 | | |
154 | 154 | | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
155 | 173 | | |
156 | 174 | | |
157 | 175 | | |
| |||
0 commit comments