Repository navigation
Conversation
The ImageOutput branch of trace_node_to_html was chained to `if active_role is not None:` as an elif, so images appended inside a role (e.g. `with user(): lm += image(...)`) were never rendered and a stray unbalanced </span> was emitted. When the branch was reached, the raw image bytes held by the Base64Bytes field were utf-8 decoded for the base64 data URI, raising UnicodeDecodeError for real images, and the format field was ignored. Render the image inside the active role wrapper, base64-encode the stored bytes, and honor the format field.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
trace_node_to_htmlnever renderedImageOutputattributes that were appended inside a role (e.g.with user(): lm += image(...)), and raisedUnicodeDecodeErrorwhen an image output appeared outside a role. This PR renders image outputs in the HTML trace, inside the active role wrapper when a role is open.Why
Two problems in
trace_node_to_html(guidance/visual/_trace.py):ImageOutputbranch was anelifchained to the role-closer block (if active_role is not None: ... active_role = None, andelif isinstance(output_attr, ImageOutput):, atguidance/visual/_trace.py:90-101on main). The closer block runs for every output attribute of a node while a role is active, so for an image appended inside a role the closer emitted an unbalanced</span>(its opening<span>was only emitted on theTextOutputpath) and theelifwas never reached -- the<img>tag was never emitted at all.ImageOutput.value-- a pydanticBase64Bytesfield (guidance/trace/_trace.py:155-158) that holds the decoded raw image bytes -- was utf-8 decoded with.decode()and interpolated directly into the data URI, which raisesUnicodeDecodeErrorfor real image bytes (e.g. the0x89first byte of a PNG) and would not be valid base64 even when the decode happened to succeed. Theformatfield was also ignored (hardcodedimage/png).Changes
guidance/visual/_trace.py: the role-opener emission (role<span>/<div>wrapper) now also applies toImageOutputwhen a role is active, so images render inside the role wrapper and the emitted tags stay balanced;ImageOutputis handled as a sibling branch ofTextOutputinstead of being chained tonot active_role; the data URI is built withbase64.b64encode(output_attr.value)and usesoutput_attr.format.tests/unit/test_visual.py: regression tests for an image output outside any role (utf-8-invalid PNG bytes; alsoformat="jpeg"), and for an image output inside a role opened on a parent trace node (mirrorswith user(): lm += image(...)), asserting the<img>tag is present and<span>tags are balanced.Text-in-role rendering is unchanged (verified byte-identical output against main for both
prettify_rolessettings where main does not already crash).Testing
New tests on the unpatched tree:
Patched tree:
Related issue
Related to #1077 (images not visible in output representations); that issue covers the string/ChatML representation and console display, while this PR fixes the HTML trace path (
trace_node_to_html).