Repository navigation
fix(shared): normalize box corner order before drawing - #52
Merged
JJJYmmm merged 2 commits intoSep 9, 2026
Merged
Conversation
`draw_boxes` passed each `bbox_pixel` straight to Pillow's `draw.rectangle`, which since Pillow 9.5 raises `ValueError` when the second corner precedes the first. A grounding reply that names the corners in the other order, `[800, 100, 200, 400]`, therefore raised out of `grounding.handle` -- the draw sits outside its try/except -- and the detections that had parsed fine were lost with the tool call. The same one-line exposure existed in core's `draw_bbox`, where the reversed box comes from the caller's arguments. Sort each pair of corners before drawing, so any corner order draws the rectangle it describes and the label lands at its top-left. Tests: grounding with a reversed model box returns its detections and the annotated image instead of raising; draw_bbox with a reversed box saves an image with the outline where the sorted corners put it.
Collaborator
|
TY! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
draw_boxes(src/shared/image.py) passes eachbbox_pixelstraight to Pillow'sdraw.rectangle, which since Pillow 9.5 raisesValueError: x1 must be greater than or equal to x0when the second corner precedes the first. This repo pinspillow<12with no lower bound, so every current Pillow raises. Two callers are exposed:grounding— a model reply that names the corners in the other order, e.g.[{"label": "cat", "bbox_2d": [800, 100, 200, 400]}], parses fine, but withreturn_img=truethe draw runs after the parser'stry/except, so the exception escapes the tool and the detections that had parsed are lost with the call.draw_bbox(core) — the same line, with the reversed box coming from the caller's arguments.The fix sorts each pair of corners before drawing, so any corner order draws the rectangle it describes and the label lands at its top-left. Well-ordered boxes are unchanged. Grounding's JSON still returns
bbox_normalized/bbox_pixelexactly as the model emitted them; I left that alone becausecropalready rejects a reversed box with a clear message, but can canonicalize them at parse time as well if you prefer.Regression tests
tests/test_api_tools.py::test_grounding_draws_boxes_with_reversed_corners— stubbed model reply with reversed corners andreturn_img=True: the tool returns its detections and the annotated image block instead of raising.tests/test_tools.py::test_draw_bbox_accepts_reversed_corners—draw_bboxwith[800, 100, 200, 400]on the 96×64 sample image saves a 96×64 file whose top-edge pixel(60, 6)carries the requested outline colour, i.e. the box was drawn where the sorted corners put it.Both fail on
mainand pass with the fix:Verification
macOS, Python 3.12.14, Pillow 11.3.0:
tests/test_tools.pyalready failsruff format --checkonmain(ruff 0.16.6); I added to it without reformatting it (24 insertions, 0 deletions), so its format diff is identical before and after.Not run: the opt-in
reachabilitytests (credentialed; also deselected in CI) and the Ubuntu leg of the CI matrix. The grounding test stubscall_openai_chat, so nothing touches the network.#51 also touches
src/shared/image.py(it adds a function afterdraw_boxes); I merged the two locally in both orders — no conflict, combined tests pass.Compatibility
This fix was developed with AI assistance; every result quoted above comes from running the listed commands locally.