Skip to content

Commit 0a933b4

Browse files
hystsgradio-pr-bot
andauthored
Fix gr.SelectData coordinates when the image in gr.Image does not fill its container (#13532)
* Fix image select coordinates in fullscreen mode The coordinate math in get_coordinates_of_clicked_image() assumed object-fit: contain semantics, where the drawn image always fills the img element box along at least one axis. The image is actually rendered with object-fit: scale-down, which never upscales, so a small image is drawn at natural size, centered with empty space on both axes. In that case (most visibly in fullscreen mode) the reported gr.SelectData coordinates were scaled as if the image filled the box. Compute the drawn rect with scale-down semantics instead: the scale is capped at 1 and the centering offset is applied on both axes. The old contain branches are special cases of the new formula, so behavior in the previously correct cases is unchanged. Clicks in the empty space around the image now report no coordinates, matching the existing handling of clicks outside the image. Fixes #10268 * add changeset --------- Co-authored-by: gradio-pr-bot <[email protected]>
1 parent a11c728 commit 0a933b4

3 files changed

Lines changed: 63 additions & 13 deletions

File tree

‎.changeset/fast-cows-hug.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
"@gradio/image": patch
3+
"gradio": patch
4+
---
5+
6+
fix:Fix `gr.SelectData` coordinates when the image in `gr.Image` does not fill its container

‎js/image/Image.test.ts‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -654,6 +654,52 @@ describe("get_coordinates_of_clicked_image", () => {
654654
expect(result).toEqual([100, 200]);
655655
});
656656

657+
test("handles image shown at natural size with empty space on both axes", () => {
658+
// `object-fit: scale-down` never upscales: a 100x100 natural image in
659+
// a 400x300 box is drawn at natural size, centered at offset (150, 100).
660+
// Click at (150, 100) = top-left corner of the drawn image.
661+
const evt = make_mock_event(
662+
150,
663+
100,
664+
{
665+
naturalWidth: 100,
666+
naturalHeight: 100
667+
},
668+
{
669+
left: 0,
670+
top: 0,
671+
width: 400,
672+
height: 300
673+
}
674+
);
675+
676+
const result = get_coordinates_of_clicked_image(evt);
677+
expect(result).toEqual([0, 0]);
678+
});
679+
680+
test("returns null when click is in the empty space around a natural-size image", () => {
681+
// 100x100 natural image in a 1920x1080 box: drawn at natural size,
682+
// centered at offset (910, 490). A click at (500, 540) is inside the
683+
// box but left of the drawn image, so no image pixel is under it.
684+
const evt = make_mock_event(
685+
500,
686+
540,
687+
{
688+
naturalWidth: 100,
689+
naturalHeight: 100
690+
},
691+
{
692+
left: 0,
693+
top: 0,
694+
width: 1920,
695+
height: 1080
696+
}
697+
);
698+
699+
const result = get_coordinates_of_clicked_image(evt);
700+
expect(result).toBeNull();
701+
});
702+
657703
test("returns [NaN, NaN] when currentTarget is not an Element", () => {
658704
const evt = {
659705
currentTarget: {},

‎js/image/shared/utils.ts‎

Lines changed: 11 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -9,19 +9,17 @@ export const get_coordinates_of_clicked_image = (
99
}
1010

1111
const imageRect = image.getBoundingClientRect();
12-
const xScale = image.naturalWidth / imageRect.width;
13-
const yScale = image.naturalHeight / imageRect.height;
14-
if (xScale > yScale) {
15-
const displayed_height = image.naturalHeight / xScale;
16-
const y_offset = (imageRect.height - displayed_height) / 2;
17-
var x = Math.round((evt.clientX - imageRect.left) * xScale);
18-
var y = Math.round((evt.clientY - imageRect.top - y_offset) * xScale);
19-
} else {
20-
const displayed_width = image.naturalWidth / yScale;
21-
const x_offset = (imageRect.width - displayed_width) / 2;
22-
var x = Math.round((evt.clientX - imageRect.left - x_offset) * yScale);
23-
var y = Math.round((evt.clientY - imageRect.top) * yScale);
24-
}
12+
// The image is rendered with `object-fit: scale-down`: centered, never
13+
// scaled above its natural size, so empty space can appear on both axes.
14+
const scale = Math.min(
15+
imageRect.width / image.naturalWidth,
16+
imageRect.height / image.naturalHeight,
17+
1
18+
);
19+
const x_offset = (imageRect.width - image.naturalWidth * scale) / 2;
20+
const y_offset = (imageRect.height - image.naturalHeight * scale) / 2;
21+
const x = Math.round((evt.clientX - imageRect.left - x_offset) / scale);
22+
const y = Math.round((evt.clientY - imageRect.top - y_offset) / scale);
2523
if (x < 0 || x >= image.naturalWidth || y < 0 || y >= image.naturalHeight) {
2624
return null;
2725
}

0 commit comments

Comments
 (0)