Clarify interpolation algorithms for resample2d by inexorabletash · Pull Request #816 · webmachinelearning/webnn

inexorabletash · 2025-02-12T22:35:46Z

This gives formal definitions for the nearest-neighbor and linear interpolation modes. The definitions are based on text given by @fdwr and baseline implementation by @BruceDai and independently verified.

Resolves #358

Preview | Diff

fdwr

Nice. Thanks for improving the spec. 🙏

index.bs

fdwr · 2025-02-13T00:43:40Z

index.bs


+
+<div class="note">
+  The specific sampling algorithms are based on those widely used in existing Machine Learning frameworks. For example, when performing {{MLInterpolationMode/linear}} resampling from the following *[4, 4]* input tensor (considering only spatial dimensions):


The specific sampling algorithms are based on those widely used in existing Machine Learning frameworks.

Note
Some ML libraries got this wrong historically and did things like stretch the centers of the input corner pixels to the centers of the output corner pixels (rather than the corner extents, including the whole pixel box rectangle rather than just a point sample), which graphics experts know is incorrect 😉 and gives you poor results. Imaging libraries like OpenCV though do the right thing, and thankfully newer versions of TF and PyTorch have fixed this behavior by default. e.g. #1 #2.

Wrong:

Right:

(no action - resolve me)

If we want to say more in the spec we can!

Updated comment with visualization - think it would help? I should probably recreate it from scratch to avoid directly reusing Jacob Richeimer's figure https://jricheimer.github.io/tensorflow/2019/02/11/resize-confusion/.

I'm thinking it will be hard to capture more of the history here without it turning into an essay equivalent to these linked resources. Maybe we should just link to these blog posts? @anssiko - any thoughts on non-normative links to potentially ephemeral resources?

No need to stall on this aspect. Happy to merge if you say go.

index.bs

huningxin

LGTM with nits, thanks much!

index.bs

fdwr

👍

@fdwr

This gives formal definitions for the `nearest-neighbor` and `linear` interpolation modes. The definitions are based on text given by @fdwr and baseline implementation by @BruceDai and independently verified. Resolves #358

Co-authored-by: Dwayne Robinson <[email protected]>

Co-authored-by: Ningxin Hu <[email protected]>

inexorabletash · 2025-02-13T23:24:16Z

If it still looks good @fdwr can you squash-merge ?

SHA: 4c34b9e Reason: push, by fdwr Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

huningxin · 2025-02-14T02:52:56Z

👍

vmukhachev · 2026-01-23T08:56:04Z

it seems, it is not what Chrome on Windows does:

nearest-neighbor of [0, 1, 2, 3, 4, 5] to the shape 21 gives [0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 5, 5, 5, 5]
the spec says to return
[0, 0, 0, 0, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5]

and for nearest-neighbor of [0, 1, 2, 3, 4, 5] to the shape 3:
[1, 3, 5]
vs
[0, 2, 4]

Some testing of all inputsizes and outputsizes between 1 and 51 shows that Chrome does something like
x = floor(inputCoordinate.x + 0.5)
instead of
x = ceil(inputCoordinate.x - 0.5)
as specified
It floor(z+0.5) fails 22 of those tests althought

inexorabletash assigned inexorabletash and unassigned inexorabletash Feb 12, 2025

fdwr approved these changes Feb 13, 2025

View reviewed changes

huningxin approved these changes Feb 13, 2025

View reviewed changes

index.bs Outdated Show resolved Hide resolved

huningxin reviewed Feb 13, 2025

View reviewed changes

index.bs Outdated Show resolved Hide resolved

index.bs Outdated Show resolved Hide resolved

index.bs Outdated Show resolved Hide resolved

inexorabletash commented Feb 13, 2025

View reviewed changes

index.bs Show resolved Hide resolved

fdwr reviewed Feb 13, 2025

View reviewed changes

index.bs Outdated Show resolved Hide resolved

fdwr reviewed Feb 13, 2025

View reviewed changes

index.bs Outdated Show resolved Hide resolved

fdwr approved these changes Feb 13, 2025

View reviewed changes

inexorabletash and others added 9 commits February 13, 2025 15:20

Clarify interpolation algorithms for resample2d

af27d69

This gives formal definitions for the `nearest-neighbor` and `linear` interpolation modes. The definitions are based on text given by @fdwr and baseline implementation by @BruceDai and independently verified. Resolves #358

Update index.bs

6b3fb66

Co-authored-by: Dwayne Robinson <[email protected]>

Update index.bs

e4b459f

Co-authored-by: Dwayne Robinson <[email protected]>

Update index.bs

f611461

Co-authored-by: Dwayne Robinson <[email protected]>

Update index.bs

aaa71b2

Co-authored-by: Ningxin Hu <[email protected]>

Update index.bs

d2a1f67

Co-authored-by: Ningxin Hu <[email protected]>

Update index.bs

1c417aa

Co-authored-by: Ningxin Hu <[email protected]>

Incorporate review feedback

e107cb4

Remove parenthetical

fa7219d

fdwr merged commit 4c34b9e into webmachinelearning:main Feb 13, 2025
2 checks passed

github-actions bot added a commit that referenced this pull request Feb 13, 2025

Clarify interpolation algorithms for resample2d (#816)

cd308b3

SHA: 4c34b9e Reason: push, by fdwr Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

inexorabletash mentioned this pull request Feb 14, 2025

Support coordinate transformation modes for Resample2d #270

Closed

inexorabletash deleted the resample-algos branch February 14, 2025 03:05

dontcallmedom mentioned this pull request Jan 15, 2026

CR Snapshot Update Request for Web Neural Network API - webnn w3c/transitions#769

Closed



		<div class="note">
		The specific sampling algorithms are based on those widely used in existing Machine Learning frameworks. For example, when performing {{MLInterpolationMode/linear}} resampling from the following [4, 4] input tensor (considering only spatial dimensions):

Comments

Conversation

inexorabletash commented Feb 12, 2025 • edited by pr-preview bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fdwr left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fdwr Feb 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

inexorabletash Feb 13, 2025

Choose a reason for hiding this comment

Uh oh!

fdwr Feb 13, 2025

Choose a reason for hiding this comment

Uh oh!

inexorabletash Feb 13, 2025

Choose a reason for hiding this comment

Uh oh!

fdwr Feb 13, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

huningxin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fdwr left a comment

Choose a reason for hiding this comment

Uh oh!

inexorabletash commented Feb 13, 2025

Uh oh!

Uh oh!

huningxin commented Feb 14, 2025

Uh oh!

vmukhachev commented Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

inexorabletash commented Feb 12, 2025 •

edited by pr-preview bot

Loading

fdwr Feb 13, 2025 •

edited

Loading

vmukhachev commented Jan 23, 2026 •

edited

Loading