Repository navigation
convert _upsample_bilinear2d_aa with the antialias filter - #2907
Open
HuzaifaChaudary wants to merge 4 commits into
Open
HuzaifaChaudary wants to merge 4 commits into
HuzaifaChaudary wants to merge 4 commits into
Conversation
it was an alias of upsample_bilinear2d , so antialias=True was dropped and the result was wrong whenever an axis is downsampled , and with align_corners=True always . it now goes through the same separable _aa path as bicubic , with ATen's triangle filter . a dynamic shape upsample without align_corners still takes the plain bilinear lowering , which gives the same result there
output sizes that upsample , downsample and do both , scale factors with and without align_corners , and a dynamic shape . without the converter change 44 of them fail , all the ones where antialiasing changes the result in torch
the gather per tap , then mul and reduce_sum over the taps , came out wrong on the core ml runtime once the input is image sized . a 512x384 to 224x168 resize was off by up to 0.83 for bilinear with antialias and 0.87 for bicubic , while the same weights applied in numpy match torch to 1e-5 . the taps are scattered into a dense (out, in) matrix and applied with matmul , which matches torch at those sizes and is fewer ops
the other cases use 8x8 inputs , which are too small to show the runtime problem . with the gather version 4 of these fail on the fp32 mlprogram backend , including plain bicubic
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
_upsample_bilinear2d_aawas registered as an alias ofupsample_bilinear2d, soantialias=Truewas dropped without a warning . torchvision'sResizeuses it by default , so most image models that resize on the way in get it . in torch the two only agree on an upsample withoutalign_corners, every downsample and everything withalign_corners=Trueis different .this sends the
_aakinds through the separable path from #2818 , with aten's triangle filter in place of the cubic one . the weight code is shared , only the filter and the tap count change .that was not enough on its own . at an image size the separable path itself comes out wrong on the core ml runtime , for bicubic on main as well . a 512x384 to 224x168 resize was off by up to 0.87 for bicubic , while the same weights applied in numpy match torch to 1e-5 . i narrowed it to the gather per tap , then
mulandreduce_sumover the taps .mulalone andreduce_sumalone are exact , the two together are not once the tensor gets past about 65k elements with a short last axis , filed as #2906 :(96, 168, 4)is exact and(98, 168, 4)is off by about 3 , cpu only , fp32 , empty pass pipeline . so the third commit scatters the taps into a dense(out, in)matrix and applies it with onematmulper axis , which is also fewer ops .torchvision.transforms.v2.Resize((224, 168))on a 512x384 image in [0, 1] was off by 0.116 on average and up to 0.497 on main . with this it is within 7.3e-06 of torch .a dynamic shape upsample without
align_cornersis the same with or without antialias , so it still goes to the plain bilinear lowering , which handles dynamic shapes . a dynamic downsample raises now instead of converting to the wrong answer .44 of the new antialias cases fail without the converter change , and the image size test fails 4 of 9 with the gather version , plain bicubic included .
TestUpsamplewithINCLUDE_MIL_FP32_UNIT_TESTS=1is 945 passed , 3 skipped , 99 xfailed .