Skip to content

Conversation

@pmaybank
Copy link

@pmaybank pmaybank commented May 28, 2025

  • Previously expected values were calculated on GPU using same dtype as result values
  • Now expected values are calculated on CPU using Float32 dtype
  • This fixes a test failure that was observed on Navi48 where difference between Eager mode (expected) and Inductor / Triton (result) did not meet the error tolerance when sum was evaluated on an array of Float16 values

Cherry-picked to release/2.7 branch via #2211

Cherry-picked to rocm6.5_internal_testing branch via #2212

Cherry-picked to rocm7.0_internal_testing branch via #2252

@pmaybank pmaybank requested review from jataylo and jeffdaily May 28, 2025 15:05
@pmaybank pmaybank changed the title Fix Float16 CooperativeReduction Test Failure [release/2.6] Fix Float16 CooperativeReduction Test Failure May 29, 2025
@pmaybank pmaybank changed the title [release/2.6] Fix Float16 CooperativeReduction Test Failure [release/2.6] [SWDEV-529824] Fix Float16 CooperativeReduction Test Failure May 29, 2025
@jeffdaily jeffdaily merged commit 8fe3cdd into release/2.6 May 29, 2025
@jeffdaily jeffdaily deleted the pmaybank/inductor-reductions-1 branch May 29, 2025 15:41
@pmaybank
Copy link
Author

!cherry-pick --onto release/2.7

okakarpa pushed a commit that referenced this pull request May 30, 2025
…ilure (#2204)

- Previously expected values were calculated on GPU using same dtype as
result values
- Now expected values are calculated on CPU using Float32 dtype
- This fixes a test failure that was observed on Navi48 where difference
between Eager mode (expected) and Inductor / Triton (result) did not
meet the error tolerance when sum was evaluated on an array of Float16
values

Co-authored-by: pnikolic-amd <[email protected]>
@okakarpa
Copy link
Collaborator

Created branch autogenerated/release/2.7_cherry-pick_pr-2204 and #2211

@pmaybank
Copy link
Author

!cherry-pick --onto rocm6.5_internal_testing

okakarpa pushed a commit that referenced this pull request May 30, 2025
…ilure (#2204)

- Previously expected values were calculated on GPU using same dtype as
result values
- Now expected values are calculated on CPU using Float32 dtype
- This fixes a test failure that was observed on Navi48 where difference
between Eager mode (expected) and Inductor / Triton (result) did not
meet the error tolerance when sum was evaluated on an array of Float16
values

Co-authored-by: pnikolic-amd <[email protected]>
@okakarpa
Copy link
Collaborator

jithunnair-amd pushed a commit that referenced this pull request Jun 5, 2025
…ilure (#2204)

- Previously expected values were calculated on GPU using same dtype as
result values
- Now expected values are calculated on CPU using Float32 dtype
- This fixes a test failure that was observed on Navi48 where difference
between Eager mode (expected) and Inductor / Triton (result) did not
meet the error tolerance when sum was evaluated on an array of Float16
values

Co-authored-by: pnikolic-amd <[email protected]>
(cherry picked from commit 8fe3cdd)
jithunnair-amd pushed a commit that referenced this pull request Jun 6, 2025
…ilure (#2204)

- Previously expected values were calculated on GPU using same dtype as
result values
- Now expected values are calculated on CPU using Float32 dtype
- This fixes a test failure that was observed on Navi48 where difference
between Eager mode (expected) and Inductor / Triton (result) did not
meet the error tolerance when sum was evaluated on an array of Float16
values

Co-authored-by: pnikolic-amd <[email protected]>
(cherry picked from commit 8fe3cdd)
pruthvistony pushed a commit that referenced this pull request Jun 6, 2025
…6 CooperativeReduction Test Failure (#2211)

Cherry-pick of #2204

Co-authored-by: pmaybank <[email protected]>
Co-authored-by: pnikolic-amd <[email protected]>
pruthvistony pushed a commit that referenced this pull request Jun 6, 2025
…4] Fix Float16 CooperativeReduction Test Failure (#2212)

Cherry-pick of #2204

Co-authored-by: pmaybank <[email protected]>
Co-authored-by: pnikolic-amd <[email protected]>
@jithunnair-amd
Copy link
Collaborator

!cherry-pick --onto rocm7.0_internal_testing

okakarpa pushed a commit that referenced this pull request Jun 7, 2025
…ilure (#2204)

- Previously expected values were calculated on GPU using same dtype as
result values
- Now expected values are calculated on CPU using Float32 dtype
- This fixes a test failure that was observed on Navi48 where difference
between Eager mode (expected) and Inductor / Triton (result) did not
meet the error tolerance when sum was evaluated on an array of Float16
values

Co-authored-by: pnikolic-amd <[email protected]>
@okakarpa
Copy link
Collaborator

okakarpa commented Jun 7, 2025

jithunnair-amd pushed a commit that referenced this pull request Jun 7, 2025
…ilure (#2204)

- Previously expected values were calculated on GPU using same dtype as
result values
- Now expected values are calculated on CPU using Float32 dtype
- This fixes a test failure that was observed on Navi48 where difference
between Eager mode (expected) and Inductor / Triton (result) did not
meet the error tolerance when sum was evaluated on an array of Float16
values

Co-authored-by: pnikolic-amd <[email protected]>
pragupta pushed a commit that referenced this pull request Jul 21, 2025
…ilure (#2204)

- Previously expected values were calculated on GPU using same dtype as
result values
- Now expected values are calculated on CPU using Float32 dtype
- This fixes a test failure that was observed on Navi48 where difference
between Eager mode (expected) and Inductor / Triton (result) did not
meet the error tolerance when sum was evaluated on an array of Float16
values

Co-authored-by: pnikolic-amd <[email protected]>
(cherry picked from commit 8fe3cdd)
(cherry picked from commit 34f3b3e)
pragupta pushed a commit to pragupta/pytorch that referenced this pull request Jul 21, 2025
…ilure (ROCm#2204)

- Previously expected values were calculated on GPU using same dtype as
result values
- Now expected values are calculated on CPU using Float32 dtype
- This fixes a test failure that was observed on Navi48 where difference
between Eager mode (expected) and Inductor / Triton (result) did not
meet the error tolerance when sum was evaluated on an array of Float16
values

Co-authored-by: pnikolic-amd <[email protected]>
(cherry picked from commit 8fe3cdd)
(cherry picked from commit 34f3b3e)
pragupta pushed a commit that referenced this pull request Jul 22, 2025
…ilure (#2204)

- Previously expected values were calculated on GPU using same dtype as
result values
- Now expected values are calculated on CPU using Float32 dtype
- This fixes a test failure that was observed on Navi48 where difference
between Eager mode (expected) and Inductor / Triton (result) did not
meet the error tolerance when sum was evaluated on an array of Float16
values

Co-authored-by: pnikolic-amd <[email protected]>
(cherry picked from commit 8fe3cdd)
(cherry picked from commit 34f3b3e)
jithunnair-amd pushed a commit that referenced this pull request Jul 22, 2025
…ilure (#2204)

- Previously expected values were calculated on GPU using same dtype as
result values
- Now expected values are calculated on CPU using Float32 dtype
- This fixes a test failure that was observed on Navi48 where difference
between Eager mode (expected) and Inductor / Triton (result) did not
meet the error tolerance when sum was evaluated on an array of Float16
values

Co-authored-by: pnikolic-amd <[email protected]>
(cherry picked from commit 8fe3cdd)
(cherry picked from commit 34f3b3e)
pragupta pushed a commit that referenced this pull request Jul 29, 2025
…ilure (#2204)

- Previously expected values were calculated on GPU using same dtype as
result values
- Now expected values are calculated on CPU using Float32 dtype
- This fixes a test failure that was observed on Navi48 where difference
between Eager mode (expected) and Inductor / Triton (result) did not
meet the error tolerance when sum was evaluated on an array of Float16
values

Co-authored-by: pnikolic-amd <[email protected]>
(cherry picked from commit 8fe3cdd)
(cherry picked from commit 34f3b3e)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants