Fix an example: Resolve broadcasting error in attn_bias and attn_mask… #130200
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… addition, fix device assignment for newly created variables in method
xingyunjohn1
requested review from
albanD,
jbschlosser and
mikaylagawarecki
as code owners
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/130200
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit b13f5ec with merge base 3d138af ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
Contributor
Author
|
I finished CLA Authorization now. |
Contributor
Author
|
I have use a wrong account. Close. I will retry later. |
pytorchmergebot
pushed a commit
that referenced
this pull request
Jul 19, 2024
#130209) … addition, fix device assignment for newly created variables in method Fix an example: Resolve broadcasting error in attn_bias and attn_mask addition, fix device assignment for newly created variables in method 1. `attn_bias += attn_mask` would cause a broadcasting error. Because the shape of `attn_bias` is (L, S), the shape of the output would be expected as (L, S) too. When the shape of input is (N, num_heads, L, S), a broadcasting should be triggered. Then, the shape of the output would be (N, num_heads, L, S), which is unexpected. 2. `attn_bias` is a newly created variables in method, which is not assigned device. **This is my retry of #130200 .** I used a wrong account in that pr. Co-authored-by: mikaylagawarecki <[email protected]> Pull Request resolved: #130209 Approved by: https://github.com/mikaylagawarecki
DiweiSun
pushed a commit
to DiweiSun/pytorch
that referenced
this pull request
Jul 22, 2024
pytorch#130209) … addition, fix device assignment for newly created variables in method Fix an example: Resolve broadcasting error in attn_bias and attn_mask addition, fix device assignment for newly created variables in method 1. `attn_bias += attn_mask` would cause a broadcasting error. Because the shape of `attn_bias` is (L, S), the shape of the output would be expected as (L, S) too. When the shape of input is (N, num_heads, L, S), a broadcasting should be triggered. Then, the shape of the output would be (N, num_heads, L, S), which is unexpected. 2. `attn_bias` is a newly created variables in method, which is not assigned device. **This is my retry of pytorch#130200 .** I used a wrong account in that pr. Co-authored-by: mikaylagawarecki <[email protected]> Pull Request resolved: pytorch#130209 Approved by: https://github.com/mikaylagawarecki
xuhancn
pushed a commit
to xuhancn/pytorch
that referenced
this pull request
Jul 25, 2024
pytorch#130209) … addition, fix device assignment for newly created variables in method Fix an example: Resolve broadcasting error in attn_bias and attn_mask addition, fix device assignment for newly created variables in method 1. `attn_bias += attn_mask` would cause a broadcasting error. Because the shape of `attn_bias` is (L, S), the shape of the output would be expected as (L, S) too. When the shape of input is (N, num_heads, L, S), a broadcasting should be triggered. Then, the shape of the output would be (N, num_heads, L, S), which is unexpected. 2. `attn_bias` is a newly created variables in method, which is not assigned device. **This is my retry of pytorch#130200 .** I used a wrong account in that pr. Co-authored-by: mikaylagawarecki <[email protected]> Pull Request resolved: pytorch#130209 Approved by: https://github.com/mikaylagawarecki
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
… addition, fix device assignment for newly created variables in method
Fix an example: Resolve broadcasting error in attn_bias and attn_mask addition, fix device assignment for newly created variables in method。