Ensure that BlockMask length must always exactly match the sequence length in flex_attention #141625
Conversation
…ength in flex_attention [ghstack-poisoned]
Chillee
requested review from
albanD,
jbschlosser and
mikaylagawarecki
as code owners
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/141625
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (2 Unrelated Failures)As of commit c1f79fe with merge base 78491d6 ( BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
UNSTABLE - The following job failed but was likely due to flakiness present on trunk and has been marked as unstable:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
… sequence length in flex_attention" Fixes #141435 cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang aakhundov [ghstack-poisoned]
… sequence length in flex_attention" Fixes #141435 cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang aakhundov [ghstack-poisoned]
| flex_attention_call(*create_inputs(2048), block_mask=block_mask) | ||
|
|
||
| block_mask = create_block_mask(mask_mod, None, None, 1023, 1023) | ||
| with self.assertRaisesRegex(ValueError, "block_mask was created for"): |
There was a problem hiding this comment.
nit: stricter assert message check
There was a problem hiding this comment.
How do you want to make it stricter? I mainly just wanted to check that it's throwing the right error.
There was a problem hiding this comment.
In one case the inputs are smaller than the block mask size, in the other they are bigger and we have two different error messages, just meant ensure the correct message is being shown where the asserts are the same here
… sequence length in flex_attention" Fixes #141435 cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang aakhundov [ghstack-poisoned]
… sequence length in flex_attention" Fixes #141435 cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang aakhundov [ghstack-poisoned]
drisspg
left a comment
drisspg
left a comment
There was a problem hiding this comment.
🍻
Probs wait for diff train unblocking before landing
… sequence length in flex_attention" Fixes #141435 cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang aakhundov [ghstack-poisoned]
|
@pytorchbot revert -m "Broken main" -c nosignal See https://hud.pytorch.org/pytorch/pytorch/commit/795f28ac552eb61d02ea02fd64637ba814133bd8 for failures. |
|
@pytorchbot successfully started a revert job. Check the current status here. |
|
@Chillee your PR has been successfully reverted. |
…quence length in flex_attention (#141625)" This reverts commit 795f28a. Reverted #141625 on behalf of https://github.com/albanD due to Broken main ([comment](#141625 (comment)))
… sequence length in flex_attention" Fixes #141435 cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang aakhundov [ghstack-poisoned]
… sequence length in flex_attention" Fixes #141435 cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang aakhundov [ghstack-poisoned]
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
|
@pytorchbot merge -i |
|
The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command |
Merge startedYour change will be merged while ignoring the following 2 checks: inductor / cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu, unstable), inductor / cuda12.1-py3.10-gcc9-sm86 / test (inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
…ength in flex_attention (pytorch#141625) Fixes pytorch#141435 Pull Request resolved: pytorch#141625 Approved by: https://github.com/drisspg ghstack dependencies: pytorch#138788
…quence length in flex_attention (pytorch#141625)" This reverts commit 795f28a. Reverted pytorch#141625 on behalf of https://github.com/albanD due to Broken main ([comment](pytorch#141625 (comment)))
…ength in flex_attention (pytorch#141625) Fixes pytorch#141435 Pull Request resolved: pytorch#141625 Approved by: https://github.com/drisspg ghstack dependencies: pytorch#138788
Stack from ghstack (oldest at bottom):
Fixes #141435
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @chauhang @aakhundov