Filter pattern matching tests based on ACL #141921

Mousius · 2024-12-02T23:43:08Z

There are a number of cases where pattern matching differs based on the presence of ACL, causing the tests to fail. This adds TEST_ACL and skipIfACL so that these tests can still run with different values or be entirely skipped if necessary.

cc @malfet @snadampal @milpuz01 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @chauhang @aakhundov

pytorch-bot · 2024-12-02T23:43:11Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/141921

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 2 Unrelated Failures

As of commit e08321e with merge base 5bc09ac ():

NEW FAILURE - The following job has failed:

pull / linux-focal-py3.9-clang10-onnx / build (gh)
##[error]Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '/home/ec2-user/actions-runner/_work/pytorch/pytorch/.github/actions/get-workflow-job-id'. Did you forget to run actions/checkout before running your local action?

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

inductor-rocm / rocm6.2-py3.10-inductor / test (inductor, 1, 2, linux.rocm.gpu.2) (gh) (similar failure)
##[error]Credentials could not be loaded, please check your action inputs: Could not load credentials from any providers
inductor-rocm / rocm6.2-py3.10-inductor / test (inductor, 2, 2, linux.rocm.gpu.2) (gh) (similar failure)
##[error]Credentials could not be loaded, please check your action inputs: Could not load credentials from any providers

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Mousius · 2024-12-02T23:45:52Z

@pytorchbot label "topic: not user facing"

Mousius · 2024-12-02T23:46:06Z

@pytorchbot label "module: arm"

malfet

Please use skipIfXYZ only if something catastrophically fails, otherwise do xfailIfXYZ
If change is compete, then please add test_mkldnn_pattern_matching to aarch64 test shard. If test is still work in progress, I would appreciate if you can split it into several PRs:

One that tweaks existing tests to match ACL patterns
Another that skips/xfails non-confirming tests, and explain why those should be be used when ACL is present (to distinguish between incorrect tests/perf tweaks and ones that actually will result in silent correctness)

torch/testing/_internal/common_utils.py

malfet · 2024-12-03T00:09:20Z

torch/testing/_internal/common_utils.py


 NOTEST_CPU = "cpu" in split_if_not_empty(os.getenv('PYTORCH_TESTING_DEVICE_EXCEPT_FOR', ''))

+skipIfACL = unittest.skipIf(TEST_ACL, "ACL is not supported")


Why not xfailIfACL

I really like this suggestion 😸 my developer senses were tingling as I wrote it.

Mousius · 2024-12-03T18:28:00Z

Please use skipIfXYZ only if something catastrophically fails, otherwise do xfailIfXYZ If change is compete, then please add test_mkldnn_pattern_matching to aarch64 test shard. If test is still work in progress, I would appreciate if you can split it into several PRs:

One that tweaks existing tests to match ACL patterns

Another that skips/xfails non-confirming tests, and explain why those should be be used when ACL is present (to distinguish between incorrect tests/perf tweaks and ones that actually will result in silent correctness)

Added to .ci/pytorch/test.sh, most of the xfail tests are to do with fusion which I've linked to in an above comment being explicitly blocked.

There are a number of cases where pattern matching differs based on the presence of ACL, causing the tests to fail. This adds `TEST_ACL` and `skipIfACL` so that these tests can still run with different values or be entirely skipped if necessary.

Mousius · 2024-12-05T17:55:32Z

@malfet this should be up to date now 😸

test/inductor/test_mkldnn_pattern_matcher.py

malfet

Thank you for the updates, looks good to me

malfet · 2024-12-05T23:52:10Z

@pytorchbot merge

pytorchmergebot · 2024-12-05T23:54:15Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2024-12-06T00:05:08Z

Merge failed

Reason: 1 jobs have failed, first few of them are: trunk / macos-py3-arm64-mps / test (mps, 1, 1, macos-m1-14)

Details for Dev Infra team

Raised by workflow job

torch/testing/_internal/common_utils.py

malfet · 2024-12-06T01:44:38Z

@pytorchbot merge

pytorchmergebot · 2024-12-06T01:46:20Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2024-12-06T01:51:46Z

Merge failed

Reason: 1 mandatory check(s) failed. The first few are:

pull / linux-focal-py3.9-clang10-onnx / build

Dig deeper by viewing the failures on hud

Details for Dev Infra team

Raised by workflow job

Failing merge rule: Core Maintainers

malfet · 2024-12-06T04:17:58Z

@pytorchbot merge -f "it finally looks ok"

pytorchmergebot · 2024-12-06T04:19:28Z

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

There are a number of cases where pattern matching differs based on the presence of ACL, causing the tests to fail. This adds `TEST_ACL` and `skipIfACL` so that these tests can still run with different values or be entirely skipped if necessary. Pull Request resolved: #141921 Approved by: https://github.com/malfet Co-authored-by: Nikita Shulga <[email protected]>

Mousius requested a review from a team as a code owner December 2, 2024 23:43

pytorch-bot bot added the module: inductor label Dec 2, 2024

pytorch-bot bot added the topic: not user facing topic category label Dec 2, 2024

pytorch-bot bot added the module: arm Related to ARM architectures builds of PyTorch. Includes Apple M1 label Dec 2, 2024

pytorchbot added the open source label Dec 2, 2024

malfet added the ciflow/linux-aarch64 linux aarch64 CI workflow label Dec 3, 2024

malfet requested changes Dec 3, 2024

View reviewed changes

cpuhrsch added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Dec 3, 2024

Mousius added 2 commits December 5, 2024 17:19

Change skipIfACL to xfailIfACL and enable tests

15e740c

Mousius force-pushed the test-mkldnn-acl-checks branch from b517f33 to 15e740c Compare December 5, 2024 17:54

malfet reviewed Dec 5, 2024

View reviewed changes

test/inductor/test_mkldnn_pattern_matcher.py Outdated Show resolved Hide resolved

Update test/inductor/test_mkldnn_pattern_matcher.py

4103846

malfet reviewed Dec 5, 2024

View reviewed changes

test/inductor/test_mkldnn_pattern_matcher.py Show resolved Hide resolved

test/inductor/test_mkldnn_pattern_matcher.py Outdated Show resolved Hide resolved

malfet and others added 2 commits December 5, 2024 14:39

Apply suggestions from code review

9ba803c

Lintrunner

ebcb689

malfet approved these changes Dec 5, 2024

View reviewed changes

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Dec 5, 2024

pytorchmergebot added the merging label Dec 5, 2024

pytorchmergebot removed the merging label Dec 6, 2024

malfet reviewed Dec 6, 2024

View reviewed changes

torch/testing/_internal/common_utils.py Outdated Show resolved Hide resolved

Update torch/testing/_internal/common_utils.py

e08321e

pytorchmergebot added the merging label Dec 6, 2024

pytorchmergebot removed the merging label Dec 6, 2024

malfet mentioned this pull request Dec 6, 2024

MacOS tests has not been running for few weeks #142206

Closed

pytorchmergebot added the merging label Dec 6, 2024

pytorchmergebot added the Merged label Dec 6, 2024

pytorchmergebot closed this in 39425fe Dec 6, 2024

pytorchmergebot removed the merging label Dec 6, 2024

malfet mentioned this pull request Dec 13, 2024

Recently enabled to ARM CI test_mkldnn_pattern_matcher.py fails on Graviton3 #143146

Closed


		NOTEST_CPU = "cpu" in split_if_not_empty(os.getenv('PYTORCH_TESTING_DEVICE_EXCEPT_FOR', ''))

		skipIfACL = unittest.skipIf(TEST_ACL, "ACL is not supported")

Filter pattern matching tests based on ACL #141921

Filter pattern matching tests based on ACL #141921

Uh oh!

Conversation

Mousius commented Dec 2, 2024 • edited by pytorch-bot bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Dec 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/141921

❌ 1 New Failure, 2 Unrelated Failures

Uh oh!

Mousius commented Dec 2, 2024

Uh oh!

Mousius commented Dec 2, 2024

Uh oh!

malfet left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

malfet Dec 3, 2024

Choose a reason for hiding this comment

Uh oh!

Mousius Dec 3, 2024

Choose a reason for hiding this comment

Uh oh!

Mousius commented Dec 3, 2024

Uh oh!

Mousius commented Dec 5, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

malfet left a comment

Choose a reason for hiding this comment

Uh oh!

malfet commented Dec 5, 2024

Uh oh!

pytorchmergebot commented Dec 5, 2024

Merge started

Uh oh!

pytorchmergebot commented Dec 6, 2024

Merge failed

Uh oh!

Uh oh!

malfet commented Dec 6, 2024

Uh oh!

pytorchmergebot commented Dec 6, 2024

Merge started

Uh oh!

pytorchmergebot commented Dec 6, 2024

Merge failed

Uh oh!

malfet commented Dec 6, 2024

Uh oh!

pytorchmergebot commented Dec 6, 2024

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Mousius commented Dec 2, 2024 •

edited by pytorch-bot bot

Loading

pytorch-bot bot commented Dec 2, 2024 •

edited

Loading