Add Intel Gaudi device/HPU to auto load in instantiate_device_type_tests #126970

ankurneog · 2024-05-23T09:38:00Z

Motivation

Intel Gaudi accelerator (device name hpu) is seen to have good pass rate with the pytorch framework UTs , however being an out-of-tree device, we face challenges in adapting the device to natively run the existing pytorch UTs under pytorch/test. The UTs however is a good indicator of the device stack health and as such we run them regularly with adaptations.
Although we can add Gaudi/HPU device to generate the device specific tests using the TORCH_TEST_DEVICES environment variable, we miss out on lot of features such as executing for specific dtypes, skipping and overriding opInfo. With significant changes introduced every Pytorch release maintaining these adaptations become difficult and time consuming.
Hence with this PR we introduce Gaudi device in common_device_type framework, so that the tests are instantiated for Gaudi when the library is loaded.
The eventual goal is to introduce Gaudi out-of-tree support as equivalent to in-tree devices

Changes

Add HPUTestBase of type DeviceTypeTestBase specifying appropriate attributes for Gaudi/HPU.
Include code to check if intel Gaudi Software library is loaded and if so, add the device to the list of devices considered for instantiation of device type tests

Additional Context

please refer the following RFC : pytorch/rfcs#63

pytorch-bot · 2024-05-23T09:38:04Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/126970

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit ca8b215 with merge base 648625b ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

linux-foundation-easycla · 2024-05-23T09:38:05Z

The committers listed above are authorized under a signed CLA.

✅ login: ankurneog / name: Ankur Neog (ca8b215)

ankurneog · 2024-05-24T06:34:35Z

@mruberry , @guangyey : could you please help with the review and merge. Thank you.

ankurneog · 2024-06-04T04:35:27Z

@albanD /@colesbury : can you please help with the review and merge, thank you.

ankurneog · 2024-06-11T10:22:43Z

@pytorchbot rebase

pytorch-bot · 2024-06-11T10:22:47Z

You don't have permissions to rebase this PR since you are a first time contributor. If you think this is a mistake, please contact PyTorch Dev Infra.

albanD

Ho sorry this dropped from my queue.
Of course, since we don't have CI signal for this, we won't be able to guarantee that this won't get broken but feel free to send follow ups as needed.

albanD · 2024-06-11T13:13:13Z

@pytorchbot merge

pytorchmergebot · 2024-06-11T13:14:58Z

Merge failed

Reason: This PR needs a release notes: label
If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Details for Dev Infra team

Raised by workflow job

ankurneog · 2024-06-11T16:29:30Z

@pytorchbot label "topic: not user facing"

ankurneog · 2024-06-11T16:32:01Z

@albanD : Thank you , there was some label missing (user facing) and hence the merge failed, I have added that , could you please trigger a merge again.

ankurneog · 2024-06-11T16:33:15Z

@pytorchbot merge

pytorchmergebot · 2024-06-11T16:34:56Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

…sts (pytorch#126970) ### Motivation Intel Gaudi accelerator (device name hpu) is seen to have good pass rate with the pytorch framework UTs , however being an out-of-tree device, we face challenges in adapting the device to natively run the existing pytorch UTs under pytorch/test. The UTs however is a good indicator of the device stack health and as such we run them regularly with adaptations. Although we can add Gaudi/HPU device to generate the device specific tests using the TORCH_TEST_DEVICES environment variable, we miss out on lot of features such as executing for specific dtypes, skipping and overriding opInfo. With significant changes introduced every Pytorch release maintaining these adaptations become difficult and time consuming. Hence with this PR we introduce Gaudi device in common_device_type framework, so that the tests are instantiated for Gaudi when the library is loaded. The eventual goal is to introduce Gaudi out-of-tree support as equivalent to in-tree devices ### Changes Add HPUTestBase of type DeviceTypeTestBase specifying appropriate attributes for Gaudi/HPU. Include code to check if intel Gaudi Software library is loaded and if so, add the device to the list of devices considered for instantiation of device type tests ### Additional Context please refer the following RFC : pytorch/rfcs#63 Pull Request resolved: pytorch#126970 Approved by: https://github.com/albanD

## Motivation This is follow up to PR:#126970 to support Gaudi devices for Pytorch UT execution. ## Changes We are adding additional hooks to: 1. Add dtype exceptions for Gaudi/HPU 2. Extend onlyNativeDevices decorator functionality to add additional devices Pull Request resolved: #128584 Approved by: https://github.com/albanD

## Motivation This is follow up to PR:pytorch#126970 to support Gaudi devices for Pytorch UT execution. ## Changes We are adding additional hooks to: 1. Add dtype exceptions for Gaudi/HPU 2. Extend onlyNativeDevices decorator functionality to add additional devices Pull Request resolved: pytorch#128584 Approved by: https://github.com/albanD

## Motivation This is follow up to PR:#126970, adding facility to run content for Intel Gaudi devices. We intend to extend similar generalization for the rest of the content in test/dynamo which is currently being written to work specifically for cuda devices. Other devices can add onto it if support is available. ## Changes carve out bert related content to another class use instantiate_device_type utility to instantiate this class for devices which support the functionality Pull Request resolved: #130714 Approved by: https://github.com/anijain2305

## Motivation This is follow up to PR:pytorch#126970, adding facility to run content for Intel Gaudi devices. We intend to extend similar generalization for the rest of the content in test/dynamo which is currently being written to work specifically for cuda devices. Other devices can add onto it if support is available. ## Changes carve out bert related content to another class use instantiate_device_type utility to instantiate this class for devices which support the functionality Pull Request resolved: pytorch#130714 Approved by: https://github.com/anijain2305

… tests on HPU as well (#133975) **MOTIVATION** We recently integrated support for Intel Gaudi devices (identified as 'hpu') into the common_device_type framework via the pull request at #126970. This integration allows tests to be automatically instantiated for Gaudi devices upon loading the relevant library. Building on this development, the current pull request extends the utility of these hooks by adapting selected CUDA tests to operate on Gaudi devices. Additionally, we have confirmed that these modifications do not interfere with the existing tests on CUDA devices. **CHANGES** - Add support for HPU devices within the payload function. - Use instantiate_device_type_tests with targeted attributes to generate device-specific test instances. - Expand the supported_activities() function to include checks for torch.profiler.ProfilerActivity.HPU. - Apply skipIfHPU decorator to bypass tests that are not yet compatible with HPU devices. Pull Request resolved: #133975 Approved by: https://github.com/briancoutinho, https://github.com/aaronenyeshi

**MOTIVATION** We recently integrated support for Intel Gaudi devices (identified as 'hpu') into the common_device_type framework via the pull request at #126970. This integration allows tests to be automatically instantiated for Gaudi devices upon loading the relevant library. Building on this development, the current pull request extends the utility of these hooks by adapting selected CUDA tests to operate on Gaudi devices. Additionally, we have confirmed that these modifications do not interfere with the existing tests on CUDA devices. **CHANGES** - Add support for HPU devices within the test_move_exported_model_bn using TEST_HPU flag - Use instantiate_device_type_tests with targeted attributes to generate device-specific test instances. - Apply skipIfHPU decorator to bypass tests that are not yet compatible with HPU devices. Pull Request resolved: #137863 Approved by: https://github.com/jerryzh168

**MOTIVATION** We recently integrated support for Intel Gaudi devices (identified as 'hpu') into the common_device_type framework via the pull request at pytorch#126970. This integration allows tests to be automatically instantiated for Gaudi devices upon loading the relevant library. Building on this development, the current pull request extends the utility of these hooks by adapting selected CUDA tests to operate on Gaudi devices. Additionally, we have confirmed that these modifications do not interfere with the existing tests on CUDA devices. **CHANGES** - Add support for HPU devices within the test_move_exported_model_bn using TEST_HPU flag - Use instantiate_device_type_tests with targeted attributes to generate device-specific test instances. - Apply skipIfHPU decorator to bypass tests that are not yet compatible with HPU devices. Pull Request resolved: pytorch#137863 Approved by: https://github.com/jerryzh168

**MOTIVATION** We recently integrated support for Intel Gaudi devices (identified as 'hpu') into the common_device_type framework via the pull request at #126970. This integration allows tests to be automatically instantiated for Gaudi devices upon loading the relevant library. Building on this development, the current pull request extends the utility of these hooks by adapting selected CUDA tests to operate on Gaudi devices. Additionally, we have confirmed that these modifications do not interfere with the existing tests on CUDA devices. Other accelerators can also extend the functionality by adding the device in the devices list. ( For eg: xpu ) **CHANGES** Create a separate class for test functions running on CUDA devices Extend the functionality of these tests to include HPUs Use instantiate_device_type_tests with targeted attributes to generate device-specific test instances within the new classes Apply skipIfHPU decorator to bypass tests that are not yet compatible with HPU devices Previously we had submitted some changes in #140131 . However, deleted that PR due to merge conflicts and other issues. Pull Request resolved: #144387 Approved by: https://github.com/ankurneog, https://github.com/EikanWang, https://github.com/yanboliang, https://github.com/guangyey

This PR is related to #145476 . That PR had two files (test_functions.py and test_misc.py) . test_functions was causing CI/rebase/merge issues and hence removed for now. This PR contains only test_misc.py. This is a continuation of #144387 . ## MOTIVATION We recently integrated support for Intel Gaudi devices (identified as 'hpu') into the common_device_type framework via the pull request at #126970. This integration allows tests to be automatically instantiated for Gaudi devices upon loading the relevant library. Building on this development, the current pull request extends the utility of these hooks by adapting selected CUDA tests to operate on Gaudi devices. Additionally, we have confirmed that these modifications do not interfere with the existing tests on CUDA devices. Other accelerators can also extend the functionality by adding the device in the devices list. ( For eg: xpu ) ## CHANGES Create a separate class for test functions running on CUDA devices Extend the functionality of these tests to include HPUs Use instantiate_device_type_tests with targeted attributes to generate device-specific test instances within the new classes Apply skipIfHPU decorator to bypass tests that are not yet compatible with HPU devices PS: Most of these changes were initially part of #147609 , but closed that PR due to merge conflicts. The review comments were handled in this PR. Pull Request resolved: #149499 Approved by: https://github.com/EikanWang, https://github.com/desertfire, https://github.com/cyyever

This PR is related to pytorch#145476 . That PR had two files (test_functions.py and test_misc.py) . test_functions was causing CI/rebase/merge issues and hence removed for now. This PR contains only test_misc.py. This is a continuation of pytorch#144387 . ## MOTIVATION We recently integrated support for Intel Gaudi devices (identified as 'hpu') into the common_device_type framework via the pull request at pytorch#126970. This integration allows tests to be automatically instantiated for Gaudi devices upon loading the relevant library. Building on this development, the current pull request extends the utility of these hooks by adapting selected CUDA tests to operate on Gaudi devices. Additionally, we have confirmed that these modifications do not interfere with the existing tests on CUDA devices. Other accelerators can also extend the functionality by adding the device in the devices list. ( For eg: xpu ) ## CHANGES Create a separate class for test functions running on CUDA devices Extend the functionality of these tests to include HPUs Use instantiate_device_type_tests with targeted attributes to generate device-specific test instances within the new classes Apply skipIfHPU decorator to bypass tests that are not yet compatible with HPU devices PS: Most of these changes were initially part of pytorch#147609 , but closed that PR due to merge conflicts. The review comments were handled in this PR. Pull Request resolved: pytorch#149499 Approved by: https://github.com/EikanWang, https://github.com/desertfire, https://github.com/cyyever

ankurneog requested review from a team and mruberry as code owners May 23, 2024 09:38

ankurneog marked this pull request as draft May 23, 2024 09:38

pytorchbot added the open source label May 23, 2024

ankurneog force-pushed the ankurneog_hpu_device_ut_support branch from 67cd3e6 to e9c5a5f Compare May 24, 2024 03:28

ankurneog marked this pull request as ready for review May 24, 2024 03:30

guangyey requested a review from albanD May 24, 2024 07:25

ankurneog force-pushed the ankurneog_hpu_device_ut_support branch from e9c5a5f to cd86ee0 Compare May 24, 2024 08:25

colesbury added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label May 24, 2024

Add Intel Gaudi device/HPU to auto load in instantiate_device_type_tests

ca8b215

ankurneog force-pushed the ankurneog_hpu_device_ut_support branch from cd86ee0 to ca8b215 Compare June 11, 2024 10:31

albanD approved these changes Jun 11, 2024

View reviewed changes

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Jun 11, 2024

pytorchmergebot added the merging label Jun 11, 2024

pytorchmergebot removed the merging label Jun 11, 2024

pytorch-bot bot added the topic: not user facing topic category label Jun 11, 2024

pytorchmergebot added the merging label Jun 11, 2024

ankurneog deleted the ankurneog_hpu_device_ut_support branch June 13, 2024 01:30

ankurneog mentioned this pull request Jun 13, 2024

Add hooks for execution on intel gaudi devices - 1 #128584

Closed

ankurneog mentioned this pull request Jul 15, 2024

Generalize devices specific UTs for dynamo #130714

Closed

ankurneog mentioned this pull request Jul 25, 2024

Generalization of distributed UT content to enable non cuda device execution #131758

Closed

This was referenced Aug 20, 2024

test_execution_trace.py: Use instantiate_device_type_tests to run GPU tests on HPU as well #133971

Closed

test_execution_trace.py: Use instantiate_device_type_tests to run GPU tests on HPU as well #133975

Closed

amathewc mentioned this pull request Oct 13, 2024

Update test_quantize_pt2e.py with HPU support #137863

Closed

amathewc mentioned this pull request Nov 8, 2024

Adapt Dynamo tests to HPUs using instantiate_device_type_tests #140131

Closed

amathewc mentioned this pull request Jan 8, 2025

Adapt Dynamo tests to HPUs using instantiate_device_type_tests #144387

Closed

amathewc mentioned this pull request Jan 23, 2025

Adapt Dynamo Tests to HPUs #145476

Closed

This was referenced Feb 18, 2025

Add HPU support to test_structured_sparsifier.py #147370

Closed

Add HPU support to test_structured_sparsifier.py #147371

Closed

Adapt test_misc.py to HPUs #147609

Closed

amathewc mentioned this pull request Mar 19, 2025

Adapt test_misc.py for HPUs #149499

Closed

Add Intel Gaudi device/HPU to auto load in instantiate_device_type_tests #126970

Add Intel Gaudi device/HPU to auto load in instantiate_device_type_tests #126970

Uh oh!

Conversation

ankurneog commented May 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes

Additional Context

Uh oh!

pytorch-bot bot commented May 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/126970

✅ No Failures

Uh oh!

linux-foundation-easycla bot commented May 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ankurneog commented May 24, 2024

Uh oh!

ankurneog commented Jun 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ankurneog commented Jun 11, 2024

Uh oh!

pytorch-bot bot commented Jun 11, 2024

Uh oh!

albanD left a comment

Choose a reason for hiding this comment

Uh oh!

albanD commented Jun 11, 2024

Uh oh!

pytorchmergebot commented Jun 11, 2024

Merge failed

Uh oh!

ankurneog commented Jun 11, 2024

Uh oh!

ankurneog commented Jun 11, 2024

Uh oh!

ankurneog commented Jun 11, 2024

Uh oh!

pytorchmergebot commented Jun 11, 2024

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ankurneog commented May 23, 2024 •

edited

Loading

pytorch-bot bot commented May 23, 2024 •

edited

Loading

linux-foundation-easycla bot commented May 23, 2024 •

edited

Loading

ankurneog commented Jun 4, 2024 •

edited

Loading