Make test_export training IR compatible #138517

tugsbayasgalan · 2024-10-21T22:22:16Z

Stack from ghstack (oldest at bottom):

-> Make test_export training IR compatible #138517

In this PR, I make test_export to be compatible with training IR. The idea is that when we flip the IR to non-functional training IR, all these tests should be green. The changes involve reading through the test case, and add necessary decomposition etc to make sure the tests pass. For example, if the tests expect to see mutated buffers returned, we need to get them via running run_decomp.

Differential Revision: D64732360

[ghstack-poisoned]

pytorch-bot · 2024-10-21T22:22:19Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/138517

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 651c24f with merge base 4299423 ():

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / cuda12.1-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu) (gh) (trunk failure)
##[error]Process completed with exit code 1.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: 9ef064e Pull Request resolved: #138517

[ghstack-poisoned]

ghstack-source-id: b661f3b Pull Request resolved: #138517

tugsbayasgalan · 2024-10-22T01:57:16Z

@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Differential Revision: [D64732360](https://our.internmc.facebook.com/intern/diff/D64732360) [ghstack-poisoned]

tugsbayasgalan · 2024-10-22T20:26:32Z

@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Differential Revision: [D64732360](https://our.internmc.facebook.com/intern/diff/D64732360) [ghstack-poisoned]

avikchaudhuri · 2024-10-23T17:26:09Z

test/export/test_export.py

                return x + y + z + w2

-        ep = export(M(), (torch.randn(2, 3),), strict=False)
+        ep = export(M(), (torch.randn(2, 3),), strict=False).run_decompositions({})


Why these extra run_decompositions? I guess I'm missing context on this PR, what does it mean to make these tests "training IR compatible"? Please add a summary to the PR.

The test expects to see mutated buffer as extra output. Only way we get it today is via functiionalizaiton.

tugsbayasgalan · 2024-10-23T17:29:48Z

test/export/test_export.py

        )
        check_users_for_graph(ep.graph)

-    def test_export_predispatch_custom_ops_warnings(self):


It doesn't matter anymore

tugsbayasgalan · 2024-10-23T17:30:10Z

test/export/test_export.py

                return x.to("cpu")

-        ep = export(Module(), (torch.tensor(1, device="cpu"),))
+        ep = export(Module(), (torch.tensor(1, device="cpu"),)).run_decompositions({})


Tests expect to see functional operator.

avikchaudhuri · 2024-10-23T17:57:59Z

test/export/test_export.py

        self.assertTrue(torch.allclose(unflattened(*inps), M2()(*inps)))

-    @testing.expectedFailureRetraceability  # Retracing tensor constants results in buffers
+    #@testing.expectedFailureRetraceability  # Retracing tensor constants results in buffers


So this goes away, is that good or bad? Just want to make sure that if there's still something to fix, we keep an expected failure.

The failure was actually not related to tensor constants lol. It was just some graph mismatch.

In this PR, I make test_export to be compatible with training IR. The idea is that when we flip the IR to non-functional training IR, all these tests should be green. The changes involve reading through the test case, and add necessary decomposition etc to make sure the tests pass. For example, if the tests expect to see mutated buffers returned, we need to get them via running run_decomp. Differential Revision: [D64732360](https://our.internmc.facebook.com/intern/diff/D64732360) [ghstack-poisoned]

avikchaudhuri · 2024-10-23T19:37:52Z

test/export/test_export.py

-            self.assertExpectedInline(
-                str(ep.graph_module.code).strip(),
-                """\
+        ep = export_for_training(M(), (torch.randn(2, 2),)).run_decompositions(


looks like you need to remove the xfail from here

tugsbayasgalan · 2024-10-23T19:59:48Z

@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

In this PR, I make test_export to be compatible with training IR. The idea is that when we flip the IR to non-functional training IR, all these tests should be green. The changes involve reading through the test case, and add necessary decomposition etc to make sure the tests pass. For example, if the tests expect to see mutated buffers returned, we need to get them via running run_decomp. Differential Revision: [D64732360](https://our.internmc.facebook.com/intern/diff/D64732360) [ghstack-poisoned]

ghstack-source-id: b81baae Pull Request resolved: #138517

In this PR, I make test_export to be compatible with training IR. The idea is that when we flip the IR to non-functional training IR, all these tests should be green. The changes involve reading through the test case, and add necessary decomposition etc to make sure the tests pass. For example, if the tests expect to see mutated buffers returned, we need to get them via running run_decomp. Differential Revision: [D64732360](https://our.internmc.facebook.com/intern/diff/D64732360) [ghstack-poisoned]

ghstack-source-id: ffa1ddd Pull Request resolved: #138517

tugsbayasgalan · 2024-10-28T18:18:22Z

@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2024-10-28T22:30:57Z

@pytorchbot merge

(Initiating merge automatically since Phabricator Diff has merged)

pytorchmergebot · 2024-10-28T22:32:42Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

In this PR, I make test_export to be compatible with training IR. The idea is that when we flip the IR to non-functional training IR, all these tests should be green. The changes involve reading through the test case, and add necessary decomposition etc to make sure the tests pass. For example, if the tests expect to see mutated buffers returned, we need to get them via running run_decomp. Differential Revision: [D64732360](https://our.internmc.facebook.com/intern/diff/D64732360) Pull Request resolved: pytorch#138517 Approved by: https://github.com/avikchaudhuri

Make test_export training IR compatible

cb67657

[ghstack-poisoned]

tugsbayasgalan requested review from angelayi, avikchaudhuri, ydwu4 and zhxchen17 as code owners October 21, 2024 22:22

This was referenced Oct 21, 2024

Training IR should preserve custom metadata #138266

Closed

Fix training IR bug by changing passes order #138292

Closed

tugsbayasgalan mentioned this pull request Oct 21, 2024

Move test_serialize to training IR #138261

Closed

pytorch-bot bot added the release notes: export label Oct 21, 2024

tugsbayasgalan mentioned this pull request Oct 21, 2024

Wrap autograd and autocast ops in training IR #138516

Closed

tugsbayasgalan added a commit that referenced this pull request Oct 21, 2024

Make test_export training IR compatible

3dad6b5

ghstack-source-id: 9ef064e Pull Request resolved: #138517

Update on "Make test_export training IR compatible"

0318288

[ghstack-poisoned]

tugsbayasgalan added a commit that referenced this pull request Oct 22, 2024

Make test_export training IR compatible

0b9cb38

ghstack-source-id: b661f3b Pull Request resolved: #138517

Update on "Make test_export training IR compatible"

f5e8502

Differential Revision: [D64732360](https://our.internmc.facebook.com/intern/diff/D64732360) [ghstack-poisoned]

Update on "Make test_export training IR compatible"

2e4195a

Differential Revision: [D64732360](https://our.internmc.facebook.com/intern/diff/D64732360) [ghstack-poisoned]

pytorch-bot bot added the topic: not user facing topic category label Oct 22, 2024

tugsbayasgalan mentioned this pull request Oct 23, 2024

Make test_torchbind.py training IR compatible #138658

Closed

Update on "Make test_export training IR compatible"

e014498

Differential Revision: [D64732360](https://our.internmc.facebook.com/intern/diff/D64732360) [ghstack-poisoned]

avikchaudhuri reviewed Oct 23, 2024

View reviewed changes

tugsbayasgalan commented Oct 23, 2024

View reviewed changes

avikchaudhuri reviewed Oct 23, 2024

View reviewed changes

tugsbayasgalan requested a review from avikchaudhuri October 23, 2024 18:12

avikchaudhuri approved these changes Oct 23, 2024

View reviewed changes

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Oct 23, 2024

tugsbayasgalan added a commit that referenced this pull request Oct 23, 2024

Make test_export training IR compatible

3eaa76d

ghstack-source-id: b81baae Pull Request resolved: #138517

tugsbayasgalan added a commit that referenced this pull request Oct 28, 2024

Make test_export training IR compatible

8b4e40c

ghstack-source-id: ffa1ddd Pull Request resolved: #138517

pytorchmergebot added the merging label Oct 28, 2024

pytorchmergebot added the Merged label Oct 28, 2024

pytorchmergebot closed this in b7524b0 Oct 28, 2024

pytorchmergebot removed the merging label Oct 28, 2024

github-actions bot deleted the gh/tugsbayasgalan/266/head branch November 28, 2024 02:15

Make test_export training IR compatible #138517

Make test_export training IR compatible #138517

Uh oh!

Conversation

tugsbayasgalan commented Oct 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Oct 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/138517

✅ You can merge normally! (1 Unrelated Failure)

Uh oh!

tugsbayasgalan commented Oct 22, 2024

Uh oh!

tugsbayasgalan commented Oct 22, 2024

Uh oh!

avikchaudhuri Oct 23, 2024

Choose a reason for hiding this comment

Uh oh!

tugsbayasgalan Oct 23, 2024

Choose a reason for hiding this comment

Uh oh!

tugsbayasgalan Oct 23, 2024

Choose a reason for hiding this comment

Uh oh!

tugsbayasgalan Oct 23, 2024

Choose a reason for hiding this comment

Uh oh!

avikchaudhuri Oct 23, 2024

Choose a reason for hiding this comment

Uh oh!

tugsbayasgalan Oct 23, 2024

Choose a reason for hiding this comment

Uh oh!

avikchaudhuri Oct 23, 2024

Choose a reason for hiding this comment

Uh oh!

tugsbayasgalan commented Oct 23, 2024

Uh oh!

tugsbayasgalan commented Oct 28, 2024

Uh oh!

facebook-github-bot commented Oct 28, 2024

Uh oh!

pytorchmergebot commented Oct 28, 2024

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

tugsbayasgalan commented Oct 21, 2024 •

edited

Loading

pytorch-bot bot commented Oct 21, 2024 •

edited

Loading