Skip to content

Fakify leaf of FunctionalTensor #107062

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ydwu4 wants to merge 17 commits into from
Closed

Fakify leaf of FunctionalTensor #107062

ydwu4 wants to merge 17 commits into from

Conversation

@ydwu4
Copy link
Contributor

@ydwu4 ydwu4 commented Aug 11, 2023 •
edited
Loading

Stack from ghstack (oldest at bottom):

This PR allows dynamo to fakify FunctionalTensorWrapper by unwrapping, replacing and wrapping again for FunctionalTensorWrapper so that FunctionalTensorWrapper can be passed in as input for dynamo.optimize and we can support something like this

ff = torch.func.functionalize(f)
torch.compile(ff)(x)

This PR didn't follow the __tensor_flatten__ and __tensor_unflatten__ protocol right now because we're not sure the plan of doing that for FunctionalTensorWrapper (it's implemented in C++).

Test Plan:
Add a new test.

cc @voznesenskym @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @aakhundov

@ydwu4 ydwu4 mentioned this pull request Aug 11, 2023
Closed
@pytorch-bot
Copy link

pytorch-bot bot commented Aug 11, 2023 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/107062

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

✅ 1 Unrelated Failure

As of commit 8cb30b9 with merge base e1ee10e (image):

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

zou3519
zou3519 reviewed Aug 11, 2023
View reviewed changes
@ydwu4
Update on "Fakify leaf of FunctionalTensor"
17c6927 
cc voznesenskym penguinwu anijain2305 EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx ipiszy chenyang78 aakhundov

[ghstack-poisoned]
@ydwu4
Update on "Fakify leaf of FunctionalTensor"
c19c325 
cc voznesenskym penguinwu anijain2305 EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx ipiszy chenyang78 aakhundov

[ghstack-poisoned]
ydwu4 added a commit that referenced this pull request Aug 11, 2023
@ydwu4
Fakify leaf of FunctionalTensor
131620b 
ghstack-source-id: 29205cf
Pull Request resolved: #107062
@ydwu4
Update on "Fakify leaf of FunctionalTensor"
c94fbfd 
cc voznesenskym penguinwu anijain2305 EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx ipiszy chenyang78 aakhundov

[ghstack-poisoned]
@ydwu4
Update on "Fakify leaf of FunctionalTensor"
3c7b466 
cc voznesenskym penguinwu anijain2305 EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx ipiszy chenyang78 aakhundov

[ghstack-poisoned]
ydwu4 added a commit that referenced this pull request Aug 14, 2023
@ydwu4
Fakify leaf of FunctionalTensor
a88bd12 
ghstack-source-id: 0d53c20
Pull Request resolved: #107062
@ydwu4
Update on "Fakify leaf of FunctionalTensor"
9c66183 
cc voznesenskym penguinwu anijain2305 EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx ipiszy chenyang78 aakhundov

[ghstack-poisoned]
@ydwu4
Update on "Fakify leaf of FunctionalTensor"
fd9d570 
cc voznesenskym penguinwu anijain2305 EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx ipiszy chenyang78 aakhundov

[ghstack-poisoned]
@ydwu4
Update on "Fakify leaf of FunctionalTensor"
fe22510 
cc voznesenskym penguinwu anijain2305 EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx ipiszy chenyang78 aakhundov

[ghstack-poisoned]
ydwu4 added a commit that referenced this pull request Aug 14, 2023
@ydwu4
Fakify leaf of FunctionalTensor
49ba357 
ghstack-source-id: 13df33b
Pull Request resolved: #107062
@ydwu4 ydwu4 added the ciflow/trunk Trigger trunk jobs on your pull request label Aug 14, 2023
@ydwu4
Update on "Fakify leaf of FunctionalTensor"
6a5b1f7 
cc voznesenskym penguinwu anijain2305 EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx ipiszy chenyang78 aakhundov

[ghstack-poisoned]
@ydwu4
Update on "Fakify leaf of FunctionalTensor"
034eabc 
cc voznesenskym penguinwu anijain2305 EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx ipiszy chenyang78 aakhundov

[ghstack-poisoned]
ydwu4 added a commit that referenced this pull request Aug 15, 2023
@ydwu4
Fakify leaf of FunctionalTensor
be433a9 
ghstack-source-id: 7122511
Pull Request resolved: #107062
@ydwu4 ydwu4 added the topic: not user facing topic category label Aug 15, 2023
@ydwu4
Update on "Fakify leaf of FunctionalTensor"
ab9e364 
cc voznesenskym penguinwu anijain2305 EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx ipiszy chenyang78 aakhundov

[ghstack-poisoned]
@ydwu4
Update on "Fakify leaf of FunctionalTensor"
e6ed211 
cc voznesenskym penguinwu anijain2305 EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx ipiszy chenyang78 aakhundov

[ghstack-poisoned]
@ydwu4
Copy link
Contributor Author

ydwu4 commented Aug 18, 2023

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Aug 18, 2023

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@ZainRizvi
Copy link
Contributor

ZainRizvi commented Aug 18, 2023

@pytorchbot revert -m "This appears to have broken the test TestDTensorCompile.test_dtensor_fullgraph. Probably a land race" -c landrace

@ZainRizvi
Copy link
Contributor

ZainRizvi commented Aug 18, 2023

See logs here for more details:
https://hud.pytorch.org/pytorch/pytorch/commit/6cb0128c8a07d626ab84516df3c9727943469d49

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Aug 18, 2023

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Aug 18, 2023

@ydwu4 your PR has been successfully reverted.

pytorchmergebot added a commit that referenced this pull request Aug 18, 2023
@pytorchmergebot
Revert "Fakify leaf of FunctionalTensor (#107062)"
3c11184 
This reverts commit 6cb0128.

Reverted #107062 on behalf of https://github.com/ZainRizvi due to This appears to have broken the test TestDTensorCompile.test_dtensor_fullgraph.  Probably a land race ([comment](#107062 (comment)))
@ydwu4 ydwu4 reopened this Aug 18, 2023
@ydwu4
Copy link
Contributor Author

ydwu4 commented Aug 19, 2023

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Aug 19, 2023

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@ydwu4
Copy link
Contributor Author

ydwu4 commented Aug 21, 2023

@pytorchbot revert -m "This appears to have broken the test TestDTensorCompile.test_dtensor_fullgraph."

@pytorch-bot
Copy link

pytorch-bot bot commented Aug 21, 2023

❌ 🤖 pytorchbot command failed:

@pytorchbot revert: error: the following arguments are required: -c/--classification

usage: @pytorchbot revert -m MESSAGE -c
                          {nosignal,ignoredsignal,landrace,weird,ghfirst}

Try @pytorchbot --help for more info.

@ydwu4
Copy link
Contributor Author

ydwu4 commented Aug 21, 2023

@pytorchbot revert -m "This appears to have broken the test TestDTensorCompile.test_dtensor_fullgraph. Probably a land race" -c landrace

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Aug 21, 2023

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Aug 21, 2023

@ydwu4 your PR has been successfully reverted.

pytorchmergebot added a commit that referenced this pull request Aug 21, 2023
@pytorchmergebot
Revert "Fakify leaf of FunctionalTensor (#107062)"
96c5be8 
This reverts commit 3349725.

Reverted #107062 on behalf of https://github.com/ydwu4 due to This appears to have broken the test TestDTensorCompile.test_dtensor_fullgraph. Probably a land race ([comment](#107062 (comment)))
@ydwu4 ydwu4 mentioned this pull request Aug 21, 2023
Closed
pytorchmergebot pushed a commit that referenced this pull request Aug 22, 2023
@ydwu4 @pytorchmergebot
Reland fakify FunctionalTensor (#107569)
a408920 
Try to rebase and reland #107062 . One difference compared with previous is to make the DTensor logic same as previously in _clone_input.

Pull Request resolved: #107569
Approved by: https://github.com/zou3519
@facebook-github-bot facebook-github-bot deleted the gh/ydwu4/2/head branch August 23, 2023 14:17
@ezyang ezyang mentioned this pull request Mar 19, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zou3519 zou3519 zou3519 approved these changes

@bdhirsh bdhirsh bdhirsh left review comments

@ezyang ezyang Awaiting requested review from ezyang

@voznesenskym voznesenskym Awaiting requested review from voznesenskym

Assignees

No one assigned

Labels

ciflow/inductor ciflow/trunk Trigger trunk jobs on your pull request Merged module: dynamo Reverted topic: not user facing topic category

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@ydwu4 @pytorchmergebot @ZainRizvi @zou3519 @bdhirsh