[aoti] Add load_constants to package api #142246
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/142246
Note: Links to docs will display an error until the docs builds have been completed. ❌ 2 New Failures, 1 Cancelled Job, 3 Unrelated FailuresAs of commit 4941079 with merge base 61a7c83 ( NEW FAILURES - The following jobs have failed:
CANCELLED JOB - The following job was cancelled. Please retry:
FLAKY - The following jobs failed but were likely due to flakiness present on trunk:
BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
This pull request was exported from Phabricator. Differential Revision: D66796206 |
✅ Deploy Preview for chimerical-cranachan-793287 ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
angelayi
added
release notes: inductor
and removed
topic: not user facing
|
This pull request was exported from Phabricator. Differential Revision: D66796206 |
angelayi
force-pushed
the
export-D66796206
branch
from
7bd4fd5 to
7806ea9
Compare
Summary: Pull Request resolved: pytorch#142246 With the changes in pytorch#140755 and pytorch#141997, I added a load_constants function to the packaging API. Currently this doesn't work for cpu. The workflow is something like: ``` ep = torch.export.export(model, example_inputs) package = torch._inductor.aoti_compile_and_package(ep, inductor_configs=inductor_configs) compiled = torch._inductor.aoti_load_package(package) print(compiled.get_constant_fqns()) # see what are the fqns needed/available compiled.load_constants(new_state_dict, check_full_update=True) # update the constants in AOTI ``` You can also use the `aot_inductor.package_constants_in_so` config to stop including the constants in the so: ``` package = torch._inductor.aoti_compile_and_package(ep, inductor_configs={`aot_inductor.package_constants_in_so`: False) compiled = torch._inductor.aoti_load_package(package) compiled(*inputs) # segfaults because there are no constants --> we should probably have a better error msg compiled.load_constants(new_state_dict, check_full_update=True) compiled(*inputs) ``` Test Plan: `buck2 run @//mode/dev-nosan //caffe2/test/inductor:aot_inductor_package -- -r "test_so_without_weight" ` Reviewed By: henrylhtsang Differential Revision: D66796206
|
This pull request was exported from Phabricator. Differential Revision: D66796206 |
angelayi
force-pushed
the
export-D66796206
branch
from
7806ea9 to
4941079
Compare
|
@pytorchbot merge (Initiating merge automatically since Phabricator Diff has merged) |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 3 jobs have failed, first few of them are: inductor / cuda12.4-py3.10-gcc9-sm86 / build, inductor / cuda12.1-py3.10-gcc9-sm86 / build, inductor / unit-test / cuda12.1-py3.12-gcc9-sm86 / build Details for Dev Infra teamRaised by workflow job |
|
@pytorchbot merge -i "landed internally" |
|
❌ 🤖 pytorchbot command failed: Try |
|
@pytorchbot merge -i |
Merge startedYour change will be merged while ignoring the following 6 checks: inductor-rocm / rocm6.2-py3.10-inductor / test (inductor, 1, 2, linux.rocm.gpu.2), inductor-rocm / rocm6.2-py3.10-inductor / test (inductor, 2, 2, linux.rocm.gpu.2), inductor / cuda12.4-py3.10-gcc9-sm86 / build, inductor / cuda12.1-py3.10-gcc9-sm86 / build, inductor / unit-test / cuda12.1-py3.10-gcc9-sm86 / build, inductor / unit-test / cuda12.1-py3.12-gcc9-sm86 / build Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Summary: With the changes in pytorch#140755 and pytorch#141997, I added a load_constants function to the packaging API. Currently this doesn't work for cpu. The workflow is something like: ``` ep = torch.export.export(model, example_inputs) package = torch._inductor.aoti_compile_and_package(ep, inductor_configs=inductor_configs) compiled = torch._inductor.aoti_load_package(package) print(compiled.get_constant_fqns()) # see what are the fqns needed/available compiled.load_constants(new_state_dict, check_full_update=True) # update the constants in AOTI ``` You can also use the `aot_inductor.package_constants_in_so` config to stop including the constants in the so: ``` package = torch._inductor.aoti_compile_and_package(ep, inductor_configs={`aot_inductor.package_constants_in_so`: False) compiled = torch._inductor.aoti_load_package(package) compiled(*inputs) # segfaults because there are no constants --> we should probably have a better error msg compiled.load_constants(new_state_dict, check_full_update=True) compiled(*inputs) ``` Test Plan: `buck2 run @//mode/dev-nosan //caffe2/test/inductor:aot_inductor_package -- -r "test_so_without_weight" ` Differential Revision: D66796206 Pull Request resolved: pytorch#142246 Approved by: https://github.com/henrylhtsang, https://github.com/desertfire
5 participants
Summary:
With the changes in #140755 and #141997, I added a load_constants function to the packaging API. Currently this doesn't work for cpu.
The workflow is something like:
You can also use the
aot_inductor.package_constants_in_soconfig to stop including the constants in the so:Test Plan:
buck2 run @//mode/dev-nosan //caffe2/test/inductor:aot_inductor_package -- -r "test_so_without_weight"Differential Revision: D66796206
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @chauhang @aakhundov