Skip to content

Commit 1a08c82

Browse files
ezyangezyang
committed
Update on "Move most methods off Variable into torch::autograd::impl functions."
Our intention is to merge the static distinction between Tensor and Variable. Ordinarily, this would entail merging the methods of Tensor and Variable. But there are a lot of "private"-ish methods on Variable that we don't actually want to dump onto the Tensor class. So, as prep work, we move all of those methods off of Variable and into the torch::autograd::impl namespace (impl as in, please don't use this end users). This ends up being a fairly large patch because all of the call sites have to play ball too. While I was on the topic, I also moved any of the touched functions into the C++ file, so that modifying them would not trigger a recompilation of all of torch. Signed-off-by: Edward Z. Yang <[email protected]> [ghstack-poisoned]
2 parents 72d6131 + 3be6674 commit 1a08c82

File tree

207 files changed

+5925
-3768
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

207 files changed

+5925
-3768
lines changed

‎.circleci/cimodel/data/pytorch_build_definitions.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313

1414
# ARE YOU EDITING THIS NUMBER? MAKE SURE YOU READ THE GUIDANCE AT THE
1515
# TOP OF .circleci/config.yml
16-
DOCKER_IMAGE_VERSION = 347
16+
DOCKER_IMAGE_VERSION = 389
1717

1818

1919
@dataclass

‎.circleci/config.yml‎

Lines changed: 97 additions & 79 deletions
Large diffs are not rendered by default.

‎.circleci/generate_config_yml.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,7 @@ def write(self, output_filehandle):
9393
File("workflows-pytorch-macos-builds.yml"),
9494
File("workflows-pytorch-android-gradle-build.yml"),
9595
File("workflows-pytorch-ios-builds.yml"),
96+
File("workflows-pytorch-ge-config-tests.yml"),
9697
Listgen(caffe2_build_definitions.get_workflow_jobs, 3),
9798
File("workflows-binary-builds-smoke-subset.yml"),
9899
Listgen(binary_build_definitions.get_binary_smoke_test_jobs, 3),

‎.circleci/scripts/should_run_job.py‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,8 @@
5454
# PyTorch Android
5555
'pytorch-linux-xenial-py3-clang5-android-ndk-r19c-x86_32-build',
5656
# PyTorch Android gradle
57-
'pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-build-only-x86_32',
57+
# disabled until fixing https://github.com/pytorch/pytorch/issues/29159
58+
#'pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-build-only-x86_32',
5859
# Pytorch iOS builds
5960
'pytorch-ios-10.2.1-x86_64_build',
6061
'pytorch-ios-10.2.1-arm64_build',
@@ -65,6 +66,10 @@
6566
# XLA
6667
'pytorch-xla-linux-xenial-py3.6-clang7',
6768

69+
# GraphExecutor config jobs
70+
'pytorch-linux-xenial-py3.6-gcc5.4-ge_config_simple-test',
71+
'pytorch-linux-xenial-py3.6-gcc5.4-ge_config_legacy-test',
72+
6873
# Other checks
6974
'pytorch-short-perf-test-gpu',
7075
'pytorch-python-doc-push',

‎.circleci/verbatim-sources/job-specs-custom.yml‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
environment:
33
BUILD_ENVIRONMENT: pytorch-python-doc-push
44
# TODO: stop hardcoding this
5-
DOCKER_IMAGE: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-cuda9-cudnn7-py3:347"
5+
DOCKER_IMAGE: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-cuda9-cudnn7-py3:389"
66
resource_class: large
77
machine:
88
image: ubuntu-1604:201903-01
@@ -47,7 +47,7 @@
4747
pytorch_cpp_doc_push:
4848
environment:
4949
BUILD_ENVIRONMENT: pytorch-cpp-doc-push
50-
DOCKER_IMAGE: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-cuda9-cudnn7-py3:347"
50+
DOCKER_IMAGE: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-cuda9-cudnn7-py3:389"
5151
resource_class: large
5252
machine:
5353
image: ubuntu-1604:201903-01
@@ -203,7 +203,7 @@
203203
pytorch_android_gradle_build:
204204
environment:
205205
BUILD_ENVIRONMENT: pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-build
206-
DOCKER_IMAGE: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-py3-clang5-android-ndk-r19c:347"
206+
DOCKER_IMAGE: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-py3-clang5-android-ndk-r19c:389"
207207
PYTHON_VERSION: "3.6"
208208
resource_class: large
209209
machine:
@@ -289,7 +289,7 @@
289289
pytorch_android_publish_snapshot:
290290
environment:
291291
BUILD_ENVIRONMENT: pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-publish-snapshot
292-
DOCKER_IMAGE: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-py3-clang5-android-ndk-r19c:347"
292+
DOCKER_IMAGE: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-py3-clang5-android-ndk-r19c:389"
293293
PYTHON_VERSION: "3.6"
294294
resource_class: large
295295
machine:
@@ -325,7 +325,7 @@
325325
pytorch_android_gradle_build-x86_32:
326326
environment:
327327
BUILD_ENVIRONMENT: pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-build-only-x86_32
328-
DOCKER_IMAGE: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-py3-clang5-android-ndk-r19c:347"
328+
DOCKER_IMAGE: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-py3-clang5-android-ndk-r19c:389"
329329
PYTHON_VERSION: "3.6"
330330
resource_class: large
331331
machine:

‎.circleci/verbatim-sources/workflows-nightly-android-binary-builds.yml‎

Lines changed: 24 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
build_environment: "pytorch-linux-xenial-py3-clang5-android-ndk-r19c-x86_32"
44
requires:
55
- setup
6-
docker_image: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-py3-clang5-android-ndk-r19c:347"
6+
docker_image: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-py3-clang5-android-ndk-r19c:389"
77
filters:
88
branches:
99
only: nightly
@@ -12,7 +12,7 @@
1212
build_environment: "pytorch-linux-xenial-py3-clang5-android-ndk-r19c-x86_64"
1313
requires:
1414
- setup
15-
docker_image: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-py3-clang5-android-ndk-r19c:347"
15+
docker_image: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-py3-clang5-android-ndk-r19c:389"
1616
filters:
1717
branches:
1818
only: nightly
@@ -21,7 +21,7 @@
2121
build_environment: "pytorch-linux-xenial-py3-clang5-android-ndk-r19c-arm-v7a"
2222
requires:
2323
- setup
24-
docker_image: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-py3-clang5-android-ndk-r19c:347"
24+
docker_image: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-py3-clang5-android-ndk-r19c:389"
2525
filters:
2626
branches:
2727
only: nightly
@@ -30,27 +30,28 @@
3030
build_environment: "pytorch-linux-xenial-py3-clang5-android-ndk-r19c-arm-v8a"
3131
requires:
3232
- setup
33-
docker_image: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-py3-clang5-android-ndk-r19c:347"
33+
docker_image: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-py3-clang5-android-ndk-r19c:389"
3434
filters:
3535
branches:
3636
only: nightly
3737

38-
- pytorch_android_gradle_build:
39-
name: nightly_pytorch_linux_xenial_py3_clang5_android_ndk_r19c_android_gradle_build
40-
requires:
41-
- nightly_pytorch_linux_xenial_py3_clang5_android_ndk_r19c_x86_32_build
42-
- nightly_pytorch_linux_xenial_py3_clang5_android_ndk_r19c_x86_64_build
43-
- nightly_pytorch_linux_xenial_py3_clang5_android_ndk_r19c_arm_v7a_build
44-
- nightly_pytorch_linux_xenial_py3_clang5_android_ndk_r19c_arm_v8a_build
45-
filters:
46-
branches:
47-
only: nightly
48-
49-
- pytorch_android_publish_snapshot:
50-
name: nightly_pytorch_linux_xenial_py3_clang5_android_ndk_r19c_x86_32_android_publish_snapshot
51-
requires:
52-
- nightly_pytorch_linux_xenial_py3_clang5_android_ndk_r19c_android_gradle_build
53-
context: org-member
54-
filters:
55-
branches:
56-
only: nightly
38+
# disabled until fixing https://github.com/pytorch/pytorch/issues/29159
39+
# - pytorch_android_gradle_build:
40+
# name: nightly_pytorch_linux_xenial_py3_clang5_android_ndk_r19c_android_gradle_build
41+
# requires:
42+
# - nightly_pytorch_linux_xenial_py3_clang5_android_ndk_r19c_x86_32_build
43+
# - nightly_pytorch_linux_xenial_py3_clang5_android_ndk_r19c_x86_64_build
44+
# - nightly_pytorch_linux_xenial_py3_clang5_android_ndk_r19c_arm_v7a_build
45+
# - nightly_pytorch_linux_xenial_py3_clang5_android_ndk_r19c_arm_v8a_build
46+
# filters:
47+
# branches:
48+
# only: nightly
49+
#
50+
# - pytorch_android_publish_snapshot:
51+
# name: nightly_pytorch_linux_xenial_py3_clang5_android_ndk_r19c_x86_32_android_publish_snapshot
52+
# requires:
53+
# - nightly_pytorch_linux_xenial_py3_clang5_android_ndk_r19c_android_gradle_build
54+
# context: org-member
55+
# filters:
56+
# branches:
57+
# only: nightly

‎.circleci/verbatim-sources/workflows-pytorch-android-gradle-build.yml‎

Lines changed: 13 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,13 @@
1-
- pytorch_android_gradle_build-x86_32:
2-
name: pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-build-x86_32
3-
requires:
4-
- pytorch_linux_xenial_py3_clang5_android_ndk_r19c_x86_32_build
5-
6-
- pytorch_android_gradle_build:
7-
name: pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-build
8-
requires:
9-
- pytorch_linux_xenial_py3_clang5_android_ndk_r19c_x86_32_build
10-
- pytorch_linux_xenial_py3_clang5_android_ndk_r19c_x86_64_build
11-
- pytorch_linux_xenial_py3_clang5_android_ndk_r19c_arm_v7a_build
12-
- pytorch_linux_xenial_py3_clang5_android_ndk_r19c_arm_v8a_build
1+
# disabled until fixing https://github.com/pytorch/pytorch/issues/29159
2+
# - pytorch_android_gradle_build-x86_32:
3+
# name: pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-build-x86_32
4+
# requires:
5+
# - pytorch_linux_xenial_py3_clang5_android_ndk_r19c_x86_32_build
6+
#
7+
# - pytorch_android_gradle_build:
8+
# name: pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-build
9+
# requires:
10+
# - pytorch_linux_xenial_py3_clang5_android_ndk_r19c_x86_32_build
11+
# - pytorch_linux_xenial_py3_clang5_android_ndk_r19c_x86_64_build
12+
# - pytorch_linux_xenial_py3_clang5_android_ndk_r19c_arm_v7a_build
13+
# - pytorch_linux_xenial_py3_clang5_android_ndk_r19c_arm_v8a_build

‎.circleci/verbatim-sources/workflows-pytorch-ge-config-tests.yml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
- pytorch_linux_test:
2+
name: pytorch_linux_xenial_py3_6_gcc5_4_ge_config_legacy_test
3+
requires:
4+
- setup
5+
- pytorch_linux_xenial_py3_6_gcc5_4_build
6+
build_environment: "pytorch-linux-xenial-py3.6-gcc5.4-ge_config_legacy-test"
7+
docker_image: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-py3.6-gcc5.4:389"
8+
resource_class: large
9+
- pytorch_linux_test:
10+
name: pytorch_linux_xenial_py3_6_gcc5_4_ge_config_simple_test
11+
requires:
12+
- setup
13+
- pytorch_linux_xenial_py3_6_gcc5_4_build
14+
build_environment: "pytorch-linux-xenial-py3.6-gcc5.4-ge_config_simple-test"
15+
docker_image: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-py3.6-gcc5.4:389"
16+
resource_class: large

‎.jenkins/pytorch/test.sh‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -109,8 +109,18 @@ test_python_nn() {
109109
assert_git_not_dirty
110110
}
111111

112+
test_python_ge_config_simple() {
113+
time python test/run_test.py --include jit_simple --verbose
114+
assert_git_not_dirty
115+
}
116+
117+
test_python_ge_config_legacy() {
118+
time python test/run_test.py --include jit_legacy jit_fuser_legacy --verbose
119+
assert_git_not_dirty
120+
}
121+
112122
test_python_all_except_nn() {
113-
time python test/run_test.py --exclude nn --verbose --bring-to-front quantization quantized quantized_tensor quantized_nn_mods
123+
time python test/run_test.py --exclude nn jit_simple jit_legacy jit_fuser_legacy --verbose --bring-to-front quantization quantized quantized_tensor quantized_nn_mods
114124
assert_git_not_dirty
115125
}
116126

@@ -219,6 +229,10 @@ if [[ "${BUILD_ENVIRONMENT}" == *backward* ]]; then
219229
elif [[ "${BUILD_ENVIRONMENT}" == *xla* || "${JOB_BASE_NAME}" == *xla* ]]; then
220230
test_torchvision
221231
test_xla
232+
elif [[ "${BUILD_ENVIRONMENT}" == *ge_config_legacy* || "${JOB_BASE_NAME}" == *ge_config_legacy* ]]; then
233+
test_python_ge_config_legacy
234+
elif [[ "${BUILD_ENVIRONMENT}" == *ge_config_simple* || "${JOB_BASE_NAME}" == *ge_config_simple* ]]; then
235+
test_python_ge_config_simple
222236
elif [[ "${BUILD_ENVIRONMENT}" == *libtorch* ]]; then
223237
# TODO: run some C++ tests
224238
echo "no-op at the moment"

0 commit comments

Comments
 (0)