Skip to content

Commit cbdcef1

Browse files
committed
Update base for Update on "Move all torch.nn.modules type annotations inline"
Just because the annotations are inline doesn't mean the files type check; most of the newly annotated files have type errors and I added exclusions for them in mypy.ini. The payoff of moving all of these modules inline is I can delete the relevant code generation logic for the pyi files (which was added ignore annotations that weren't actually relevant anymore.) Because we aren't actually typechecking these modules in most cases, it is inevitable that some of these type annotations are wrong. I slavishly copied the old annotations from the pyi files unless there was an obvious correction I could make. These annotations will probably need fixing up later. Moving these annotations inline was really hairy because of interactions with JIT, and also the fact that Python type erasure is a lie (inheriting from Generic *does* change the behavior of your object). Here is the list of things I had to fix and/or work around: - The quantization translation passes previously barfed if the weight/bias arguments were inferred to be Optional. Previously, TorchScript type inference would have inferred that these arguments were non-Optional (because type inference happens after module construction), but accurate type annotations on these parameters override this inference process, causing the arguments to be optional. I fixed this by making the quantized operator signatures line up exactly with the non-quantized signatures, so we never change the types of the arguments. This change involved mostly making a bunch of quantized kernels take optional, and then error if they were passed nullopt. (You can have any color you like, as long as it's non-null.) - I removed Generic support for Module and ModuleList. The intentions behind this were admirable, but making Module inherit from Generic ended up being more headache than it was worth. First, in Python 3.6 and earlier, Generic has a nontrivial metaclass, which means all subsequent metaclass shenanigans (e.g., ScriptModule) need to line up the right metaclass. Second, Generic defines `__new__` specially, which means that `inspect.signature` doesn't work (see https://bugs.python.org/issue40897), and I found a case of people using precisely this in the wild. Between these two problems, and also the general problem which is that the parametrization here is an incomplete fix (parametrization helps with output typing, but it doesn't solve problems with input typing (and with mypy as it stands this is unfixable, see python/mypy#3028) I decided to just eliminate Module generics entirely. We still apply the Callable trick so that subclasses of Module don't cause mypy to complain, but otherwise you are on your own for getting accurate type information out of Modules. - The `Callable` trick on `forward` caused TorchScript to stop performing inference on the forward body, which is bad because in general we can only figure out the most accurate type by doing TorchScript inference. I added a special case to `infer_type` to ensure we always do inference for `Module.forward`, even if it is annotated (which it is), and another special case to make sure we ignore references to Callable (which we shouldn't process) recursively. - When `__annotations__` is set on a class (as is the case when you add type annotations), JIT will incorrectly add further annotations to the parent class. This PR fixes #39463 by testing if `__annotations__` is defined on the specific class, excluding parent classes from the test. - Added a missing fake source range to the invocation of `get_signature` - In some cases, we cannot provide accurate typing for parameters on modules. This usually occurs when you have an `Optional[Tensor]` parameter, whose optional-ness is determined at `__init__` time. Without the annotation, TorchScript will infer the correct refined type depending on arguments to the constructor, but with the annotation, it will never do a refinement at `__init__` time, and you'll end up with the wrong type. I ended up just straight up deleting type annotations in all of these cases. A more robust fix might be to make some way to force TorchScript to do inference even if there is an explicit annotation, in case of refinement. Signed-off-by: Edward Z. Yang <[email protected]> Differential Revision: [D21497397](https://our.internmc.facebook.com/intern/diff/D21497397) [ghstack-poisoned]
2 parents 2ea2b8d + a212513 commit cbdcef1

58 files changed

Lines changed: 2599 additions & 806 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.circleci/config.yml‎

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1047,10 +1047,7 @@ jobs:
10471047
machine:
10481048
image: ubuntu-1604:201903-01
10491049
steps:
1050-
- attach_workspace:
1051-
at: ~/workspace
1052-
- attach_workspace:
1053-
at: /home/circleci/project
1050+
- checkout
10541051
- setup_linux_system_environment
10551052
- setup_ci_environment
10561053
- run:
@@ -1076,10 +1073,7 @@ jobs:
10761073
macos:
10771074
xcode: "9.4.1"
10781075
steps:
1079-
- attach_workspace:
1080-
at: ~/workspace
1081-
- attach_workspace: # TODO - we can `cp` from ~/workspace
1082-
at: /Users/distiller/project
1076+
- checkout
10831077
- run:
10841078
<<: *binary_checkout
10851079
- run:

‎.circleci/verbatim-sources/job-specs/binary-job-specs.yml‎

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -113,10 +113,7 @@
113113
machine:
114114
image: ubuntu-1604:201903-01
115115
steps:
116-
- attach_workspace:
117-
at: ~/workspace
118-
- attach_workspace:
119-
at: /home/circleci/project
116+
- checkout
120117
- setup_linux_system_environment
121118
- setup_ci_environment
122119
- run:
@@ -142,10 +139,7 @@
142139
macos:
143140
xcode: "9.4.1"
144141
steps:
145-
- attach_workspace:
146-
at: ~/workspace
147-
- attach_workspace: # TODO - we can `cp` from ~/workspace
148-
at: /Users/distiller/project
142+
- checkout
149143
- run:
150144
<<: *binary_checkout
151145
- run:

‎aten/src/ATen/core/List_inl.h‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,8 @@ List<T>::List(TypePtr elementType)
4343
: List(make_intrusive<c10::detail::ListImpl>(
4444
typename c10::detail::ListImpl::list_type(),
4545
std::move(elementType))) {
46-
static_assert(std::is_same<T, IValue>::value, "This constructor is only valid for c10::impl::GenericList.");
46+
static_assert(std::is_same<T, IValue>::value || std::is_same<T, c10::intrusive_ptr<ivalue::Future>>::value,
47+
"This constructor is only valid for c10::impl::GenericList or List<Future>.");
4748
}
4849

4950
namespace impl {

‎aten/src/ATen/core/ivalue.cpp‎

Lines changed: 48 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,7 @@ TypePtr IValue::type() const {
7979
case Tag::GenericList:
8080
return ListType::create(toList().elementType());
8181
case Tag::Future:
82-
return toFuture()->type();
82+
return FutureType::create(toFuture()->elementType());
8383
case Tag::RRef:
8484
return RRefType::create(toRRef()->type());
8585
case Tag::Device:
@@ -629,4 +629,51 @@ getClassConverter() {
629629
classConverter;
630630
return classConverter;
631631
}
632+
633+
CAFFE2_API intrusive_ptr<ivalue::Future> collectAll(
634+
List<intrusive_ptr<ivalue::Future>> srcs) {
635+
struct Ctx {
636+
explicit Ctx(List<intrusive_ptr<ivalue::Future>> srcs)
637+
: remaining(srcs.size()),
638+
srcFutures(std::move(srcs)),
639+
asIvalue(srcFutures),
640+
dstFuture(make_intrusive<ivalue::Future>(asIvalue.type())) {}
641+
std::atomic<int32_t> remaining{0};
642+
List<intrusive_ptr<ivalue::Future>> srcFutures;
643+
IValue asIvalue;
644+
intrusive_ptr<ivalue::Future> dstFuture;
645+
};
646+
647+
auto ctx = std::make_shared<Ctx>(std::move(srcs));
648+
std::function<void()> func = [ctx]() {
649+
if (--ctx->remaining == 0) {
650+
ctx->dstFuture->markCompleted(ctx->asIvalue);
651+
}
652+
};
653+
for (int32_t tot = ctx->srcFutures.size(), i = 0; i < tot; ++i) {
654+
ctx->srcFutures.get(i)->addCallback(func);
655+
}
656+
if (ctx->srcFutures.size() == 0) {
657+
ctx->dstFuture->markCompleted(ctx->asIvalue);
658+
}
659+
return ctx->dstFuture;
660+
}
661+
662+
CAFFE2_API intrusive_ptr<ivalue::Future> collectAll(
663+
std::vector<intrusive_ptr<ivalue::Future>> srcs) {
664+
auto typePtr = srcs.empty() ? AnyType::get() : srcs[0]->elementType();
665+
// Lists are generally expected to have a consistent type, check
666+
// for this unless we have a compelling target use case.
667+
for (size_t i = 1, len = srcs.size(); i < len; ++i) {
668+
TORCH_CHECK(*srcs[i]->elementType() == *typePtr,
669+
"Expected ", typePtr->str(), " type in list but saw ",
670+
srcs[i]->elementType()->str());
671+
}
672+
List<intrusive_ptr<ivalue::Future>> asList(FutureType::create(typePtr));
673+
asList.reserve(srcs.size());
674+
for (auto&& s : srcs) {
675+
asList.push_back(std::move(s));
676+
}
677+
return collectAll(asList);
678+
}
632679
} // namespace c10

‎aten/src/ATen/core/ivalue_inl.h‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -230,7 +230,7 @@ struct C10_EXPORT ivalue::Future final : c10::intrusive_ptr_target {
230230
}
231231

232232
public:
233-
Future(TypePtr type) : type_(type) {}
233+
explicit Future(TypePtr type) : type_(type) {}
234234
struct CAFFE2_API FutureError final : public std::exception {
235235
explicit FutureError(std::string&& error_msg_)
236236
: error_msg(std::move(error_msg_)) {}
@@ -380,7 +380,7 @@ struct C10_EXPORT ivalue::Future final : c10::intrusive_ptr_target {
380380
std::ostream& out,
381381
const Future& v);
382382

383-
TypePtr type() const {
383+
TypePtr elementType() const {
384384
return type_;
385385
}
386386

@@ -412,6 +412,13 @@ struct C10_EXPORT ivalue::Future final : c10::intrusive_ptr_target {
412412
c10::optional<FutureError> error_;
413413
};
414414

415+
// Input is a list of Futures with the same target type.
416+
// Output is a Future to the List of completed Futures.
417+
CAFFE2_API intrusive_ptr<ivalue::Future> collectAll(
418+
c10::List<c10::intrusive_ptr<ivalue::Future>> srcs);
419+
CAFFE2_API c10::intrusive_ptr<ivalue::Future> collectAll(
420+
std::vector<c10::intrusive_ptr<ivalue::Future>> srcs);
421+
415422
// User-defined object.
416423
struct C10_EXPORT ivalue::Object final : c10::intrusive_ptr_target {
417424
public:

‎aten/src/ATen/function_wrapper.py‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -513,7 +513,6 @@ def __getitem__(self, x):
513513
'type_definition_body': List[str],
514514
'type_method_definition_dispatch': str,
515515
'variants': str,
516-
'with_gil': bool,
517516
})
518517

519518
OutputDeclaration = NamedTuple('OutputDeclaration', [

‎aten/src/ATen/native/AdaptiveAveragePooling3d.cpp‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -138,7 +138,7 @@ void adaptive_avg_pool3d_out_cpu_template(
138138
} else {
139139
output.resize_({input.size(-5), sizeD, osizeT, osizeH, osizeW});
140140
at::internal::lazy_init_num_threads();
141-
int64_t b;
141+
int64_t b = 0;
142142
#pragma omp parallel for private(b)
143143
for (b = 0; b < input.size(0); b++) {
144144
AT_DISPATCH_FLOATING_TYPES_AND_HALF(
@@ -256,7 +256,7 @@ Tensor& adaptive_avg_pool3d_backward_out_cpu_template(
256256
});
257257
} else {
258258
at::internal::lazy_init_num_threads();
259-
int64_t b;
259+
int64_t b = 0;
260260
#pragma omp parallel for private(b)
261261
for (b = 0; b < input.size(0); b++) {
262262
AT_DISPATCH_FLOATING_TYPES_AND_HALF(

‎aten/src/ATen/native/Resize.h‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,6 +147,11 @@ inline void setStrided(
147147
if (self_->sizes() == size && self_->strides() == stride) {
148148
return;
149149
}
150+
for (auto val : stride) {
151+
TORCH_CHECK(val >= 0,
152+
"as_strided: Negative strides are not supported at the moment, "
153+
"got strides: ", stride);
154+
}
150155
self_->set_sizes_and_strides(size, stride);
151156
}
152157

aten/src/ATen/native/ScatterGatherShapeChecks.h renamed to aten/src/ATen/native/ScatterGatherChecks.h

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,28 @@ namespace at { namespace native {
88

99
namespace {
1010

11+
// checks whether index.dtype == int64
12+
// and self.dtyp == src.dtype if src is a Tensor
13+
static void scatter_gather_dtype_check(
14+
const std::string& method_name,
15+
const Tensor& self,
16+
const Tensor& index,
17+
const c10::optional<Tensor>& src_opt = c10::nullopt
18+
) {
19+
TORCH_CHECK(
20+
index.scalar_type() == at::ScalarType::Long,
21+
method_name, "(): Expected dtype int64 for index"
22+
);
23+
24+
if (src_opt.has_value()) {
25+
auto src = src_opt.value();
26+
TORCH_CHECK(
27+
self.scalar_type() == src.scalar_type(),
28+
method_name, "(): Expected self.dtype to be equal to src.dtype"
29+
);
30+
}
31+
}
32+
1133
// Used for `gather`-like methods
1234
// Test:
1335
// 1. index.size(d) == self.size(d) for all d != dim

‎aten/src/ATen/native/TensorAdvancedIndexing.cpp‎

Lines changed: 4 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -520,48 +520,40 @@ Tensor index_fill(const Tensor & self, int64_t dim, const Tensor & index, const
520520
}
521521

522522
Tensor & gather_out_cpu_cuda(Tensor & result, const Tensor & self, int64_t dim, const Tensor & index, bool sparse_grad) {
523-
TORCH_CHECK_INDEX(index.scalar_type() == ScalarType::Long, "gather_out(): Expected dtype int64 for index");
524523
result.resize_(index.sizes());
525524
gather_stub(result.device().type(), result, self, dim, index);
526525
return result;
527526
}
528527

529528
Tensor gather(const Tensor & self, int64_t dim, const Tensor & index, bool sparse_grad) {
530-
TORCH_CHECK_INDEX(index.scalar_type() == ScalarType::Long, "gather(): Expected dtype int64 for index");
531529
Tensor result = at::empty({0}, self.options());
532530
return gather_out_cpu_cuda(result, self, dim, index, sparse_grad);
533531
}
534532

535-
Tensor & scatter_(Tensor & self, int64_t dim, const Tensor & index, const Tensor & src) {
536-
TORCH_CHECK_INDEX(index.scalar_type() == ScalarType::Long, "scatter_(): Expected dtype int64 for index");
537-
scatter_stub(self.device().type(), self, dim, index, src);
533+
Tensor & scatter_(Tensor & self, int64_t dim, const Tensor & index, const Tensor & source) {
534+
scatter_stub(self.device().type(), self, dim, index, source);
538535
return self;
539536
}
540537

541-
Tensor & scatter_fill_(Tensor & self, int64_t dim, const Tensor & index, Scalar src) {
542-
TORCH_CHECK_INDEX(index.scalar_type() == ScalarType::Long, "scatter_fill_(): Expected dtype int64 for index");
543-
scatter_fill_stub(self.device().type(), self, dim, index, src);
538+
Tensor & scatter_fill_(Tensor & self, int64_t dim, const Tensor & index, Scalar source) {
539+
scatter_fill_stub(self.device().type(), self, dim, index, source);
544540
return self;
545541
}
546542

547543
Tensor scatter(const Tensor & self, int64_t dim, const Tensor & index, const Tensor & source) {
548-
TORCH_CHECK_INDEX(index.scalar_type() == ScalarType::Long, "scatter(): Expected dtype int64 for index");
549544
return self.clone(at::MemoryFormat::Preserve).scatter_(dim, index, source);
550545
}
551546

552547
Tensor scatter(const Tensor & self, int64_t dim, const Tensor & index, Scalar source) {
553-
TORCH_CHECK_INDEX(index.scalar_type() == ScalarType::Long, "scatter(): Expected dtype int64 for index");
554548
return self.clone(at::MemoryFormat::Preserve).scatter_(dim, index, source);
555549
}
556550

557551
Tensor & scatter_add_(Tensor & self, int64_t dim, const Tensor & index, const Tensor & src) {
558-
TORCH_CHECK_INDEX(index.scalar_type() == ScalarType::Long, "scatter_add_(): Expected dtype int64 for index");
559552
scatter_add_stub(self.device().type(), self, dim, index, src);
560553
return self;
561554
}
562555

563556
Tensor scatter_add(const Tensor & self, int64_t dim, const Tensor & index, const Tensor & source) {
564-
TORCH_CHECK_INDEX(index.scalar_type() == ScalarType::Long, "scatter_add(): Expected dtype int64 for index");
565557
return self.clone(at::MemoryFormat::Preserve).scatter_add_(dim, index, source);
566558
}
567559

0 commit comments

Comments
 (0)