Switch interpreter to use IValue's primitive int/floats #9584

zdevito · 2018-07-19T06:12:26Z

This patch switches the interpreter to use IValue's primitive numbers rather than tensors for computing on integers and floats. In addition to preparing the interpreter for first-class support of other types, this cleans up the handling of primitive numbers, making it possible to just use the normal operator overloading dispatch to find the right implementation for numbers. As a result of this change, a lot of other functionality needed to be updated since it was the first time we use non-tensors in a lot of places in the code base.

Notes:

Fixes code_template.py so that multi-line strings are indented correctly when used on a standalone line
Cast operators (int(x)) now are functional. Some tests have addition conversions to integers because
we no longer allow implicit tensor -> integer conversions following the same convention as in python
prim::ListConstruct/createList has been added to the interpreter for creating lists and this has
replaced aten::stack for integers lists
gen_jit_dispatch.py has been refactored so that non-tensor types use operators on IValues to extract
the primitives
IValue gains a .to method that is the equivalent of tensor_as but for IValue instead of at::Tensor
constant_as<T> is switched over to using IValues's .to<T> method, to make conversion from constant->IValue->C++ type
more consistent. This functionality combined with toIValue(Value*) replaces the tensor_as and as_tensor family of functions.
conditional expressions (if, loop) and operators related to them are now computed on integers rather than tensors
IValue gains constructors for constructing from at::Scalar and converting to it. However, IValue itself will always store
the scalars as a double or int64.
To align with python 3 syntax, TK_INT, TK_FLOAT, and TK_BOOL have been removed from the parser, and int/float/bool are just treated as special identifiers in the compiler,
along with print. These are represented as special sugared values with a call method implemented. For int/float/bool this implements casting behavior.
Dropped shared_from_this from Type/Module. They were not needed and they making debugging harder because they internally throw/catch exceptions.
Shape propagation has been updated to support running nodes that include floating point primitive types, this required some refactoring of internal functions.
TensorToNum and NumToTensor have actual implementations as operators now
regster_prim_ops now contains implementations of math operators for float/int primitive types, and for mixed (prim <+> tensor) versions. This removes the need for special handling in compiler.cpp
Primitive math is now entirely handled by letting the compiler choose the right overloads. This removes tons of special casing in the compiler.
incorporates @eellison's change to allow casting from return values. Due to the addition of primitive support, the code need slight modifications, so I just pre-merged it here.
stack.h gains generic vararg versions of push/pop that know how to convert to/from C++ types:

at::Tensor a;
at::Scalar b;
pop(stack, a, b);
at::Tensor c = a + b;
push(stack, c);

@apaszke

facebook-github-bot

@zdevito has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot

@zdevito has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

apaszke

Looks nice. Can you please post a sample of the operator code we now generate?

test/test_jit.py

torch/csrc/jit/interpreter.cpp

torch/csrc/jit/ir.h

torch/csrc/jit/passes/erase_number_types.cpp

torch/csrc/jit/script/compiler.cpp

constants.h/constants.cpp centralizes where we create constants, and how the interpreter handles constants. This makes it easier to keep the constant operator inline with the code that constructs constant nodes.

@eellison

This patch switches the interpreter to use IValue's primitive numbers rather than tensors for computing on integers and floats. In addition to preparing the interpreter for first-class support of other types, this cleans up the handling of primitive numbers, making it possible to just use the normal operator overloading dispatch to find the right implementation for numbers. As a result of this change, a lot of other functionality needed to be updated since it was the first time we use non-tensors in a lot of places in the code base. Notes: * Fixes code_template.py so that multi-line strings are indented correctly when used on a standalone line * Cast operators (`int(x)`) now are functional. Some tests have addition conversions to integers because we no longer allow implicit tensor -> integer conversions following the same convention as in python * prim::ListConstruct/createList has been added to the interpreter for creating lists and this has replaced aten::stack for integers lists * gen_jit_dispatch.py has been refactored so that non-tensor types use operators on IValues to extract the primitives * IValue gains a .to<T> method that is the equivalent of tensor_as but for IValue instead of at::Tensor * `constant_as<T>` is switched over to using IValues's `.to<T>` method, to make conversion from constant->IValue->C++ type more consistent. This functionality combined with `toIValue(Value*)` replaces the `tensor_as` and `as_tensor` family of functions. * conditional expressions (if, loop) and operators related to them are now computed on integers rather than tensors * IValue gains constructors for constructing from at::Scalar and converting to it. However, IValue itself will always store the scalars as a double or int64. * To align with python 3 syntax, TK_INT, TK_FLOAT, and TK_BOOL have been removed from the parser, and int/float/bool are just treated as special identifiers in the compiler, along with print. These are represented as special sugared values with a `call` method implemented. For int/float/bool this implements casting behavior. * Dropped shared_from_this from Type/Module. They were not needed and they making debugging harder because they internally throw/catch exceptions. * Shape propagation has been updated to support running nodes that include floating point primitive types, this required some refactoring of internal functions. * TensorToNum and NumToTensor have actual implementations as operators now * regster_prim_ops now contains implementations of math operators for float/int primitive types, and for mixed (prim <+> tensor) versions. This removes the need for special handling in compiler.cpp * Primitive math is now entirely handled by letting the compiler choose the right overloads. This removes tons of special casing in the compiler. * incorporates @eellison's change to allow casting from return values. Due to the addition of primitive support, the code need slight modifications, so I just pre-merged it here. * stack.h gains generic vararg versions of push/pop that know how to convert to/from C++ types: ``` at::Tensor a; at::Scalar b; pop(stack, a, b); at::Tensor c = a + b; push(stack, c); ```

zdevito · 2018-07-22T06:28:34Z

Rebase notes:

Non mechanical changes were required in shape_analysis.cpp to get everything working again. I enabled shape_analysis to work "ByRunningIt" when an input is constant in addition to other scenarios, which works around a lot of the corner cases for attributes. I also implemented gatherTensor even for flattened varargs because the logic was clearer than handling cat specifically (still would be good to just get rid of varargs and have actual lists)
ir.cpp also saw some changes to hook the get infrastructure up to the new way of representing constants. One change: I removed all the default-value logic here. Nodes are never missing default values when they exist in a graph. The compiler processes the schema and always inserts the defaults. So it was possible to remove than handling and then simply the way the functions work. Further simplification was possible because IValue already handles the templated conversions.
In the process I discovered that input(size_t) and input(Symbol name) cause ambiguity problems. This only appeared once the Symbol version got marked const, but I believe was lurking there subtly the whole time. I rename the Symbol version to namedInput for now. Maybe we can sort something out here.

zdevito · 2018-07-22T06:31:03Z

This PR will generated a lot of follow-ups and unblock a lot of other changes. These are the known followups:

Implement: Graph::insert(Symbol name, at::ArrayRef<Value*> args, at::ArrayRef<NamedValue> kwargs, const SourceRange& range)
Replace all create calls of aten::op things with insert rather than create, especially in SymbolicVariable.
Consider if there is a way to mix IValue's and Value* in args/kwargs so you can, e.g. write g->emit(aten::add,{a, 1});
(larger) Allow non-Tensor inputs to the graph executor. This requires re-design of ArgumentSpec and the specialization process.
This change will require discussion.
Add sane TensorToNum implicit conversions. This requires favoring operators with no-implicit conversions over operators with
them, similar to C++ overloading rules. This will allow use of scalar tensors without using casts.
Print should be able to print any IValue. We should also be able to print IValues for debugging purposes.
This means adding operator<< logic to ivalues and modifying the print command.
Change ->expect and ->cast on TypePtr to return shared pointers to the subtype rather than raw pointers.
Also change isSubtypeOf to accept a TypePtr rather than a Type&. This will make isSubtype cleaner:
x->isSubtypeOf(NumberType::get()) rather than x->isSubtypeOf(*NumberType::get())
Allow int->float and float->int casts. Current we only allow tensor->int and tensor->float castws.
(easy) shape prop formulat for NumToTensor
(easy) move insertConstant back into Graph. Currently the optional SourceRange for insertConstant pulls the entire lexer into ir.h,
we need to do some refactoring so that we can reference SourceRange in ir.h. This may be a very simple change.
Remove 'varargs' behavior of cat and other things with tensor lists, and implement them with actual lists of tensors. This will be a difficult change
because some passes like fusion operator on cat nodes, and will need to continue to work to not slow down the LSTM examples.

apaszke

Awesome to see all those simplifications! I only have some minor questions (mostly to point out possible future improvements), and small nits.

tools/jit/gen_jit_dispatch.py

-    'Layout': 'tensor_as<int64_t>',
-    'IntList': 'tensor_as<std::vector<int64_t>>',
+FROM_IVALUE = {
+    'Device': 'as_device({}.toIntList()->elements())',


torch/csrc/jit/interned_strings.cpp

  if (pos == std::string::npos) {
-    throw std::runtime_error(
-        "all symbols must have a namespace, <namespace>::<string>");
+    std::stringstream ss;


torch/csrc/jit/ir.cpp

+    // and once we remove attributes it no longer will
+    Value* v = insertConstant(const_cast<Graph&>(*owningGraph()), get(name).value());
+    // XXX - insert point can be anywhere since modifying the graph is unexpected,
+    // so this is completely unsafe and needs to be gone as soon as possible.


torch/csrc/jit/ir.h

    return inputs_.at(i);
  }

+  Value* input(Symbol name) const { throw std::runtime_error("NO!"); }


torch/csrc/jit/ir.h

+  }
+  Node* createTensorToNum(const TypePtr& type, Value* value) {
+    auto* result = create(prim::TensorToNum, {value});
+    result->output()->setType(type);


torch/csrc/jit/passes/shape_analysis.cpp

-      node->output()->inferTypeFrom(node->t(attr::value));
+      if(node->output()->type()->isSubtypeOf(*DynamicType::get())) {
+        node->output()->inferTypeFrom(node->t(attr::value));
+      }


torch/csrc/jit/register_prim_ops.cpp

          return [=](Stack& stack) {
-            auto shape = pop(stack).toTensor().contiguous();
-            auto input = pop(stack).toTensor();
+            at::Tensor input, shape;


torch/csrc/jit/tensor_conversions.h

 template<typename T>
 inline at::Tensor as_variable(const T& t) {
  return autograd::make_variable(as_tensor(t));
 }


torch/csrc/jit/tensor_conversions.h

 template<typename T>
 inline T tensor_as(at::Tensor t) {
  return detail::tensor_as_impl<T>()(std::move(t));
 }


torch/csrc/jit/script/compiler.cpp

+  CastValue(TypePtr type)
+  : type(type) {}
+  std::string kind() const override {
+    std::stringstream ss;


apaszke · 2018-07-23T01:13:39Z

The test failures look legit:

ONNX export changed/failed for alexnet (C2 build)
unused variable at shape_analysis.cpp:143:10 causes a failure with -Werror (macOS build)

test/onnx/test_caffe2.py

facebook-github-bot

@zdevito has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot

@zdevito has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot

@zdevito has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: This patch switches the interpreter to use IValue's primitive numbers rather than tensors for computing on integers and floats. In addition to preparing the interpreter for first-class support of other types, this cleans up the handling of primitive numbers, making it possible to just use the normal operator overloading dispatch to find the right implementation for numbers. As a result of this change, a lot of other functionality needed to be updated since it was the first time we use non-tensors in a lot of places in the code base. Notes: * Fixes code_template.py so that multi-line strings are indented correctly when used on a standalone line * Cast operators (`int(x)`) now are functional. Some tests have addition conversions to integers because we no longer allow implicit tensor -> integer conversions following the same convention as in python * prim::ListConstruct/createList has been added to the interpreter for creating lists and this has replaced aten::stack for integers lists * gen_jit_dispatch.py has been refactored so that non-tensor types use operators on IValues to extract the primitives * IValue gains a .to<T> method that is the equivalent of tensor_as but for IValue instead of at::Tensor * `constant_as<T>` is switched over to using IValues's `.to<T>` method, to make conversion from constant->IValue->C++ type more consistent. This functionality combined with `toIValue(Value*)` replaces the `tensor_as` and `as_tensor` family of functions. * conditional expressions (if, loop) and operators related to them are now computed on integers rather than tensors * IValue gains constructors for constructing from at::Scalar and converting to it. However, IValue itself will always store the scalars as a double or int64. * To align with python 3 syntax, TK_INT, TK_FLOAT, and TK_BOOL have been removed from the parser, and int/float/bool are just treated as special identifiers in the compiler, along with print. These are represented as special sugared values with a `call` method implemented. For int/float/bool this implements casting behavior. * Dropped shared_from_this from Type/Module. They were not needed and they making debugging harder because they internally throw/catch exceptions. * Shape propagation has been updated to support running nodes that include floating point primitive types, this required some refactoring of internal functions. * TensorToNum and NumToTensor have actual implementations as operators now * regster_prim_ops now contains implementations of math operators for float/int primitive types, and for mixed (prim <+> tensor) versions. This removes the need for special handling in compiler.cpp * Primitive math is now entirely handled by letting the compiler choose the right overloads. This removes tons of special casing in the compiler. * incorporates eellison's change to allow casting from return values. Due to the addition of primitive support, the code need slight modifications, so I just pre-merged it here. * stack.h gains generic vararg versions of push/pop that know how to convert to/from C++ types: ``` at::Tensor a; at::Scalar b; pop(stack, a, b); at::Tensor c = a + b; push(stack, c); ``` apaszke Pull Request resolved: pytorch#9584 Differential Revision: D8910546 fbshipit-source-id: f4266666d509431dfe082cefa100d1a31c9e87f7

test/expect/TestScript.test_onnx_export_script_module_if.expect

        Node {type: "ReduceSum", inputs: [x.1], outputs: [1], attributes: [{ name: 'keepdims', type: int, value: 0}]},
        Node {type: "Constant", inputs: [], outputs: [2], attributes: [{ name: 'value', type: tensor, value:TensorProto shape: []}]},
-        Node {type: "Greater", inputs: [1,2], outputs: [3], attributes: []},
+        Node {type: "Greater", inputs: [1,2], outputs: [3], attributes: [{ name: 'broadcast', type: int, value: 1}]},


Summary: Pull Request resolved: #9718 This patch switches the interpreter to use IValue's primitive numbers rather than tensors for computing on integers and floats. In addition to preparing the interpreter for first-class support of other types, this cleans up the handling of primitive numbers, making it possible to just use the normal operator overloading dispatch to find the right implementation for numbers. As a result of this change, a lot of other functionality needed to be updated since it was the first time we use non-tensors in a lot of places in the code base. Notes: * Fixes code_template.py so that multi-line strings are indented correctly when used on a standalone line * Cast operators (`int(x)`) now are functional. Some tests have addition conversions to integers because we no longer allow implicit tensor -> integer conversions following the same convention as in python * prim::ListConstruct/createList has been added to the interpreter for creating lists and this has replaced aten::stack for integers lists * gen_jit_dispatch.py has been refactored so that non-tensor types use operators on IValues to extract the primitives * IValue gains a .to<T> method that is the equivalent of tensor_as but for IValue instead of at::Tensor * `constant_as<T>` is switched over to using IValues's `.to<T>` method, to make conversion from constant->IValue->C++ type more consistent. This functionality combined with `toIValue(Value*)` replaces the `tensor_as` and `as_tensor` family of functions. * conditional expressions (if, loop) and operators related to them are now computed on integers rather than tensors * IValue gains constructors for constructing from at::Scalar and converting to it. However, IValue itself will always store the scalars as a double or int64. * To align with python 3 syntax, TK_INT, TK_FLOAT, and TK_BOOL have been removed from the parser, and int/float/bool are just treated as special identifiers in the compiler, along with print. These are represented as special sugared values with a `call` method implemented. For int/float/bool this implements casting behavior. * Dropped shared_from_this from Type/Module. They were not needed and they making debugging harder because they internally throw/catch exceptions. * Shape propagation has been updated to support running nodes that include floating point primitive types, this required some refactoring of internal functions. * TensorToNum and NumToTensor have actual implementations as operators now * regster_prim_ops now contains implementations of math operators for float/int primitive types, and for mixed (prim <+> tensor) versions. This removes the need for special handling in compiler.cpp * Primitive math is now entirely handled by letting the compiler choose the right overloads. This removes tons of special casing in the compiler. * incorporates eellison's change to allow casting from return values. Due to the addition of primitive support, the code need slight modifications, so I just pre-merged it here. * stack.h gains generic vararg versions of push/pop that know how to convert to/from C++ types: ``` at::Tensor a; at::Scalar b; pop(stack, a, b); at::Tensor c = a + b; push(stack, c); ``` apaszke Pull Request resolved: #9584 Reviewed By: apaszke Differential Revision: D8910546 Pulled By: zdevito fbshipit-source-id: 0f3e60d4d22217f196a8f606549430e43b7e7e30

Summary: Pull Request resolved: pytorch/pytorch#9718 This patch switches the interpreter to use IValue's primitive numbers rather than tensors for computing on integers and floats. In addition to preparing the interpreter for first-class support of other types, this cleans up the handling of primitive numbers, making it possible to just use the normal operator overloading dispatch to find the right implementation for numbers. As a result of this change, a lot of other functionality needed to be updated since it was the first time we use non-tensors in a lot of places in the code base. Notes: * Fixes code_template.py so that multi-line strings are indented correctly when used on a standalone line * Cast operators (`int(x)`) now are functional. Some tests have addition conversions to integers because we no longer allow implicit tensor -> integer conversions following the same convention as in python * prim::ListConstruct/createList has been added to the interpreter for creating lists and this has replaced aten::stack for integers lists * gen_jit_dispatch.py has been refactored so that non-tensor types use operators on IValues to extract the primitives * IValue gains a .to<T> method that is the equivalent of tensor_as but for IValue instead of at::Tensor * `constant_as<T>` is switched over to using IValues's `.to<T>` method, to make conversion from constant->IValue->C++ type more consistent. This functionality combined with `toIValue(Value*)` replaces the `tensor_as` and `as_tensor` family of functions. * conditional expressions (if, loop) and operators related to them are now computed on integers rather than tensors * IValue gains constructors for constructing from at::Scalar and converting to it. However, IValue itself will always store the scalars as a double or int64. * To align with python 3 syntax, TK_INT, TK_FLOAT, and TK_BOOL have been removed from the parser, and int/float/bool are just treated as special identifiers in the compiler, along with print. These are represented as special sugared values with a `call` method implemented. For int/float/bool this implements casting behavior. * Dropped shared_from_this from Type/Module. They were not needed and they making debugging harder because they internally throw/catch exceptions. * Shape propagation has been updated to support running nodes that include floating point primitive types, this required some refactoring of internal functions. * TensorToNum and NumToTensor have actual implementations as operators now * regster_prim_ops now contains implementations of math operators for float/int primitive types, and for mixed (prim <+> tensor) versions. This removes the need for special handling in compiler.cpp * Primitive math is now entirely handled by letting the compiler choose the right overloads. This removes tons of special casing in the compiler. * incorporates eellison's change to allow casting from return values. Due to the addition of primitive support, the code need slight modifications, so I just pre-merged it here. * stack.h gains generic vararg versions of push/pop that know how to convert to/from C++ types: ``` at::Tensor a; at::Scalar b; pop(stack, a, b); at::Tensor c = a + b; push(stack, c); ``` apaszke Pull Request resolved: pytorch/pytorch#9584 Reviewed By: apaszke Differential Revision: D8910546 Pulled By: zdevito fbshipit-source-id: 0f3e60d4d22217f196a8f606549430e43b7e7e30

…Of accept TypePtr (#9786) Summary: Follow up task of #9584. Commit 1: - change expect/cast to return shared pointers instead of raw pointer - isSubtypeOf accept TypePtr instead. Use `x->isSubtypeOf(NumberType::get())` rather than `x->isSubtypeOf(*NumberType::get())` Commit 2: - to address enable_shared_from_this pitfalls, we make the constructor private and expose the factory method to make sure user can only create it using our factory method. Pull Request resolved: #9786 Reviewed By: zdevito Differential Revision: D8980441 Pulled By: wanchaol fbshipit-source-id: e5c923fc57a701014310e77cf29985b43bb25364

Summary: Pull Request resolved: pytorch#9718 This patch switches the interpreter to use IValue's primitive numbers rather than tensors for computing on integers and floats. In addition to preparing the interpreter for first-class support of other types, this cleans up the handling of primitive numbers, making it possible to just use the normal operator overloading dispatch to find the right implementation for numbers. As a result of this change, a lot of other functionality needed to be updated since it was the first time we use non-tensors in a lot of places in the code base. Notes: * Fixes code_template.py so that multi-line strings are indented correctly when used on a standalone line * Cast operators (`int(x)`) now are functional. Some tests have addition conversions to integers because we no longer allow implicit tensor -> integer conversions following the same convention as in python * prim::ListConstruct/createList has been added to the interpreter for creating lists and this has replaced aten::stack for integers lists * gen_jit_dispatch.py has been refactored so that non-tensor types use operators on IValues to extract the primitives * IValue gains a .to<T> method that is the equivalent of tensor_as but for IValue instead of at::Tensor * `constant_as<T>` is switched over to using IValues's `.to<T>` method, to make conversion from constant->IValue->C++ type more consistent. This functionality combined with `toIValue(Value*)` replaces the `tensor_as` and `as_tensor` family of functions. * conditional expressions (if, loop) and operators related to them are now computed on integers rather than tensors * IValue gains constructors for constructing from at::Scalar and converting to it. However, IValue itself will always store the scalars as a double or int64. * To align with python 3 syntax, TK_INT, TK_FLOAT, and TK_BOOL have been removed from the parser, and int/float/bool are just treated as special identifiers in the compiler, along with print. These are represented as special sugared values with a `call` method implemented. For int/float/bool this implements casting behavior. * Dropped shared_from_this from Type/Module. They were not needed and they making debugging harder because they internally throw/catch exceptions. * Shape propagation has been updated to support running nodes that include floating point primitive types, this required some refactoring of internal functions. * TensorToNum and NumToTensor have actual implementations as operators now * regster_prim_ops now contains implementations of math operators for float/int primitive types, and for mixed (prim <+> tensor) versions. This removes the need for special handling in compiler.cpp * Primitive math is now entirely handled by letting the compiler choose the right overloads. This removes tons of special casing in the compiler. * incorporates eellison's change to allow casting from return values. Due to the addition of primitive support, the code need slight modifications, so I just pre-merged it here. * stack.h gains generic vararg versions of push/pop that know how to convert to/from C++ types: ``` at::Tensor a; at::Scalar b; pop(stack, a, b); at::Tensor c = a + b; push(stack, c); ``` apaszke Pull Request resolved: pytorch#9584 Reviewed By: apaszke Differential Revision: D8910546 Pulled By: zdevito fbshipit-source-id: 0f3e60d4d22217f196a8f606549430e43b7e7e30

…Of accept TypePtr (pytorch#9786) Summary: Follow up task of pytorch#9584. Commit 1: - change expect/cast to return shared pointers instead of raw pointer - isSubtypeOf accept TypePtr instead. Use `x->isSubtypeOf(NumberType::get())` rather than `x->isSubtypeOf(*NumberType::get())` Commit 2: - to address enable_shared_from_this pitfalls, we make the constructor private and expose the factory method to make sure user can only create it using our factory method. Pull Request resolved: pytorch#9786 Reviewed By: zdevito Differential Revision: D8980441 Pulled By: wanchaol fbshipit-source-id: e5c923fc57a701014310e77cf29985b43bb25364

Summary: Pull Request resolved: pytorch#9718 This patch switches the interpreter to use IValue's primitive numbers rather than tensors for computing on integers and floats. In addition to preparing the interpreter for first-class support of other types, this cleans up the handling of primitive numbers, making it possible to just use the normal operator overloading dispatch to find the right implementation for numbers. As a result of this change, a lot of other functionality needed to be updated since it was the first time we use non-tensors in a lot of places in the code base. Notes: * Fixes code_template.py so that multi-line strings are indented correctly when used on a standalone line * Cast operators (`int(x)`) now are functional. Some tests have addition conversions to integers because we no longer allow implicit tensor -> integer conversions following the same convention as in python * prim::ListConstruct/createList has been added to the interpreter for creating lists and this has replaced aten::stack for integers lists * gen_jit_dispatch.py has been refactored so that non-tensor types use operators on IValues to extract the primitives * IValue gains a .to<T> method that is the equivalent of tensor_as but for IValue instead of at::Tensor * `constant_as<T>` is switched over to using IValues's `.to<T>` method, to make conversion from constant->IValue->C++ type more consistent. This functionality combined with `toIValue(Value*)` replaces the `tensor_as` and `as_tensor` family of functions. * conditional expressions (if, loop) and operators related to them are now computed on integers rather than tensors * IValue gains constructors for constructing from at::Scalar and converting to it. However, IValue itself will always store the scalars as a double or int64. * To align with python 3 syntax, TK_INT, TK_FLOAT, and TK_BOOL have been removed from the parser, and int/float/bool are just treated as special identifiers in the compiler, along with print. These are represented as special sugared values with a `call` method implemented. For int/float/bool this implements casting behavior. * Dropped shared_from_this from Type/Module. They were not needed and they making debugging harder because they internally throw/catch exceptions. * Shape propagation has been updated to support running nodes that include floating point primitive types, this required some refactoring of internal functions. * TensorToNum and NumToTensor have actual implementations as operators now * regster_prim_ops now contains implementations of math operators for float/int primitive types, and for mixed (prim <+> tensor) versions. This removes the need for special handling in compiler.cpp * Primitive math is now entirely handled by letting the compiler choose the right overloads. This removes tons of special casing in the compiler. * incorporates eellison's change to allow casting from return values. Due to the addition of primitive support, the code need slight modifications, so I just pre-merged it here. * stack.h gains generic vararg versions of push/pop that know how to convert to/from C++ types: ``` at::Tensor a; at::Scalar b; pop(stack, a, b); at::Tensor c = a + b; push(stack, c); ``` apaszke Pull Request resolved: pytorch#9584 Reviewed By: apaszke Differential Revision: D8910546 Pulled By: zdevito fbshipit-source-id: 0f3e60d4d22217f196a8f606549430e43b7e7e30

…Of accept TypePtr (pytorch#9786) Summary: Follow up task of pytorch#9584. Commit 1: - change expect/cast to return shared pointers instead of raw pointer - isSubtypeOf accept TypePtr instead. Use `x->isSubtypeOf(NumberType::get())` rather than `x->isSubtypeOf(*NumberType::get())` Commit 2: - to address enable_shared_from_this pitfalls, we make the constructor private and expose the factory method to make sure user can only create it using our factory method. Pull Request resolved: pytorch#9786 Reviewed By: zdevito Differential Revision: D8980441 Pulled By: wanchaol fbshipit-source-id: e5c923fc57a701014310e77cf29985b43bb25364

zdevito added the oncall: jit Add this issue/PR to JIT oncall triage queue label Jul 19, 2018

zdevito requested review from Yangqing, anderspapitto, apaszke, bddppq, colesbury, dzhulgakov, ezyang, gchanan, houseroad, jamesr66a, smessmer and soumith as code owners July 19, 2018 06:12

facebook-github-bot reviewed Jul 19, 2018

View reviewed changes

zdevito force-pushed the pr/ivalue2 branch from d6e3527 to 433dcdc Compare July 20, 2018 00:52

facebook-github-bot reviewed Jul 20, 2018

View reviewed changes

apaszke approved these changes Jul 20, 2018

View reviewed changes

zdevito added 5 commits July 21, 2018 23:07

Add push/pop for stack that can unwrap/wrap IValues.

878f9f6

Move constant handling into a single file

ed6ab3b

constants.h/constants.cpp centralizes where we create constants, and how the interpreter handles constants. This makes it easier to keep the constant operator inline with the code that constructs constant nodes.

remove march=native, remove =delete that angers clang 3.8

c503428

PR Comments

141caf3

zdevito force-pushed the pr/ivalue2 branch from f34b5c8 to 4874e7e Compare July 22, 2018 06:08

Fixups required due to the rebase.

63bc34a

zdevito force-pushed the pr/ivalue2 branch from 4874e7e to 63bc34a Compare July 22, 2018 06:21

apaszke approved these changes Jul 23, 2018

View reviewed changes

apaszke reviewed Jul 23, 2018

View reviewed changes

test/onnx/test_caffe2.py Outdated

This comment was marked as off-topic.

Sign in to view

This comment was marked as off-topic.

Sign in to view

ONNX add ListConstruct

b96e113

zdevito force-pushed the pr/ivalue2 branch from be11b07 to b96e113 Compare July 23, 2018 02:00

zdevito added 2 commits July 22, 2018 19:56

Remove tensor_conversions

ba13d40

Make type propagation for NumToTensor more accurate.

54a553c

zdevito force-pushed the pr/ivalue2 branch from 800e85f to 54a553c Compare July 23, 2018 04:17

apaszke approved these changes Jul 23, 2018

View reviewed changes

facebook-github-bot reviewed Jul 23, 2018

View reviewed changes

Fix subtle copy of Argument in findArgument

af8c653

facebook-github-bot reviewed Jul 23, 2018

View reviewed changes

Fix missing constructor call in IValue

6a9e59d

facebook-github-bot reviewed Jul 23, 2018

View reviewed changes

zdevito mentioned this pull request Jul 23, 2018

Switch interpreter to use IValue's primitive int/floats (#9584) #9718

Closed

jamesr66a reviewed Jul 23, 2018

View reviewed changes

facebook-github-bot closed this in pytorch/translate@a0547a8 Jul 23, 2018

wanchaol mentioned this pull request Jul 24, 2018

[JIT] Change expect, cast on Type to return shared pointers, make isSubtypeOf accept TypePtr #9786

Closed

ezyang added the merged label Jun 26, 2019

Switch interpreter to use IValue's primitive int/floats #9584

Switch interpreter to use IValue's primitive int/floats #9584

Uh oh!

Conversation

zdevito commented Jul 19, 2018

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

apaszke left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

zdevito commented Jul 22, 2018

Uh oh!

zdevito commented Jul 22, 2018

Uh oh!

apaszke left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!