C++ API parity: isfinite #28918

pbelevich · 2019-10-30T21:08:24Z

Stack from ghstack:

C++ API parity: BCEWithLogitsLoss #28783 C++ API parity: BCEWithLogitsLoss
C++ API parity: isfinite #28918 C++ API parity: isfinite

Differential Revision: D18233037

[ghstack-poisoned]

test/cpp/api/functional.cpp

torch/csrc/api/include/torch/functional.h

Differential Revision: [D18233037](https://our.internmc.facebook.com/intern/diff/D18233037) [ghstack-poisoned]

aten/src/ATen/native/TensorCompare.cpp

torch/csrc/api/include/torch/functional.h

Differential Revision: [D18233037](https://our.internmc.facebook.com/intern/diff/D18233037) [ghstack-poisoned]

zou3519

Some questions

aten/src/ATen/native/TensorCompare.cpp

zou3519 · 2019-11-18T21:06:51Z

test/cpp/api/functional.cpp

+      inf
+    }, torch::TensorOptions().dtype(S).device(device));
+    ASSERT_TRUE(torch::allclose(
+      torch::isfinite(x).toType(torch::kInt),


We're taking a BoolTensor, converting to IntTensor, and then comparing it to another IntTensor that was converted from a BoolTensor. Is there a way to just directly compare the BoolTensors ?

torch::allclose does not support torch::kBool

Could you toss this information into an in-line comment in the testing code?

Differential Revision: [D18233037](https://our.internmc.facebook.com/intern/diff/D18233037) [ghstack-poisoned]

facebook-github-bot · 2019-11-19T03:38:01Z

@pbelevich merged this pull request in 8df5e10.

Summary: Pull Request resolved: pytorch/pytorch#28918 Test Plan: Imported from OSS Differential Revision: D18233037 Pulled By: pbelevich fbshipit-source-id: c76b9467bbc1fbb2c9bf49855895c98438b36c12

yf225 · 2019-11-19T04:29:18Z

test/cpp/api/functional.cpp

+}
+
+TEST_F(FunctionalTest, isfinite) {
+  for (const auto device : {at::Device("cpu"), at::Device("cuda")}) {


CPU-only tests seem to fail with

Nov 19 00:33:35 [ RUN ] FunctionalTest.isfinite Nov 19 00:33:35 unknown file: Failure Nov 19 00:33:35 C++ exception with description "Cannot initialize CUDA without ATen_cuda library. PyTorch splits its backend into two shared libraries: a CPU library and a CUDA library; this error has occurred because you are trying to use some CUDA functionality, but the CUDA library has not been loaded by the dynamic linker for some reason. The CUDA library MUST be loaded, EVEN IF you don't directly use any symbols from the CUDA library! One common culprit is a lack of -Wl,--no-as-needed in your link arguments; many dynamic linkers will delete dynamic library dependencies if you don't depend on any of their symbols. You can check if this has occurred by using ldd on your binary to see if there is a dependency on *_cuda.so library. (initCUDA at /var/lib/jenkins/workspace/aten/src/ATen/detail/CUDAHooksInterface.h:63) Nov 19 00:33:35 frame #0: <unknown function> + 0xcfa83 (0x7fabe4f6ca83 in /var/lib/jenkins/workspace/build/lib/libc10.so) Nov 19 00:33:35 frame #1: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 0x1c8 (0x7fabe4f6b118 in /var/lib/jenkins/workspace/build/lib/libc10.so) Nov 19 00:33:35 frame #2: <unknown function> + 0xb1c616f (0x7fabf03e916f in /var/lib/jenkins/workspace/build/lib/libtorch.so) Nov 19 00:33:35 frame #3: <unknown function> + 0xb12b6a2 (0x7fabf034e6a2 in /var/lib/jenkins/workspace/build/lib/libtorch.so) Nov 19 00:33:35 frame #4: <unknown function> + 0xea99 (0x7fabdd57ba99 in /lib/x86_64-linux-gnu/libpthread.so.0) Nov 19 00:33:35 frame #5: <unknown function> + 0xb12b363 (0x7fabf034e363 in /var/lib/jenkins/workspace/build/lib/libtorch.so) Nov 19 00:33:35 frame #6: <unknown function> + 0xea99 (0x7fabdd57ba99 in /lib/x86_64-linux-gnu/libpthread.so.0) Nov 19 00:33:35 frame #7: <unknown function> + 0xbcffbc (0x56250335cfbc in build/bin/test_api) Nov 19 00:33:35 frame #8: <unknown function> + 0xbea6f1 (0x5625033776f1 in build/bin/test_api) Nov 19 00:33:35 frame #9: <unknown function> + 0xbb270e (0x56250333f70e in build/bin/test_api) Nov 19 00:33:35 frame #10: void test_isfinite<(c10::ScalarType)0, unsigned char>(c10::Device const&) + 0x771 (0x5625037cfe31 in build/bin/test_api) Nov 19 00:33:35 frame #11: FunctionalTest_isfinite_Test::TestBody() + 0x2f5 (0x5625037b2195 in build/bin/test_api) Nov 19 00:33:35 frame #12: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) + 0x2bb (0x5625040aafdb in build/bin/test_api) Nov 19 00:33:35 frame #13: testing::Test::Run() + 0x25b (0x562504059acb in build/bin/test_api) Nov 19 00:33:35 frame #14: testing::TestInfo::Run() + 0x543 (0x56250405c413 in build/bin/test_api) Nov 19 00:33:35 frame #15: testing::TestCase::Run() + 0x5fb (0x56250405e33b in build/bin/test_api) Nov 19 00:33:35 frame #16: testing::internal::UnitTestImpl::RunAllTests() + 0x13d5 (0x562504080745 in build/bin/test_api) Nov 19 00:33:35 frame #17: bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) + 0x2b2 (0x5625040b0712 in build/bin/test_api) Nov 19 00:33:35 frame #18: testing::UnitTest::Run() + 0x29a (0x56250407f1ea in build/bin/test_api) Nov 19 00:33:35 frame #19: main + 0x458 (0x5625032bc328 in build/bin/test_api) Nov 19 00:33:35 frame #20: __libc_start_main + 0xf0 (0x7fabdc9ea830 in /lib/x86_64-linux-gnu/libc.so.6) Nov 19 00:33:35 frame #21: _start + 0x29 (0x5625032bbcd9 in build/bin/test_api) Nov 19 00:33:35 " thrown in the test body. Nov 19 00:33:35 [ FAILED ] FunctionalTest.isfinite (12 ms)

We might need to split this test into TEST_F(FunctionalTest, isfinite) and TEST_F(FunctionalTest, isfinite_CUDA), and the latter would contain only the CUDA tests.

yf225 · 2019-11-19T04:35:07Z

test/cpp/api/functional.cpp

+    test_isfinite<torch::kInt8, char>(device);
+    test_isfinite<torch::kInt16, short>(device);
+    test_isfinite<torch::kInt32, int>(device);
+    test_isfinite<torch::kInt64, long>(device);


I suspect that long represents types of different bit-length on different platforms, and we might want to use the following list instead:

torch::kByte, uint8_t torch::kChar, int8_t torch::kShort, int16_t torch::kInt, int torch::kLong, int64_t torch::kFloat, float torch::kDouble, double

yf225 · 2019-11-19T04:35:53Z

@pbelevich This PR seems to be breaking master with errors such as:

Nov 18 19:47:30 ../test/cpp/api/functional.cpp:2001:40: error: conversion from 'const long' to 'at::Scalar' is ambiguous
Nov 18 19:47:30     const auto x = torch::full({3, 3}, value, torch::TensorOptions().dtype(S).device(device));
Nov 18 19:47:30                                        ^~~~~

I will have to revert it for now, and I commented about the possible solutions in-line :D

As part of #30786, this moves `isinf` to C++. It's mostly the same as #28918.

C++ API parity: isfinite

dc22ba7

[ghstack-poisoned]

pbelevich requested review from ebetica, goldsborough and yf225 as code owners October 30, 2019 21:08

This was referenced Oct 30, 2019

C++ API parity: CTCLoss #28654

Closed

C++ API parity: PoissonNLLLoss #28755

Closed

C++ API parity: BCEWithLogitsLoss #28783

Closed

pbelevich removed request for ebetica and goldsborough October 30, 2019 21:09

yf225 reviewed Oct 30, 2019

View reviewed changes

test/cpp/api/functional.cpp Show resolved Hide resolved

torch/csrc/api/include/torch/functional.h Outdated Show resolved Hide resolved

zou3519 reviewed Oct 30, 2019

View reviewed changes

torch/csrc/api/include/torch/functional.h Outdated Show resolved Hide resolved

pbelevich added 2 commits October 30, 2019 15:53

Update on "C++ API parity: isfinite"

48fa119

Differential Revision: [D18233037](https://our.internmc.facebook.com/intern/diff/D18233037) [ghstack-poisoned]

Update on "C++ API parity: isfinite"

c81aa1f

Differential Revision: [D18233037](https://our.internmc.facebook.com/intern/diff/D18233037) [ghstack-poisoned]

pbelevich mentioned this pull request Oct 31, 2019

C++ API parity: MarginRankingLoss #29000

Closed

pbelevich added 10 commits October 31, 2019 17:23

Update on "C++ API parity: isfinite"

9ca3000

Differential Revision: [D18233037](https://our.internmc.facebook.com/intern/diff/D18233037) [ghstack-poisoned]

Update on "C++ API parity: isfinite"

f47d608

Differential Revision: [D18233037](https://our.internmc.facebook.com/intern/diff/D18233037) [ghstack-poisoned]

Update on "C++ API parity: isfinite"

7948286

Differential Revision: [D18233037](https://our.internmc.facebook.com/intern/diff/D18233037) [ghstack-poisoned]

Update on "C++ API parity: isfinite"

7fee0c1

Differential Revision: [D18233037](https://our.internmc.facebook.com/intern/diff/D18233037) [ghstack-poisoned]

Update on "C++ API parity: isfinite"

332de85

Differential Revision: [D18233037](https://our.internmc.facebook.com/intern/diff/D18233037) [ghstack-poisoned]

Update on "C++ API parity: isfinite"

07693a5

Differential Revision: [D18233037](https://our.internmc.facebook.com/intern/diff/D18233037) [ghstack-poisoned]

Update on "C++ API parity: isfinite"

4da5908

Differential Revision: [D18233037](https://our.internmc.facebook.com/intern/diff/D18233037) [ghstack-poisoned]

Update on "C++ API parity: isfinite"

9aab75c

Differential Revision: [D18233037](https://our.internmc.facebook.com/intern/diff/D18233037) [ghstack-poisoned]

Update on "C++ API parity: isfinite"

1ae3ff6

Differential Revision: [D18233037](https://our.internmc.facebook.com/intern/diff/D18233037) [ghstack-poisoned]

Update on "C++ API parity: isfinite"

91f865c

Differential Revision: [D18233037](https://our.internmc.facebook.com/intern/diff/D18233037) [ghstack-poisoned]

pbelevich requested review from yf225 and zou3519 November 15, 2019 17:48

eellison mentioned this pull request Nov 15, 2019

[jit] torch.isfinite is broken #29340

Closed

zou3519 reviewed Nov 15, 2019

View reviewed changes

aten/src/ATen/native/TensorCompare.cpp Show resolved Hide resolved

zou3519 reviewed Nov 15, 2019

View reviewed changes

aten/src/ATen/native/TensorCompare.cpp Outdated Show resolved Hide resolved

zou3519 reviewed Nov 15, 2019

View reviewed changes

aten/src/ATen/native/TensorCompare.cpp Outdated Show resolved Hide resolved

zou3519 reviewed Nov 15, 2019

View reviewed changes

torch/csrc/api/include/torch/functional.h Outdated Show resolved Hide resolved

Update on "C++ API parity: isfinite"

77c5402

Differential Revision: [D18233037](https://our.internmc.facebook.com/intern/diff/D18233037) [ghstack-poisoned]

pbelevich requested a review from zou3519 November 18, 2019 21:01

zou3519 reviewed Nov 18, 2019

View reviewed changes

Update on "C++ API parity: isfinite"

793904a

Differential Revision: [D18233037](https://our.internmc.facebook.com/intern/diff/D18233037) [ghstack-poisoned]

zou3519 approved these changes Nov 18, 2019

View reviewed changes

Update on "C++ API parity: isfinite"

afd79f0

Differential Revision: [D18233037](https://our.internmc.facebook.com/intern/diff/D18233037) [ghstack-poisoned]

yf225 approved these changes Nov 18, 2019

View reviewed changes

facebook-github-bot closed this in 8df5e10 Nov 19, 2019

facebook-github-bot added the merged label Nov 19, 2019

yf225 reviewed Nov 19, 2019

View reviewed changes

pbelevich reopened this Nov 19, 2019

pbelevich closed this Nov 19, 2019

facebook-github-bot deleted the gh/pbelevich/51/head branch November 23, 2019 15:16

eellison mentioned this pull request Dec 5, 2019

C++ / JIT Parity for ops in torch/functional.py and torch/tensor.py #30786

Open

driazati mentioned this pull request Jan 8, 2020

Move torch.isinf to C++ #31931

Closed

driazati pushed a commit that referenced this pull request Jan 8, 2020

Move torch.isinf to C++

b9c4819

As part of #30786, this moves `isinf` to C++. It's mostly the same as #28918.

driazati mentioned this pull request Jan 15, 2020

torch.isfinite not supported in TorchScript #32245

Closed

mruberry added the Merged label Oct 28, 2020

C++ API parity: isfinite #28918

C++ API parity: isfinite #28918

Uh oh!

Conversation

pbelevich commented Oct 30, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zou3519 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

zou3519 Nov 18, 2019

Choose a reason for hiding this comment

Uh oh!

pbelevich Nov 18, 2019

Choose a reason for hiding this comment

Uh oh!

zou3519 Nov 18, 2019

Choose a reason for hiding this comment

Uh oh!

pbelevich Nov 18, 2019

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Nov 19, 2019

Uh oh!

yf225 Nov 19, 2019

Choose a reason for hiding this comment

Uh oh!

yf225 Nov 19, 2019

Choose a reason for hiding this comment

Uh oh!

yf225 commented Nov 19, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

pbelevich commented Oct 30, 2019 •

edited

Loading