Allow Tensor to be compared to non-tensor types (like None) via eq and ne #695

alykhantejani · 2017-02-07T16:50:15Z

Hi,

I currently ran into the following problem when trying to build my own collate_fn for a torch.utils.data.DataLoader. For example, when loading data over the network it sometimes fails and my Dataset returns a tuple where one of the elements is None (depending if the input or target failed). I wanted to write a collate function to remove the data that failed to load by filtering as follows:

import torch
from torch.utils.data.dataloader import default_collate
dummy_tensor = torch.Tensor(5)

dummy_batch = [(dummy_tensor, dummy_tensor), (None, dummy_tensor)]

def filter_out_failed_data(batch):
    return default_collate(list(filter(lambda x: None not in x, batch)))

However, this does not work as you cannot compare a tensor with None via the equality check (==) and returns the following error:

TypeError: eq received an invalid combination of arguments - got (NoneType), but expected one of:
 * (float value)
      didn't match because some of the arguments have invalid types: (NoneType)
 * (torch.FloatTensor other)
      didn't match because some of the arguments have invalid types: (NoneType)

This PR is a patch to allow:

dummy_tensor == None >> False
dummy_tensor != None >> True

apaszke · 2017-02-07T17:20:51Z

Not sure if it's a good idea. It's actually quite ambiguous. I understand that the in operator doesn't do the thing you want, but you normally never compare with None (except for the is operator).

If you try that in numpy, it prints a deprecation warning, and the behaviour will be different than what you propose:

In [4]: q == None
/usr/local/bin/ipython3:1: FutureWarning: comparison to `None` will result in an elementwise object comparison in the future.
  #!/usr/local/opt/python3/bin/python3.5
Out[4]: False

apaszke · 2017-02-07T17:26:07Z

A simple solution would be this. It's not a lot longer:

default_collate(list(filter(lambda x: any(t is None for t in x), batch)))

alykhantejani · 2017-02-07T18:15:44Z

Yeah, I essentially did what you suggested above as a workaround, just thought it'd be nicer if it were automatic.

Although looking at the build logs, it appears I've broken some tests. and __eq__ /__ne__ actually don't return bools (in master) but instead ByteTensors, which is more in-line with what numpy does. Given this, I'm inclined to agree with you that this is not a good idea - we should stay as close to the numpy interface as possible

…9c90c8 Previous import was a4dcc47791eb127652f5aaddd51d8896d446a067 Included changes: - **[985af3f](onnx/onnx@985af3f)**: Update PythonAPIOverview.md (pytorch#738) <Dmytro Dzhulgakov> - **[b69be33](onnx/onnx@b69be33)**: Add backend test for upsample (pytorch#729) <Sebastian Meßmer> - **[0d9496e](onnx/onnx@0d9496e)**: Input test data of concat op should be float (pytorch#711) <Changming Sun> - **[20bcb8b](onnx/onnx@20bcb8b)**: Fix the spec for batchnorm and instancenorm (pytorch#733) <Lu Fang> - **[c9f825f](onnx/onnx@c9f825f)**: Refine a little bit about op spec. (pytorch#666) <Ke Zhang> - **[a484eb2](onnx/onnx@a484eb2)**: Fix an error in Conv doc (pytorch#731) <Lu Fang> - **[7410cc4](onnx/onnx@7410cc4)**: Fix incorrect package output paths (pytorch#730) <bddppq> - **[be546e2](onnx/onnx@be546e2)**: Improve optimizer's API and docs (pytorch#713) <Lu Fang> - **[c61506f](onnx/onnx@c61506f)**: Fix the shape inference python API (pytorch#716) <Lu Fang> - **[e9d4134](onnx/onnx@e9d4134)**: Fix cmake on windows when not building python extension (pytorch#728) <bddppq> - **[72187aa](onnx/onnx@72187aa)**: Add value_info support in make_graph (pytorch#726) <Lu Fang> - **[67b7d89](onnx/onnx@67b7d89)**: Fix gen_proto in cmake (pytorch#719) <bddppq> - **[fcb4ae3](onnx/onnx@fcb4ae3)**: docs rewording: Important Python Functions -> Python API Overview (pytorch#721) <anderspapitto> - **[24275d6](onnx/onnx@24275d6)**: Ignore .eggs directory when doing lint (pytorch#722) <bddppq> - **[54be8fa](onnx/onnx@54be8fa)**: Use cmake3 if it's available (pytorch#718) <bddppq> - **[b8c4238](onnx/onnx@b8c4238)**: Add python function docs (pytorch#714) <Lu Fang> - **[e177493](onnx/onnx@e177493)**: Remove unused cmake utils (pytorch#712) <bddppq> - **[72d6ad6](onnx/onnx@72d6ad6)**: Remove pycmd from CMake (pytorch#710) <bddppq> - **[93f0d40](onnx/onnx@93f0d40)**: Fix windows local build (pytorch#709) <Raymond Yang> - **[6734224](onnx/onnx@6734224)**: CMake fixes and setup.py cleanup (pytorch#706) <bddppq> - **[7f6a4fd](onnx/onnx@7f6a4fd)**: Add docs to explain important functions in ONNX Infra (pytorch#682) <Lu Fang> - **[f0f6b3d](onnx/onnx@f0f6b3d)**: fix hardmax test cases make output dtype same as input (pytorch#705) <Wenhao Hu> - **[c970f0c](onnx/onnx@c970f0c)**: Fix the Dummy backend (pytorch#701) <Lu Fang> - **[2af45df](onnx/onnx@2af45df)**: setup.py uses cmake build system (pytorch#606) <anderspapitto> - **[dfcaade](onnx/onnx@dfcaade)**: clean up unused variable left by removing consumed_input (pytorch#697) <bddppq> - **[accfc74](onnx/onnx@accfc74)**: Remove incorrect backend test (pytorch#700) <Lu Fang> - **[e558732](onnx/onnx@e558732)**: add max inclusive version to defs.get_schema function (pytorch#695) <Wenhao Hu> - **[16f02eb](onnx/onnx@16f02eb)**: add API to add domain to min/max version for extension. (pytorch#694) <Ke Zhang> - **[3e560dd](onnx/onnx@3e560dd)**: Fix doc for initializer (pytorch#690) <bddppq> - **[6cc4f53](onnx/onnx@6cc4f53)**: Add model save function (pytorch#692) <Lu Fang> - **[21eaf9b](onnx/onnx@21eaf9b)**: Changing the string discussing versions in operator specifications. (pytorch#691) <Niklas Gustafsson> - **[3b0cdf4](onnx/onnx@3b0cdf4)**: Minor code quality improvements in optimizer/ (pytorch#612) <Sebastian Meßmer> - **[641f126](onnx/onnx@641f126)**: Fix Gemm doc wording (pytorch#689) <bddppq> - **[4a0ec75](onnx/onnx@4a0ec75)**: Clarifies installation error message when external protobuf dependencies are missing (pytorch#684) <Daniel J. H> - **[960a2c3](onnx/onnx@960a2c3)**: Check outputs dtype in backend tests (pytorch#567) <bddppq> - **[1d7dee4](onnx/onnx@1d7dee4)**: Fix Average pool test cases converted from PyTorch (pytorch#677) <Lu Fang> - **[36d7fff](onnx/onnx@36d7fff)**: Fix Attribute default value pybind11 binding (pytorch#671) <bddppq> - **[0536866](onnx/onnx@0536866)**: git ignore .pytest_cache (pytorch#674) <bddppq> - **[afc84ac](onnx/onnx@afc84ac)**: Update README.md (pytorch#672) <Dmytro Dzhulgakov> - **[9d2b530](onnx/onnx@9d2b530)**: Revert "[Typing 1/3] Setup mypy type checker (pytorch#607)" (pytorch#667) <bddppq> - **[086727e](onnx/onnx@086727e)**: [Typing 1/3] Setup mypy type checker (pytorch#607) <Sebastian Meßmer> - **[5716e20](onnx/onnx@5716e20)**: Convert all Node tests to Model tests (pytorch#651) <bddppq> - **[6fe932a](onnx/onnx@6fe932a)**: Replace unittest.skip with custom exception (pytorch#659) <Dmytro Dzhulgakov> - **[ecac1c1](onnx/onnx@ecac1c1)**: Merge Rel 1.1.0 branch into master (pytorch#657) <Anirudh> - **[5cb999d](onnx/onnx@5cb999d)**: Minor cleanups to shape inference (pytorch#653) <anderspapitto> - **[f4acf28](onnx/onnx@f4acf28)**: Remove allowconsumed enforceconsumed from op schema. (pytorch#617) <Ke Zhang> - **[a8e4648](onnx/onnx@a8e4648)**: Adjust link flags when built in Windows Debug mode (pytorch#647) <Yinghai Lu> - **[7c009fe](onnx/onnx@7c009fe)**: Fix lint error in optimizer test (pytorch#656) <bddppq> - **[063d12f](onnx/onnx@063d12f)**: Fix optimizer split pass for models with constant output (pytorch#652) <bddppq>

Signed-off-by: David Pollack <[email protected]>

added torch.is_tensor check in TensorBase.__eq__

04589ca

alykhantejani closed this Feb 7, 2017

colesbury pushed a commit to colesbury/pytorch that referenced this pull request Feb 9, 2017

Static build support + Query CUDA driver, runtime versions (pytorch#695)

a96a8c8

ezyang added the open source label Jun 24, 2019

mrshenli pushed a commit to mrshenli/pytorch that referenced this pull request Apr 11, 2020

Fix CrossEntropyLoss in transformers tutorial (pytorch#695)

809f27e

Signed-off-by: David Pollack <[email protected]>

mcarilli pushed a commit to mcarilli/pytorch that referenced this pull request Mar 18, 2021

Mult gpu pw fusion fix (pytorch#695)

2535189

KyleCZH pushed a commit to KyleCZH/pytorch that referenced this pull request Sep 20, 2021

Fixing MAGMA with HIP issues (pytorch#695)

0b73152

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Allow Tensor to be compared to non-tensor types (like None) via eq and ne #695

Allow Tensor to be compared to non-tensor types (like None) via eq and ne #695

Uh oh!

alykhantejani commented Feb 7, 2017 •

edited

Loading

Uh oh!

apaszke commented Feb 7, 2017

Uh oh!

apaszke commented Feb 7, 2017

Uh oh!

alykhantejani commented Feb 7, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Allow Tensor to be compared to non-tensor types (like None) via __eq__ and __ne__ #695

Allow Tensor to be compared to non-tensor types (like None) via __eq__ and __ne__ #695

Uh oh!

Conversation

alykhantejani commented Feb 7, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

apaszke commented Feb 7, 2017

Uh oh!

apaszke commented Feb 7, 2017

Uh oh!

alykhantejani commented Feb 7, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Allow Tensor to be compared to non-tensor types (like None) via eq and ne #695

Allow Tensor to be compared to non-tensor types (like None) via eq and ne #695

alykhantejani commented Feb 7, 2017 •

edited

Loading