Strided masked reduction: mean (2nd try) by pearu · Pull Request #67088 · pytorch/pytorch

pearu · 2021-10-22T12:06:28Z

Stack from ghstack:

-> Strided masked reduction: mean (2nd try) #67088

Differential Revision: D32070264

[ghstack-poisoned]

pytorch-probot · 2021-10-22T12:06:30Z

CI Flow Status

⚛️ CI Flow

Ruleset - Version: v1
Ruleset - File: https://github.com/pytorch/pytorch/blob/171dd33e46afe178e6ebf68bf6cc62c37354e42c/.github/generated-ciflow-ruleset.json
PR ciflow labels: ciflow/default

Workflows	Labels (bold enabled)	Status
Triggered Workflows
linux-bionic-py3.6-clang9	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/noarch`, `ciflow/xla`	✅ triggered
linux-vulkan-bionic-py3.6-clang9	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/vulkan`	✅ triggered
linux-xenial-cuda11.3-py3.6-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/default`, `ciflow/linux`	✅ triggered
linux-xenial-py3-clang5-mobile-build	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/mobile`	✅ triggered
linux-xenial-py3-clang5-mobile-custom-build-dynamic	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/mobile`	✅ triggered
linux-xenial-py3-clang5-mobile-custom-build-static	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/mobile`	✅ triggered
linux-xenial-py3.6-clang7-asan	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/sanitizers`	✅ triggered
linux-xenial-py3.6-clang7-onnx	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/onnx`	✅ triggered
linux-xenial-py3.6-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`	✅ triggered
linux-xenial-py3.6-gcc7	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`	✅ triggered
linux-xenial-py3.6-gcc7-bazel-test	`ciflow/all`, `ciflow/bazel`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`	✅ triggered
win-vs2019-cpu-py3	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/win`	✅ triggered
win-vs2019-cuda11.3-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/default`, `ciflow/win`	✅ triggered
Skipped Workflows
caffe2-linux-xenial-py3.6-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`	🚫 skipped
docker-builds	`ciflow/all`	🚫 skipped
libtorch-linux-xenial-cuda10.2-py3.6-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`	🚫 skipped
libtorch-linux-xenial-cuda11.3-py3.6-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`	🚫 skipped
linux-bionic-cuda10.2-py3.9-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/slow`	🚫 skipped
linux-xenial-py3-clang5-mobile-code-analysis	`ciflow/all`, `ciflow/linux`, `ciflow/mobile`	🚫 skipped
parallelnative-linux-xenial-py3.6-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`	🚫 skipped
periodic-libtorch-linux-xenial-cuda11.1-py3.6-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-linux-xenial-cuda10.2-py3-gcc7-slow-gradcheck	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`, `ciflow/slow`, `ciflow/slow-gradcheck`	🚫 skipped
periodic-linux-xenial-cuda11.1-py3.6-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-win-vs2019-cuda11.1-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/scheduled`, `ciflow/win`	🚫 skipped

You can add a comment to the PR and tag @pytorchbot with the following commands:

# ciflow rerun, "ciflow/default" will always be added automatically
@pytorchbot ciflow rerun

# ciflow rerun with additional labels "-l <ciflow/label_name>", which is equivalent to adding these labels manually and trigger the rerun
@pytorchbot ciflow rerun -l ciflow/scheduled -l ciflow/slow

For more information, please take a look at the CI Flow Wiki.

facebook-github-bot · 2021-10-22T12:06:33Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/67088
📄 Preview docs built from this PR
📄 Preview C++ docs built from this PR
🔧 Opt-in to CIFlow to control what jobs run on your PRs

💊 CI failures summary and remediations

As of commit 171dd33 (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

[ghstack-poisoned]

ghstack-source-id: 93cf74e Pull Request resolved: #67088

[ghstack-poisoned]

ghstack-source-id: 88fe849 Pull Request resolved: #67088

pearu · 2021-10-22T18:07:29Z

I can reproduce the RuntimeError: cov(): weights sum to zero, can't be normalized error locally. It turns out that make_tensor in

pytorch/torch/testing/_internal/common_methods_invocations.py

Line 4351 in f2582a5

    
           fweights = make_tensor((num_observations,), device, torch.int, low=0, high=10, requires_grad=requires_grad)

produces a tensor with all items being zeros but such a tensor is invalid for fweights.
Using low=1 fixes the issue... ~~Update: regenerating the cov sample fixes the issue~~, see #67039 .

[ghstack-poisoned]

ghstack-source-id: a909057 Pull Request resolved: #67088

[ghstack-poisoned]

ghstack-source-id: fd34b77 Pull Request resolved: #67088

cpuhrsch · 2021-10-26T00:21:11Z

@cpuhrsch has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: Pull Request resolved: #67088 Test Plan: Imported from OSS Reviewed By: pbelevich Differential Revision: D31914868 Pulled By: cpuhrsch fbshipit-source-id: beda9d32ea65bcae31c2c0181f95ad23c6631075

ngimel · 2021-10-26T21:16:14Z

Unlanded, broke windows builds, e.g. https://github.com/pytorch/pytorch/runs/4014552209?check_suite_focus=true

facebook-github-bot · 2021-10-26T21:20:01Z

This pull request has been reverted by 273ab55. To re-land this change, follow these steps.

facebook-github-bot · 2021-10-26T21:20:19Z

This pull request has been reverted by 526053706411ec3f2f8024849d951d6bf37b53a4. To re-land this change, follow these steps.

pearu · 2021-10-26T22:47:30Z

The failure is likely related to numpy/numpy#18552 which has a fix available in numpy 1.20.2.
The resolution is to replace 1.20.0 with 1.20.2 in
https://github.com/pytorch/pytorch/pull/67088/files#diff-18a0c3e7cd07089b34760028c387d39de1aa393172f1d5f123692106b3a3d144R11079

ngimel · 2021-10-27T00:23:10Z

TIL about np.mean with where argument. @cpuhurch, @mruberry, should we be adding where to our reductions?

Differential Revision: [D31914868](https://our.internmc.facebook.com/intern/diff/D31914868) [ghstack-poisoned]

ghstack-source-id: 1437d5b Pull Request resolved: #67088

pearu · 2021-11-01T14:42:14Z

@cpuhrsch could you try importing this PR?

cpuhrsch · 2021-11-01T17:03:09Z

@ngimel NumPy defines the input to where in its general documentation for reductions as "A boolean array which is broadcasted to match the dimensions of array, and selects elements to include in the reduction."

I think that functionality is covered by the masked reduction RFC, but I could see us potentially wanting to add this for NumPy compatibility? There is also some discussion around using where in the RFC itself, but only to decide on the definition of the mask semantics (i.e. does True mean inclusion or exclusion).

cpuhrsch · 2021-11-01T17:28:58Z

Just for completeness note that the definition of mask differs between NumPy's MaskedArray and the where argument.

>>> npm1 = np.ma.masked_array([0., 1., 2.], [False, False, True])
>>> npm1
masked_array(data=[0.0, 1.0, --],
             mask=[False, False,  True],
       fill_value=1e+20)
>>> npm1.sum()
1.0
>>> np.sum([0., 1, 2], where=[False, False, True])
2.0

ngimel · 2021-11-01T17:38:49Z

@cpuhrsch I agree that where functionality is covered by the masked reduction RFC, my question was about adding it for better numpy compat.

cpuhrsch · 2021-11-01T17:46:40Z

@ngimel - Ideally we're pythonic in the sense of "one obvious way to do it", but I can also see that there's value in adding this feature for compatibility. I do worry though that we'll end up in a situation where people are doing the usual "masked out row" or "fully masked out" inputs and then run into gradient issues, which we're currently working on resolving via MaskedTensor.

cpuhrsch · 2021-11-01T18:33:14Z

@pearu - could you rebase?

Differential Revision: [D31914868](https://our.internmc.facebook.com/intern/diff/D31914868) [ghstack-poisoned]

ghstack-source-id: 3b9a69c Pull Request resolved: #67088

pearu · 2021-11-01T18:43:43Z

@cpuhrsch rebase done.

anjali411 · 2021-11-01T18:49:27Z

@anjali411 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: Pull Request resolved: #67088 Stack from [ghstack](https://github.com/ezyang/ghstack): * __->__ #67088 Test Plan: Imported from OSS Reviewed By: anjali411 Differential Revision: D32070264 Pulled By: cpuhrsch fbshipit-source-id: 08a91550dd24fb0f51abf06591a0e26186c4f9f9

Strided masked reduction: mean (2nd try)

3f3daee

[ghstack-poisoned]

pytorch-probot bot added the ciflow/default label Oct 22, 2021

pearu mentioned this pull request Oct 22, 2021

Improve assert failure message in test_get_torch_func_signature_exhaustive #67039

Closed

facebook-github-bot added the cla signed label Oct 22, 2021

pytorchbot added the open source label Oct 22, 2021

Update on "Strided masked reduction: mean (2nd try)"

4895a2c

[ghstack-poisoned]

pearu added the module: reductions label Oct 22, 2021

Update on "Strided masked reduction: mean (2nd try)"

e8cdef4

[ghstack-poisoned]

pearu added a commit that referenced this pull request Oct 22, 2021

Temporarily disable _apply_docstring_templates on all operations.

d1c3673

ghstack-source-id: 93cf74e Pull Request resolved: #67088

Update on "Strided masked reduction: mean (2nd try)"

c6a9b3c

[ghstack-poisoned]

Update on "Strided masked reduction: mean (2nd try)"

edc78c0

[ghstack-poisoned]

pearu added a commit that referenced this pull request Oct 22, 2021

Restore _masked.mean OpInfo but disable ref.

fca81ce

ghstack-source-id: 88fe849 Pull Request resolved: #67088

Update on "Strided masked reduction: mean (2nd try)"

aa06a65

[ghstack-poisoned]

pearu added a commit that referenced this pull request Oct 22, 2021

Re-enable _masked.mean ref.

140871d

ghstack-source-id: a909057 Pull Request resolved: #67088

pearu requested review from cpuhrsch and ngimel October 22, 2021 19:57

IvanYashchuk mentioned this pull request Oct 23, 2021

Strided masked reduction: mean. #66784

Closed

Update on "Strided masked reduction: mean (2nd try)"

e231766

[ghstack-poisoned]

pearu added a commit that referenced this pull request Oct 25, 2021

Re-enable _masked.mean ref.

f01ffc0

ghstack-source-id: fd34b77 Pull Request resolved: #67088

facebook-github-bot closed this Oct 26, 2021

facebook-github-bot added the Reverted label Oct 26, 2021

pearu reopened this Nov 1, 2021

Update on "Strided masked reduction: mean (2nd try)"

6c773c9

Differential Revision: [D31914868](https://our.internmc.facebook.com/intern/diff/D31914868) [ghstack-poisoned]

Update on "Strided masked reduction: mean (2nd try)"

bc410c7

Differential Revision: [D31914868](https://our.internmc.facebook.com/intern/diff/D31914868) [ghstack-poisoned]

pearu added a commit that referenced this pull request Nov 1, 2021

Require numpy 1.20.2 for _masked.mean ref.

0ff6bfe

ghstack-source-id: 1437d5b Pull Request resolved: #67088

Update on "Strided masked reduction: mean (2nd try)"

171dd33

Differential Revision: [D31914868](https://our.internmc.facebook.com/intern/diff/D31914868) [ghstack-poisoned]

pearu added a commit that referenced this pull request Nov 1, 2021

Require numpy 1.20.2 for _masked.mean ref.

5a4cfc3

ghstack-source-id: 3b9a69c Pull Request resolved: #67088

janeyx99 mentioned this pull request Nov 1, 2021

[Meta] CI Revert Tracker #66178

Closed

facebook-github-bot closed this Nov 1, 2021

facebook-github-bot deleted the gh/pearu/9/head branch December 2, 2021 15:15

Conversation

pearu commented Oct 22, 2021 • edited by anjali411 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-probot bot commented Oct 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚛️ CI Flow

Uh oh!

facebook-github-bot commented Oct 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

💊 CI failures summary and remediations

Uh oh!

pearu commented Oct 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cpuhrsch commented Oct 26, 2021

Uh oh!

ngimel commented Oct 26, 2021

Uh oh!

facebook-github-bot commented Oct 26, 2021

Uh oh!

facebook-github-bot commented Oct 26, 2021

Uh oh!

pearu commented Oct 26, 2021

Uh oh!

ngimel commented Oct 27, 2021

Uh oh!

pearu commented Nov 1, 2021

Uh oh!

cpuhrsch commented Nov 1, 2021

Uh oh!

cpuhrsch commented Nov 1, 2021

Uh oh!

ngimel commented Nov 1, 2021

Uh oh!

cpuhrsch commented Nov 1, 2021

Uh oh!

cpuhrsch commented Nov 1, 2021

Uh oh!

pearu commented Nov 1, 2021

Uh oh!

anjali411 commented Nov 1, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

pearu commented Oct 22, 2021 •

edited by anjali411

Loading

pytorch-probot bot commented Oct 22, 2021 •

edited

Loading

facebook-github-bot commented Oct 22, 2021 •

edited

Loading

pearu commented Oct 22, 2021 •

edited

Loading