reformulate bce_with_logits to not use abs #2195
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
|
About |
soumith
approved these changes
Jul 24, 2017
Contributor
|
thanks Aly! |
jjsjann123
pushed a commit
to jjsjann123/pytorch
that referenced
this pull request
Nov 30, 2022
…#2195) * Add support for a symbolic output_shape for broadcast_in_dim in the Python Frontend * Fixed compilation of BroadcastInDimOpRecord with template specialization for defining expand sizes. Added SymbolicSizesRecord. Added python binding for symbolic_sizes(). * Fix up code with some suggestions from Ivan. Changed symbolic_sizes() to tensor_sizes(). * Add some tests and fix up python based Tensor and Scalar printing. * Added testing for tensor_sizes(). Made some minor changes to faciliate string captured definition testing. * Add comments and fix lint issues. * Add an output broadcast test with tensor_sizes(). * Added a test for tensor_size usage when each operand of a binary op has a broadcast. * Fix tensor_sizes() to reflect an expand in extent. Add appropriate test.
jagadish-amd
pushed a commit
to jagadish-amd/pytorch
that referenced
this pull request
Jun 2, 2025
- Changed to support new Hipblas 3.0.0 which is part of ROCm 7.0 is done. CP of ROCm@ec0c539 Co-authored-by: Pruthvi Madugundu <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a fix for the bug reported here in which
binary_cross_entropy_with_logitsgives the wrong gradient withinputandtargetare0.After some investigation, this is because the gradient of
absat0is0(see here)In this PR I have reformulated the numerically stable
binary_cross_entropy_with_logitsto not useabs(originally I used what tensorflow does, which is use abs).I guess in general we should think about whether we want the grad of
abs(0)to be0?