Add pos_weight for binary_crossentropy_logits#10855
Add pos_weight for binary_crossentropy_logits#10855chenyuxyz merged 5 commits intotinygrad:masterfrom
Conversation
Changes |
| return (self.maximum(0) - Y * self + (1 + self.abs().neg().exp()).log())._do_reduction(reduction) | ||
| log_exp = (1 + self.abs().neg().exp()).log() | ||
| base = self.maximum(0) - Y * self | ||
| if pos_weight is None: |
There was a problem hiding this comment.
can pos_weight be integrated so that there's only one return code path?
There was a problem hiding this comment.
Thanks for the feedbacks. Can you elaborate a bit more what did you mean by that? Do you have an existing example in tinygrad doing so I can refer to?
Also, any rationale for doing so?
There was a problem hiding this comment.
I thought about it a bit, assuming that pos_weight is not provied, it's actually just [1, 1, 1, ...], i.e, all weights for the ration between positive and negative case is just one. If my understanding of what you want correctly, you want to have a pos_weight tensor as part of the UOps graph regardless if it's provided as an argument or not. Is that understanding correct? If so, yeah, I can make it so. Please let me know 🙏
As for rationale, I am not expert in terms of Tinygrad's compiler, but I guess doing so would make it much easier to optimize?
There was a problem hiding this comment.
we don't like if when it's not needed. unnecessary condition is hard to test well and maintain
fangpenlin
force-pushed
the
binary_crossentropy_logits_pos_weight
branch
from
03213a8 to
1318a2d
Compare
2 participants
No description provided.