Eliminate tensor copies from compute_common_type_ in TensorIterator. #30018
Conversation
This requires refactoring at::native::result_type to operate as a state machine, processing the input types one at a time. There may be other places in the code base that could benefit from adopting this approach as well.
|
@ngimel This does appear to be slightly faster in my local benchmark of very-small-tensor workloads, but the noise is pretty high. If you have a better benchmark for TensorIterator setup costs, I'd be interested to know if this helps. |
|
Regular (fast) path where all types are the same returns before ever calling at::native::result_type, so this will help only (rare) cases where actual type promotion is happening https://github.com/pytorch/pytorch/pull/30018/files#diff-b47a50873394e38a005b4c1acd151957L115 |
|
@pytorchbot retest this please |
facebook-github-bot
left a comment
facebook-github-bot
left a comment
There was a problem hiding this comment.
@resistor has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
5 participants
This requires refactoring at::native::result_type to operate as a
state machine, processing the input types one at a time. There may
be other places in the code base that could benefit from adopting
this approach as well.