You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fused function dispatch is slow for Python calls #1385
Fused function dispatch currently generates a lot of generic code. In most cases, however, the number of distinct fused types in a function signature will be 1 or 2, so a short if-cascade would likely produce less code and be much faster.
The dispatch code is generated in make_fused_cpdef() in FusedNode.py. It generates Cython code. This could be optimised for the common case of a few (positional) arguments by dispatching directly based on the argument types, instead of comparing full signatures in a loop at runtime.
@scoder Thank you for pointing me to this issue. I did try searching for it, but I didn't see it. I'd like to try working on it as I get the time.
I'm not sure I fully understand the reasons behind the current implementation, though. As far as I can tell, it first attempts to find a matching Cython-space type name for each of the passed Python-object arguments and stores it in dest_sig, in the code generated by line 290 of FusedNode.py under ._fused_instance_checks() (as well as analogues for other special-cased objects like Numpy arrays), and then it iterates through and parses the signature of every single available permutation of specializations (presumably at O(k**n)), in the code generated around line 700 under make_fused_cpdef(), to search for exact matches to the received arguments (with a check to ignore unmatched arguments that are still None).
Assuming that the signatures for all available specializations are known ahead of time, though, and seeing how it's more or less checking for exact matches between dest_sig and signatures.keys(), why is it not possible or viable to just directly generate the signature of the required specialization from the inferred types of the input arguments in dest_sig?
I suppose this issue is officially closed now, but I'd just like to note that AFAIK, even in the best-case scenarios, running a fused cpdef no-op function from CPython still takes several times longer than running a statically typed Cython function, a dynamically typed function Python function, or even an untyped Cython function using dynamic calls to isinstance() to type local variables at run time.
In particular, I noted that the first, "instance_checks" stage in .make_fused_cpdef() that generates a sequence of type name strings for incoming arguments uses quite a bit of list indexing and assignment, which could probably be sped up considerably as a single list comprehension.
That shouldn't be too hard to do I think, but there's also a bunch of conditional branching and conditional code generation for conditional branching IIRC, representing a fair bit of special-casing for specific types and external libraries. It's also already O(n) IIRC, so I stopped short of trying to change it for #3366.
In the ideal scenario, I think it may also be possible to just do a nested if-cascade, and skip the string-based signature checking, matching, indexing, and lookup entirely by directly calling the appropriate specialization with appropriately typed arguments. This is a workaround that I'm still using in project code for performance-critical fused cpdef functions that accept dynamic arguments, even with O(n) dispatch.
@will-ca I closed this ticket because there is a major improvement now, but I agree that an if-cascade would likely give even better speedups. Basically, the more we can do at code generation time, the less time needs to be spent at runtime. And an if-cascade is perfect for code generation (and not so great for manual coding).
I'll reopen it, in case you or someone else wants to give it another try.
Activity
will-ca commentedon Feb 14, 2020
@scoder Thank you for pointing me to this issue. I did try searching for it, but I didn't see it. I'd like to try working on it as I get the time.
I'm not sure I fully understand the reasons behind the current implementation, though. As far as I can tell, it first attempts to find a matching Cython-space type name for each of the passed Python-object arguments and stores it in
dest_sig, in the code generated by line290ofFusedNode.pyunder._fused_instance_checks()(as well as analogues for other special-cased objects like Numpy arrays), and then it iterates through and parses the signature of every single available permutation of specializations (presumably atO(k**n)), in the code generated around line700undermake_fused_cpdef(), to search for exact matches to the received arguments (with a check to ignore unmatched arguments that are stillNone).Assuming that the signatures for all available specializations are known ahead of time, though, and seeing how it's more or less checking for exact matches between
dest_sigandsignatures.keys(), why is it not possible or viable to just directly generate the signature of the required specialization from the inferred types of the input arguments indest_sig?will-ca commentedon Apr 5, 2020
I suppose this issue is officially closed now, but I'd just like to note that AFAIK, even in the best-case scenarios, running a fused
cpdefno-op function from CPython still takes several times longer than running a statically typed Cython function, a dynamically typed function Python function, or even an untyped Cython function using dynamic calls toisinstance()to type local variables at run time.In particular, I noted that the first, "
instance_checks" stage in.make_fused_cpdef()that generates a sequence of type name strings for incoming arguments uses quite a bit of list indexing and assignment, which could probably be sped up considerably as a single list comprehension.That shouldn't be too hard to do I think, but there's also a bunch of conditional branching and conditional code generation for conditional branching IIRC, representing a fair bit of special-casing for specific types and external libraries. It's also already
O(n)IIRC, so I stopped short of trying to change it for #3366.In the ideal scenario, I think it may also be possible to just do a nested
if-cascade, and skip the string-based signature checking, matching, indexing, and lookup entirely by directly calling the appropriate specialization with appropriately typed arguments. This is a workaround that I'm still using in project code for performance-critical fusedcpdeffunctions that accept dynamic arguments, even withO(n)dispatch.scoder commentedon Apr 6, 2020
@will-ca I closed this ticket because there is a major improvement now, but I agree that an if-cascade would likely give even better speedups. Basically, the more we can do at code generation time, the less time needs to be spent at runtime. And an if-cascade is perfect for code generation (and not so great for manual coding).
I'll reopen it, in case you or someone else wants to give it another try.
MuellerSeb commentedon Jul 30, 2021
Why was this reopened?
da-woods commentedon Jul 30, 2021
Because there's probably still room for improvement.
scoder commentedon Aug 25, 2025
Fixed in #7098