Skip to content

fused function dispatch is slow for Python calls #1385

Closed

Description

@robertwb

Fused function dispatch currently generates a lot of generic code. In most cases, however, the number of distinct fused types in a function signature will be 1 or 2, so a short if-cascade would likely produce less code and be much faster.

The dispatch code is generated in make_fused_cpdef() in FusedNode.py. It generates Cython code. This could be optimised for the common case of a few (positional) arguments by dispatching directly based on the argument types, instead of comparing full signatures in a loop at runtime.

Migrated from http://trac.cython.org/ticket/829

Activity

  1. added this to the wishlist milestone on Aug 16, 2016
  2. deleted a comment from robertwb on Feb 14, 2020
  3. will-ca commented on Feb 14, 2020

    @will-ca
    Contributor

    @scoder Thank you for pointing me to this issue. I did try searching for it, but I didn't see it. I'd like to try working on it as I get the time.

    I'm not sure I fully understand the reasons behind the current implementation, though. As far as I can tell, it first attempts to find a matching Cython-space type name for each of the passed Python-object arguments and stores it in dest_sig, in the code generated by line 290 of FusedNode.py under ._fused_instance_checks() (as well as analogues for other special-cased objects like Numpy arrays), and then it iterates through and parses the signature of every single available permutation of specializations (presumably at O(k**n)), in the code generated around line 700 under make_fused_cpdef(), to search for exact matches to the received arguments (with a check to ignore unmatched arguments that are still None).

    Assuming that the signatures for all available specializations are known ahead of time, though, and seeing how it's more or less checking for exact matches between dest_sig and signatures.keys(), why is it not possible or viable to just directly generate the signature of the required specialization from the inferred types of the input arguments in dest_sig?

  4. modified the milestones: wishlist, 3.0 on Apr 1, 2020
  5. will-ca commented on Apr 5, 2020

    @will-ca
    Contributor

    I suppose this issue is officially closed now, but I'd just like to note that AFAIK, even in the best-case scenarios, running a fused cpdef no-op function from CPython still takes several times longer than running a statically typed Cython function, a dynamically typed function Python function, or even an untyped Cython function using dynamic calls to isinstance() to type local variables at run time.

    In particular, I noted that the first, "instance_checks" stage in .make_fused_cpdef() that generates a sequence of type name strings for incoming arguments uses quite a bit of list indexing and assignment, which could probably be sped up considerably as a single list comprehension.

    That shouldn't be too hard to do I think, but there's also a bunch of conditional branching and conditional code generation for conditional branching IIRC, representing a fair bit of special-casing for specific types and external libraries. It's also already O(n) IIRC, so I stopped short of trying to change it for #3366.

    In the ideal scenario, I think it may also be possible to just do a nested if-cascade, and skip the string-based signature checking, matching, indexing, and lookup entirely by directly calling the appropriate specialization with appropriately typed arguments. This is a workaround that I'm still using in project code for performance-critical fused cpdef functions that accept dynamic arguments, even with O(n) dispatch.

  6. scoder commented on Apr 6, 2020

    @scoder
    Contributor

    @will-ca I closed this ticket because there is a major improvement now, but I agree that an if-cascade would likely give even better speedups. Basically, the more we can do at code generation time, the less time needs to be spent at runtime. And an if-cascade is perfect for code generation (and not so great for manual coding).

    I'll reopen it, in case you or someone else wants to give it another try.

  7. reopened this on Apr 6, 2020
  8. MuellerSeb commented on Jul 30, 2021

    @MuellerSeb

    Why was this reopened?

  9. da-woods commented on Jul 30, 2021

    @da-woods
    Collaborator

    Why was this reopened?

    Because there's probably still room for improvement.

  10. removed this from the 3.0 milestone on Jul 23, 2023
  11. added this to the 3.2 milestone on Aug 25, 2025
  12. scoder commented on Aug 25, 2025

    @scoder
    Contributor

    Fixed in #7098

  13. linked a pull request that will close this issueOptimise fused function dispatch #7098on Aug 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions