cython in setup_requires doesn't work #1317

Activity

1. jaraco commented on Apr 10, 2018

   

   jaraco

   on Apr 10, 2018

   Member

   More actions

   Sorry to hear that. You should be able to include setup_requires=['cython'], in your setup.py and it should work. Did you try that?
2. louisabraham commented on Apr 10, 2018

   

   louisabraham

   on Apr 10, 2018

   Author

   More actions

   I did that. "the behaviour of setup.py is not the same if cython was installed before or is installed using setup_requires"

   I think the problem is that Extension is evaluated before any setup_requires installation.
3. pganssle commented on Apr 10, 2018

   

   pganssle

   on Apr 10, 2018

   Member

   More actions

   Yeah, I've encountered this before, but I don't know that there's an easy solution to it. The extension and the setup_requires both need to be passed to the setup function, so Extension needs to be pulled in before setup is executed.

   In Rust extensions, I've seen this done, which basically solves the problem.

   Another possible option would be to add some mechanism for deferred execution, so the import statements can be nested in commands run only after setup_requires has installed the missing modules.

   I was under the impression that the one true solution to this is PEP 518, and setup-time requirements will be resolved from pyproject.toml.

   👍React with 👍3Reacted by Louis Abraham, David Ignacio Cortes and Ihor Slabkyy
4. louisabraham commented on Apr 10, 2018

   

   louisabraham

   on Apr 10, 2018

   Last edited by louisabraham

   Author

   More actions

   Nice one for rust. I think that's the best thing to do. A simple wrapper would call this for all the setup_requires. The only problem is that it would also mess with the installed package (the setup_requires are not installed in sites-packages but in a temp folder instead), so I guess doing it like that is explicit and simple enough.
5. jaraco commented on Apr 10, 2018

   

   jaraco

   on Apr 10, 2018

   Member

   More actions

   Indeed, the setup_requires options aren't available until after setup is executed, but I thought that the Extensions could be defined in such a way that they weren't evaluated until after setup_requires was evaluated. I went to a lot of work to get this all to work, and I'm pretty sure I tested it when I worked through the issue. Do you have a simple example that doesn't work?

   I created this minimal setup script with an empty foo.pyx and it runs with setup.py build without error:

   import setuptools
   
   setuptools.setup(
   	name='foo',
   	setup_requires=['cython'],
   	ext_modules=[
   		setuptools.Extension(name='foo', sources=['foo.pyx']),
   	],
   )
   

   Can you share an example of what doesn't work (preferably a minimal example)?
6. pganssle commented on Apr 10, 2018

   

   pganssle

   on Apr 10, 2018

   Member

   More actions

   In the past I've had problems with the Cython documentation examples, which use cythonize, because they tend to use from Cython.build import cythonize

   That also includes this example:

   setup(
       setup_requires=[
           'cython>=0.x',
       ],
       extensions = [Extension("*", ["*.pyx"])],
       cmdclass={'build_ext': Cython.Build.build_ext},
       ...
   )

   I'm not quite sure how that ever worked without cython already installed on the system. If Cython is special-cased in this way, maybe it's worth making a PR or issue against the documentation. I'm not sure that helps the rust situation I mentioned, but I guess pyproject.toml will solve that eventually.

   👍React with 👍1Reacted by Shixin Zhang
7. pganssle commented on Apr 10, 2018

   

   pganssle

   on Apr 10, 2018

   Last edited by pganssle

   Member

   More actions

   If we did want to support this use case, we could create a MagicMock-like class that defers imports in the case of missing modules until after setup_requires has fired off. Something like:

   from setuptools import deferred_import
   Cython = deferred_import('Cython')
   cythonize = Cython.cythonize

   Edit Thinking about this - this would probably be very complicated to do correctly. Probably better to just have a separate mechanism for firing off setup_requires before setup is called.

   👍React with 👍1Reacted by Vlad Blazhko
8. louisabraham commented on Apr 11, 2018

   

   louisabraham

   on Apr 11, 2018

   Author

   More actions

   @jaraco I can reproduce your behavior, on your example. I originally got the bug from bootphon/ABXpy#11, I will work on a minimal example.
9. jkp commented on May 8, 2018

   

   jkp

   on May 8, 2018

   More actions

   Just to add another voice - this issue is affecting me as well: I'll subscribe and watch for updates.
10. jaraco commented on May 9, 2018

    

    jaraco

    on May 9, 2018

    Member

    More actions

    Probably better to just have a separate mechanism for firing off setup_requires before setup is called.

    Indeed, and PEP518 does just that (via pyproject.toml). It does require pip 10. It obviates setup_requires, but only for environments where pip 10 can be relied upon. I figure about 18 months before it will be generally available. I had hoped the setup_requires functionality would be a useful stopgap, but for cases where it's not viable, I think the only option is to wait for pip 10 to become dominant (and of course validate/test those concepts in the meantime).

    I don't think there's anything left that setuptools can do here, so I'm closing the ticket, but I'm not seeking to stifle discussion. If there's something setuptools can or should do here, we can re-open and explore.

    👍React with 👍3Reacted by Louis Abraham, Jamie Kirkpatrick and Thomas Walther
11. jaraco

    closed this as completedon May 9, 2018