Skip to content

Comments

Support XCode 10 on Mac OS#1274

Closed
andrew-humu wants to merge 3 commits intopyenv:masterfrom
andrew-humu:#1219-support-xcode10+
Closed

Support XCode 10 on Mac OS#1274
andrew-humu wants to merge 3 commits intopyenv:masterfrom
andrew-humu:#1219-support-xcode10+

Conversation

@andrew-humu
Copy link

@andrew-humu andrew-humu commented Feb 5, 2019

Make sure you have checked all steps below.

Prerequisite

  • Please consider implementing the feature as a hook script or plugin as a first step.
    • pyenv has some powerful support for plugins and hook scripts. Please refer to Authoring plugins for details and try to implement it as a plugin if possible.
  • Please consider contributing the patch upstream to rbenv, since we have borrowed most of the code from that project.
    • We occasionally import the changes from rbenv. In general, you can expect changes made in rbenv will be imported to pyenv too, eventually.
    • Generally speaking, we prefer not to make changes in the core in order to keep compatibility with rbenv.
  • My PR addresses the following pyenv issue (if any)

Description

Fixes #1219

On MacOS 10.9 and later, uses xcrun --show-sdk-path to find SDK path, instead of relying on Mac OS headers being available at the system level. This is required because of a change in XCode 10's default behaviour.

https://developer.apple.com/documentation/xcode_release_notes/xcode_10_release_notes#3035624

Inspired by these comments:

#1219 (comment)

#1219 (comment)

#1219 (comment)

and this blog post:

https://medium.com/@pimterry/setting-up-pyenv-on-os-x-with-homebrew-56c7541fd331

@dakl
Copy link

dakl commented Feb 16, 2019

What's the status on this?

@chrahunt
Copy link
Member

@dakl this needs to be reviewed and tested. I don't personally have access to a mac, so I cannot test or do any of the sanity checks I would normally for a code review. Here are some details that might help things along:

Review

Anyone can do a code review. If you have access to a mac that is better since you can quickly address small questions yourself.

Things to consider for any reviewer (in order):

  1. What is the proposed change trying to fix? Is the approach a good one? - I'm not familiar with the file that was edited but it looks like it is only used for the build of the executable that may be used to speed up path resolution. It doesn't look like this is related to the Python build.
  2. Is there anything basically wrong, e.g. syntax or typos? - One note, shobj-conf is a shell script but the fix here looks like it is trying to invoke a subcommand using ${}. Not sure what shell that is compatible with but it isn't bash. Maybe $() was meant instead?
  3. Is the fix compatible with all relevant macOS and XCode versions? What versions was it tested on?
  4. What are the failure scenarios and how are they handled? e.g. xcrun not available, --show-sdk-path invalid option. Would there at least be something in the logs so we'd be able to debug it if someone has an issue?

Testing

Since this repo does not currently have a CI setup that tests builds, we rely on people on various platforms and configurations to report whether the fix worked for them. A 👍 is nice, but a comment stating that you were able to pull this PR locally and successfully compile (including your macOS and XCode version) is better.

@andrew-humu
Copy link
Author

andrew-humu commented Feb 19, 2019

Thanks for commenting, @chrahunt!

I'll just respond to a couple points with clarification:

  1. If I'm being honest, ${} is just syntax I copied from elsewhere in this file. From what I can tell it is standard bash syntax for variable expansion (different from $(), which evaluates a command in a subshell). It seems likely to me that if this were problematic, that would have surfaced before now.
    https://stackoverflow.com/questions/27472540/difference-between-and-in-bash

Whoops, I have read through my code and my answer again, and I recognize the error you were calling out! ${} is not appropriate for the xcrun invocation, which is a command and needs $() syntax. Thanks @chrahunt!


  1. I think this is a good question to raise. I'm not a MacOS guru, and I'm not sure what exactly the conditions are for xcrun to be available. However, I figured it was better to propose a solution and start the conversation than leave things in their current, broken state.

Don't try to use variable expansion syntax for command execution.
@joshfriend
Copy link
Member

I really want to get this fixed, so I will try to find some time to test it since I am a full time macOS user.

I just need to figure out how to undo the installation of the sdk to the legacy location that I did previously:

sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /

@jeffb4
Copy link

jeffb4 commented Mar 1, 2019

On 10.14.1 this patch still exhibits bad pyenv install behavior
CFLAGS="-I$(xcrun --show-sdk-path)/usr/include" pyenv install -v 3.6.7 works fine, whereas pyenv install on this branch still fails with zlib error

I do not have the global sdk installation Josh mentions above, and I'm happy to test any iterations you might have to fix this

@andrew-humu
Copy link
Author

Thanks @jeffb4! I will try to find some time!

Co-Authored-By: andrew-humu <[email protected]>
@cclauss
Copy link
Contributor

cclauss commented Apr 15, 2019

Can you please add the change from #1333 and make sure the tests pass? Thanks!

cclauss added a commit to cclauss/pyenv that referenced this pull request Apr 16, 2019
ktham added a commit to ktham/my_setup that referenced this pull request Apr 16, 2019
@joshfriend
Copy link
Member

Firstly: really sorry to let this sit so long, haven't done much python at work recently, which means my related side work also takes a hit.

Secondly: The approach in this PR doesn't work because if you trace the path of the scripts calling shobj-conf (which is what's modified here) upward, you get one path:

test/run
    -> src/configure
        -> shobj-conf

So basically at no point during pyenv install is the file modified here called.

I'm working on this right now actually, have it working, but it needs polishing (helpful errors if there is no xcrun, potentially falling back to homebrew zlib, idk)

@joshfriend
Copy link
Member

Closing in favor of #1339

Thank you for your valiant effort, @andrew-humu 😄

@joshfriend joshfriend closed this May 1, 2019
@andrew-humu andrew-humu deleted the #1219-support-xcode10+ branch May 1, 2019 06:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Install failed, "zlib not available" on macOS Mojave

7 participants