You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
pip install lightgm requires building from sources in my MBP (ARM Monterey 12.4), that wasn't the case in my previous MBP (Intel Big Sur). I assume the problem is with the ARM vs Intel part, not the Monterey vs Big Sur one.
Motivation
Building from sources requires previous installation of wheel, additional non-python dependencies (cmake) and is slower.
Description
pip install lightgbm should install binaries OOB.
Reacted by Tyler BoddySpargo, apesternikov, Jon Tong, Jean Poudroux, MaximeLaurenty, jtlz2, Andrew Martin, derfryday, Manny Setegn, Yasuhiro Fujita and 15 more
Unfortunately, we're blocked on this until we're able to build and test arm64 wheels on M1 Macs in a free CI service (#4843 (comment)). Without that ability to test, we can't take on this support.
Reacted by Kevin Montoya, Matheus Torquato, Jacopo Farina, elliottlin, Ruslan Yermak and RiccardoTonini
It's also possible that LightGBM might not be able to use scikit-build or cibuildwheel in its current configuration, because of the patterns that lightgbm's wheels use to build lib_lightgbm.so. See #5106 for significant discussion around this.
If you'd like to help with this issue, we're open to proposals and can answer questions as you try to develop. But providing M1 binaries in this project unfortunately won't be as easy as just copying that GitHub Actions config from scikit-learn, and will require a significant amount of time.
Otherwise, you can subscribe to this issue for updates.
It isn't available for free though. Could you look into getting Microsoft to sponsor such a runner for this project, so we can build Python wheels targeting the M1/M2 Macs?
Are there any updates for LightGBM to fully work on M1/M2 Macs?
From an outsider perspective it's a bit weird that now it looks that the blocking is github actions not been free but both GitHub and LightGBM are Microsoft companies/projects 🤔
I have tried several approaches over the last 2 years to get support from Microsoft or GitHub for native M1/M2 macOS builds in this project... so far none of those attenpts have resulted in any financial or development support towards resolving this.
I personally have an M2 laptop now (thanks to @guolinke !), so we may give up on native builds and attempt cross-compiling wheels in the near future...at least I'd be able to test them on that laptop.
Until then, you will have to continue building from source, and we're here to help with any issues you encounter doing that.
Reacted by Guolin Ke, Santiago Víquez and Kevin Trebing
Reacted by Kevin Trebing, José Morales, Santiago Víquez, Neto and Salil Mishra
@jameslamb thank you for the effort here. Is the issue purely financial? And if you don't mind me asking do you know roughly how much it might cost to add the M1 runners?
I am interested to see if I can help find a solution to this.
@jameslamb You can try FlyCI's M1 and M2 runners. They are on average 2x faster and 2x cheaper than GitHub's AND we have a free tier for OSS projects (see below).
Since your repo is public, FlyCI offers 500 mins/month of free M1 runner usage with the flyci-macos-large-latest-m1 runner for public projects.
Don't hesitate to contact us in case the free tier doesn't suit your needs or you experience any issues with the runners. Our team is here to support you!
Best Regards,
Veselina Radeva
Product Manager at FlyCI
they have a program for open source projects to apply to get free minutes, but I don't think LightGBM would qualify because Microsoft is a for-profit company and the terms say "your organization ... can't seek to make a profit by selling services" (docs).
I have been talking to @darbyfrey from GitLab about options there but for now that isn't available to use.
I am interested to see if I can help find a solution to this.
Thanks very much!
I think the best path available to us is to pay for CirrusCI and run a single job there that builds wheels + runs the tests natively on an arm64 macOS machine. I hope we could convince Microsoft to pay for that (@shiyu1994 could you comment on this?).
However... I don't want to wait around any longer not working on this, waiting for financial support. I'd like to focus on improved macOS support in the next release of LightGBM (after v4.3.0)
I think LightGBM should pursue cross-compilation. That's what conda-forge does (conda-forge docs) and at this point it's clear to me that providing experimental macOS arm64 wheels (cross-compiled but not tested) would be better for LightGBM's users than forcing them to build from source.
@stephenpardy do you or others from your organization have some capacity to help with adding x86_64 -> arm64 macOS cross-compilation support to LightGBM + the lightgbm Python package?
To start, I'd welcome some help generating a reproducible example for #5269 to help narrow down some of the build issues and to be able to test fixes (even if those fixes are only run interactively on our laptops).
documentation describing the architecture and security model
For example
would processes on the FlyCI runner be able to access this repo's secrets?
what software is installed on the runners? what will artifacts I build there be linking and testing against?
versions of macOS?
how long are logs retained?
To be clear, I'm not looking for answers to these questions. I am just mentioning some examples of reasons that I am not open to investing this project's maintainers' limited time and attention into using FlyCI at this time. I'd be happy to explore it in the future if some of those things were addressed.
Thank you @jameslamb for the detailed feedback. We really appreciate it!
We're already working on providing the mentioned information and we'll get back to you once it is available.
Definitely could be an option! Although I don't think we should resort to bundling and redistributing a copy of OpenMP lightly, as that can cause other portability and legal issues.
I don't think we should pursue cibuildwheel until we can reliably build working arm64 wheels using just cmake commands and things like build and pip. I strongly suspect that some of the issues I've mentioned above have fixes that lie in the project's CMake setup.
Well that statement from January turned out to not be true, for most definitions of "next few days" 😬
I am working actively on this now. For anyone subscribed to this issue, please stay subscribed... when we have wheels that I feel are ready, I'll push them to test-PyPI and ask for volunteers to try installing them.
GitHub recently on 2024-04-22 upgraded the GitHub Actions macos-latest image to macOS 14, and as part of that, changed the runners to be ARM64 instead of Intel/x86_64. This means that a wheel is no longer available if running pip install lightgbm on macos-latest, and pip will attempt to build the LightGBM Python package from source.
*** Building project with Ninja...
2024-04-29 16:00:35,864 - scikit_build_core - INFO - RUN: /opt/homebrew/bin/cmake --build /var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/tmp6n6643rr/build
ninja: error: '/opt/homebrew/opt/libomp/lib/libomp.dylib', needed by '/private/var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/pip-install-f6cmspx3/lightgbm_e756e84[80](https://github.com/jayqi/github-actions-sandbox/actions/runs/8881677094/job/24384593834#step:4:81)cc541db80a2e503a263c854/lib_lightgbm.so', missing and no known rule to make it
*** CMake build failed
I just published LightGBM's first arm64 wheels for macOS 😁
To test, install like this:
pip install 'lightgbm>=4.4.0'
pip will only give you a wheel if you're on macOS Sonoma (macOS 14.+), the latest version of macOS as of this writing (released September 2023).
I'll leave this open for a few days to allow for feedback, in case we've made some mistake and issues are discovered. But I think that as of lightgbm>=4.4.0, this is resolved.
Thank you all for your patience. We'd welcome contributions in the future if you'd like to come help out some more with LightGBM.
Reacted by 0x26res
Reacted by Julien Cornebise, Jay Qi, Kevin Trebing, Geoffrey Bolmier, CQD, Mathieu Rousseau, Florian Schwarb and ALLEN
This issue has been automatically closed because it has been awaiting a response for too long. When you have time to to work with the maintainers to resolve this issue, please post a new comment and it will be re-opened. If the issue has been locked for editing by the time you return to it, please open a new issue and reference this one. Thank you for taking the time to improve LightGBM!
Activity
jameslamb commentedon Jul 6, 2022
If you use
conda, you can get an arm64 precompiled binary fromconda-forge. See #4369 (comment).Otherwise, we already have some feature requests for providing binaries for the M1 Macs:
Unfortunately, we're blocked on this until we're able to build and test arm64 wheels on M1 Macs in a free CI service (#4843 (comment)). Without that ability to test, we can't take on this support.
dandreevv commentedon Aug 1, 2022
@jameslamb Hello!
scikit-learn has already use MacOS ARM jobs to build wheels (https://github.com/scikit-learn/scikit-learn/blob/main/.github/workflows/wheels.yml#L110).
Isn't that what you're waiting for?
jameslamb commentedon Aug 1, 2022
They are using
cibuildwheelto cross-compile ARM wheels for Mac on Macs that have Intel chips.https://github.com/scikit-learn/scikit-learn/blob/ec6efc57fac36820919184ffb5fae70a7e871b62/.github/workflows/wheels.yml#L111
https://github.com/scikit-learn/scikit-learn/blob/ec6efc57fac36820919184ffb5fae70a7e871b62/.github/workflows/wheels.yml#L141
Which we have been hesitant to rely on in this project because we don't have a way to actually test that wheels produced that way work on a real M1.
It's also possible that LightGBM might not be able to use
scikit-buildorcibuildwheelin its current configuration, because of the patterns thatlightgbm's wheels use to buildlib_lightgbm.so. See #5106 for significant discussion around this.If you'd like to help with this issue, we're open to proposals and can answer questions as you try to develop. But providing M1 binaries in this project unfortunately won't be as easy as just copying that GitHub Actions config from
scikit-learn, and will require a significant amount of time.Otherwise, you can subscribe to this issue for updates.
jameslamb commentedon Oct 2, 2023
@shiyu1994 Github Actions now supports a GitHub-hosted Apple Silicon runner: actions/runner-images#8439
It isn't available for free though. Could you look into getting Microsoft to sponsor such a runner for this project, so we can build Python wheels targeting the M1/M2 Macs?
santiviquez commentedon Nov 17, 2023
Are there any updates for LightGBM to fully work on M1/M2 Macs?
From an outsider perspective it's a bit weird that now it looks that the blocking is github actions not been free but both GitHub and LightGBM are Microsoft companies/projects 🤔
jameslamb commentedon Nov 17, 2023
I have tried several approaches over the last 2 years to get support from Microsoft or GitHub for native M1/M2 macOS builds in this project... so far none of those attenpts have resulted in any financial or development support towards resolving this.
I personally have an M2 laptop now (thanks to @guolinke !), so we may give up on native builds and attempt cross-compiling wheels in the near future...at least I'd be able to test them on that laptop.
Until then, you will have to continue building from source, and we're here to help with any issues you encounter doing that.
santiviquez commentedon Nov 17, 2023
So sad there hasn't been any financial support from them yet :/
I really appreciate your work and all the time you have put into this!
stephenpardy commentedon Jan 4, 2024
@jameslamb thank you for the effort here. Is the issue purely financial? And if you don't mind me asking do you know roughly how much it might cost to add the M1 runners?
I am interested to see if I can help find a solution to this.
radeva commentedon Jan 18, 2024
@jameslamb You can try FlyCI's M1 and M2 runners. They are on average 2x faster and 2x cheaper than GitHub's AND we have a free tier for OSS projects (see below).
Install Instructions
500 mins/month Free for Public Repos
Since your repo is public, FlyCI offers 500 mins/month of free M1 runner usage with the
flyci-macos-large-latest-m1runner for public projects.Don't hesitate to contact us in case the free tier doesn't suit your needs or you experience any issues with the runners. Our team is here to support you!
Best Regards,
Veselina Radeva
Product Manager at FlyCI
jameslamb commentedon Jan 22, 2024
No.
Even with budget and CI resources, this project is still facing a significant lack of people-hours doing work to move the project forward.
I don't know for sure. Here's what I've seen looking into this:
arm64wheels with maybe $100/month of spending thereThanks very much!
I think the best path available to us is to pay for CirrusCI and run a single job there that builds wheels + runs the tests natively on an
arm64macOS machine. I hope we could convince Microsoft to pay for that (@shiyu1994 could you comment on this?).However... I don't want to wait around any longer not working on this, waiting for financial support. I'd like to focus on improved macOS support in the next release of LightGBM (after v4.3.0)
I think LightGBM should pursue cross-compilation. That's what
conda-forgedoes (conda-forge docs) and at this point it's clear to me that providing experimental macOS arm64 wheels (cross-compiled but not tested) would be better for LightGBM's users than forcing them to build from source.@stephenpardy do you or others from your organization have some capacity to help with adding
x86_64 -> arm64macOS cross-compilation support to LightGBM + thelightgbmPython package?To start, I'd welcome some help generating a reproducible example for #5269 to help narrow down some of the build issues and to be able to test fixes (even if those fixes are only run interactively on our laptops).
jameslamb commentedon Jan 22, 2024
@radeva thanks very much for your post. I saw a similar on on XGBoost as well (dmlc/xgboost#9570 (comment)).
At this time, I don't support adding FlyCI to LightGBM's CI setup.
I'm nervous about the security implications of using an externally-provided hosted runner with GitHub Actions.
I looked at https://www.flyci.net/ and don't see the things I'd expect from a company that's offering CI services compliant with Apple's licensing terms for offering M1/M2/M3 services, like:
For example
To be clear, I'm not looking for answers to these questions. I am just mentioning some examples of reasons that I am not open to investing this project's maintainers' limited time and attention into using FlyCI at this time. I'd be happy to explore it in the future if some of those things were addressed.
radeva commentedon Jan 22, 2024
Thank you @jameslamb for the detailed feedback. We really appreciate it!
We're already working on providing the mentioned information and we'll get back to you once it is available.
jmoralez commentedon Jan 22, 2024
We could look at using cibuildwheel, I've used it in a couple of projects and it's worked great for me. It also allows to customize the build process, for example scikit-learn packages conda's OpenMP for MacOs in their wheels.
jameslamb commentedon Jan 23, 2024
Definitely could be an option! Although I don't think we should resort to bundling and redistributing a copy of OpenMP lightly, as that can cause other portability and legal issues.
I don't think we should pursue
cibuildwheeluntil we can reliably build workingarm64wheels using justcmakecommands and things likebuildandpip. I strongly suspect that some of the issues I've mentioned above have fixes that lie in the project'sCMakesetup.jameslamb commentedon Jan 30, 2024
It looks like GitHub is now providing free, GitHub-hosted macOS M1 runners!
https://github.blog/changelog/2024-01-30-github-actions-introducing-the-new-m1-macos-runner-available-to-open-source/
I'll test that in the next few days 😁
(thanks @leofang for telling me about this 😊 )
jameslamb commentedon Mar 29, 2024
Well that statement from January turned out to not be true, for most definitions of "next few days" 😬
I am working actively on this now. For anyone subscribed to this issue, please stay subscribed... when we have wheels that I feel are ready, I'll push them to test-PyPI and ask for volunteers to try installing them.
jayqi commentedon Apr 29, 2024
Just to document:
GitHub recently on 2024-04-22 upgraded the GitHub Actions
macos-latestimage to macOS 14, and as part of that, changed the runners to be ARM64 instead of Intel/x86_64. This means that a wheel is no longer available if runningpip install lightgbmonmacos-latest, and pip will attempt to build the LightGBM Python package from source.I got the following failure during build time:
Example: https://github.com/jayqi/github-actions-sandbox/actions/runs/8881677094/job/24384594174
Installing OpenMP explicitly beforehand solves the problem, e.g.,
Example: https://github.com/jayqi/github-actions-sandbox/actions/runs/8881677094/job/24384593509
Full job source: https://github.com/jayqi/github-actions-sandbox/blob/bf292c1afd105ce77f4c855893f4d1d1b0e99da9/.github/workflows/lightgbm-macos.yml#L53-L79
Thanks to @jameslamb for helping me debug this offline.
jameslamb commentedon Jun 15, 2024
I just published LightGBM's first arm64 wheels for macOS 😁
To test, install like this:
pip install 'lightgbm>=4.4.0'pipwill only give you a wheel if you're on macOS Sonoma (macOS 14.+), the latest version of macOS as of this writing (released September 2023).I'll leave this open for a few days to allow for feedback, in case we've made some mistake and issues are discovered. But I think that as of
lightgbm>=4.4.0, this is resolved.Thank you all for your patience. We'd welcome contributions in the future if you'd like to come help out some more with LightGBM.
jucor commentedon Jun 17, 2024
Awesome, thanks for the hard work!!
github-actions commentedon Jul 16, 2024
This issue has been automatically closed because it has been awaiting a response for too long. When you have time to to work with the maintainers to resolve this issue, please post a new comment and it will be re-opened. If the issue has been locked for editing by the time you return to it, please open a new issue and reference this one. Thank you for taking the time to improve LightGBM!