-
Notifications
You must be signed in to change notification settings - Fork 173
build(c): allow to build with system dependencies #2546
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Adds two build options to disable vendored dependencies fmt and nanoarrow - WITH_VENDORED_FMT - WITH_VENDORED_NANOARROW
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
David will have to speak to the build system since I am not an expert, but at least conceptually I have done this for nanoarrow for a few projects I've worked on. It might be helpful to specify a version (I think >= 0.5.0 would do it but nanoarrow is pre-1.0).
I also wonder if there's a way to use a pre-built driver (e.g., Python, R, a pre-built .so) in the project you are working on? The R and Python packages I believe both provide a way to locate the shared library (which may reduce you build to just the driver manager). Apologies if I'm missing something obvious!
|
Ah, is this for the vcpkg port? If so thank you for taking a stab at it. I agree with Dewey's suggestion of prefixing the options, otherwise this makes sense. And we should add a CI config to test it (I can follow up with that) |
Co-authored-by: Dewey Dunnington <[email protected]>
|
Thanks for all the suggestions. Yes, it is for vcpkg, I am also working on nanoarrow and fmt ports, see microsoft/vcpkg#43721
If 0.5.0 is the correct version, I can add it.
There's also a python port which could/shoule be used eventually. R is not bundled in the project I'm working on so I'll leave this exercise for someone else.
CI tests would be great, what are your plans, building on top of the vcpkg dependencies or something else? |
I think having one pipeline that enables vcpkg and tries to build with all deps from vcpkg would be the starting point. Let's get this building first though |
|
I can't explain the failing tests, in my opinion the libraries built by the pipelines should be completely equal to before (as the default is still to build with vendored dependencies). Does someone have a hint where to start looking? |
Again, the CMake here isn't something I'm familiar with, but you could try flipping the default (i.e., make building against system opt-in) in case there are some builds that for some reason don't see the default options). Something like |
|
I kicked all the jobs just to see. Something seems off in that it appears SQLite tests are failing in odd ways... |
|
Like this - I wonder if something else in the build environment changed? |
|
Last successful build here: https://github.com/apache/arrow-adbc/actions/runs/13381287796/job/37370182406#step:8:53 |
|
Note to self: the I'll continue with this cleanup effort once the tests are restored. P.S. it might be as easy as renaming the target here to unify this with the namespaced version arrow-adbc/c/vendor/nanoarrow/CMakeLists.txt Lines 18 to 26 in 8dade5b
|
|
I see this is breaking elsewhere so let me investigate, thanks |
|
Appears to be #2555 |
|
Alright, hopefully if you rebase these problems should be fixed! |
|
Filed #2557 to track adding the CI config |
|
@lidavidm any idea for the last failing test? |
|
The last failing test is known flaky (I need to go in and disable ASAN for Meson or otherwise figure out how to replicate it...) |
For the record, |
|
I believe the ASAN issue is an upstream bug, at least according to: It is confirmed to be flaky, so not sure how much luck we will have with a reproducer. Does it just affect Dremio? I wonder if we should have the ASAN test run exclude that driver for now. I'd personally hate to disable the entire ASAN suite if the scope of the issue can be limited to one driver |
|
Looks like this is green again - is this OK to re-review/merge? |
|
Yes , good from my side |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@paleolimbot seems we should upstream this change?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this CMakeLists.txt lives in ADBC...is the suggestion that nanoarrow should include a CMakeLists.txt with the bundled nanoarrow.c/h? It may also be time to just use CMake install or build-time dependency for nanoarrow (since our existing CMakeLists.txt now supports that and is tested against a number of scenarios).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, sorry, I see then.
Adds two build options to disable vendored dependencies fmt and nanoarrow
WITH_VENDORED_FMTWITH_VENDORED_NANOARROWThe default will still use the vendored copies