Skip to content

Prepare for TileDB's superbuild removal.#316

Merged
NullHypothesis merged 2 commits intomasterfrom
teo/install-tiledb
May 29, 2024
Merged

Prepare for TileDB's superbuild removal.#316
NullHypothesis merged 2 commits intomasterfrom
teo/install-tiledb

Conversation

@teo-tsirpanis
Copy link
Copy Markdown
Member

Preparation for SC-36913

This PR updates the shell scripts to use the install-tiledb target to build and install TileDB, instead of invoking make twice – once in the build directory, and once in the tiledb subdirectory.

TileDB is going to remove the superbuild and become a single CMake project without the tiledb subdirectory, and with this PR the Go API will be prepared for this.

@teo-tsirpanis teo-tsirpanis marked this pull request as draft May 28, 2024 22:27
@NullHypothesis
Copy link
Copy Markdown
Collaborator

PR looks good to me but I'm not sure why this test failed with:

-- Build files have been written to: /home/runner/work/TileDB-Go/TileDB-Go/TileDB/build
++ sudo make -j4 install-tiledb
Error: could not load cache

Do you happen to know what that means @teo-tsirpanis?

@teo-tsirpanis teo-tsirpanis marked this pull request as ready for review May 29, 2024 11:45
@teo-tsirpanis
Copy link
Copy Markdown
Member Author

I fixed the failures.

@teo-tsirpanis teo-tsirpanis changed the title Use the install-tiledb target to install TileDB. Prepare for TileDB's superbuild removal. May 29, 2024
@NullHypothesis
Copy link
Copy Markdown
Collaborator

I fixed the failures.

Thanks! It looks like commit 27bcb55 introduced a submodule. Was this intentional?

With the superbuild, `tiledb` will configure and build the TileDB inner project, and `install-tiledb` will build the `install` target of the inner project.
Without the superbuild, `tiledb` will build the TileDB library and `install-tiledb` will alias the `install` target, so there would not be any difference when the superbuild gets removed.
@teo-tsirpanis
Copy link
Copy Markdown
Member Author

Weird. I removed it, thanks for noticing!

@NullHypothesis NullHypothesis merged commit 7347e9e into master May 29, 2024
@NullHypothesis NullHypothesis deleted the teo/install-tiledb branch May 29, 2024 21:50
KiterLuc pushed a commit to TileDB-Inc/TileDB that referenced this pull request Jul 17, 2024
[SC-36912](https://app.shortcut.com/tiledb-inc/story/36912/remove-cmake-variable-tiledb-vcpkg-from-the-build)

[SC-36913](https://app.shortcut.com/tiledb-inc/story/36913/remove-superbuild)

Historically, the Core's build system has been using CMake external
projects to download and build external dependencies, and a "superbuild"
architecture to ensure a build order. With the advent of vcpkg, we have
stopped building the dependencies ourselves and instead rely on vcpkg
(or the "system" in general) to provide them for us. The superbuild has
thus became a relic of the past, consisting of only the inner `tiledb`
project when the new system is enabled (formerly by specifying
`-DTILEDB_VCPKG=ON`, now always).

This PR removes the superbuild. TileDB became a regular CMake project,
whose targets can be built directly without first building the outer
project, and then building the `build/tiledb` subdirectory. The CI
scripts were updated accordingly to not use the subdirectory.

This is inevitably a breaking change in the build system. For starters,
local development environment will certainly need to make a clean build
after this change. Furthermore, there will need to be changes in build
scripts to not build again on the `tiledb` subdirectory.

For examples of downstream migrations, TileDB-Inc/TileDB-Go#316 uses a
`make` invocation that has a similar effect both with and without the
superbuild, and conda-forge/tiledb-feedstock#290 uses a semi-documented
CMake option to disable the superbuild (which will have no effect after
the superbuild gets removed).

The majority of first-party downstreams (VCF, SOMA, MariaDB, Vector
Search, the Python and Java APIs) use the `install-tiledb` target, which
currently is defiend on the superbuild and builds the `install` target
in the inner `tiledb` project. With this PR the `install-tiledb` target
will be kept for compatibility, but alias to `install`.

~~I tried to build VCF with a TileDB external project from this branch,
but it fails with an error that will be fixed with #4989. I will try
again once that PR gets merged.~~ Never mind, VCF builds with the latest
changes.

---
TYPE: BUILD
DESC: The superbuild architecture of the build system has been removed
and TileDB is a regular CMake project. Build commands of the form `make
&& make -C tiledb <targets>` will have to be replaced by `make
<targets>`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants