Skip to content

Commit 93f1578

Browse files
alopezzalopezz
authored
Add (experimental) ability to run agent.build against locally bazel-built rtloader (#48342)
### What does this PR do? This introduces a way to plug together the bazel-built rtloader with the existing, not-yet-bazelified, `agent.build`-driven Agent build. It adds: - A `dda inv rtloader.install-with-bazel` command that builds rtloader and all its runtime dependencies, and puts them under the local `dev` (on linux) or under `bin`, roughly matching the expectations on a working Agent. - A `--enable-bazel` flag to `dda inv agent.build` that will first run `rtloader.install-with-bazel` and then build the agent pointing at what bazel installed. This includes a few extra tweaks to facilitate the installation of all the dependencies from this temporary rule. ### Motivation Put the result of [ABLD-323](https://datadoghq.atlassian.net/browse/ABLD-323) in the hands of those that may want to try to reap some of the benefits locally. In particular, many contributors complain about problems with the linking to a python environment for rtloader when building it, which this helps with by installing one which should closely match the one used by the agent in actual builds. ### Describe how you validated your changes I ran `agent.build --enable-bazel` on all three platforms and ran the resulting Agent without any obvious errors, and with the "python home" being properly detected. ### Additional Notes This is opt-in behavior as there may be rough edges and we don't want to commit too much effort to making this a "blessed path", given that we're already working towards building the agent binary itself with Bazel, which would remove the need to have this kind of bridge. We won't be able to fully deprecate CMake usage until further down the bazel migration. ---- The two commands can also be run separately in this sequence, such that we can just run the second one on subsequent builds: ``` dda inv rtloader.install-with-bazel dda inv agent.build --exclude-rtloader --python-home-3=dev/embedded --embedded-path=dev/embedded ``` [ABLD-323]: https://datadoghq.atlassian.net/browse/ABLD-323?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Co-authored-by: alex.lopez <[email protected]>
1 parent 8b46676 commit 93f1578

File tree

9 files changed

+111
-10
lines changed

9 files changed

+111
-10
lines changed

‎deps/bzip2/overlay.BUILD.bazel‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -153,7 +153,7 @@ pkg_filegroup(
153153
":lib_files",
154154
],
155155
prefix = "embedded",
156-
visibility = ["@@//deps/openscap:__pkg__", "@@//packages:__subpackages__"],
156+
visibility = ["//visibility:public"],
157157
)
158158

159159
pkg_install(

‎deps/cpython.BUILD.bazel‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -436,6 +436,7 @@ pkg_filegroup(
436436
"@platforms//os:windows": "embedded3",
437437
"//conditions:default": "embedded",
438438
}),
439+
visibility = ["//visibility:public"],
439440
)
440441

441442
pkg_install(

‎deps/sqlite3.BUILD.bazel‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,7 @@ pkg_filegroup(
5757
":lib_files",
5858
],
5959
prefix = "embedded",
60+
visibility = ["//visibility:public"],
6061
)
6162

6263
pkg_install(

‎deps/xz.BUILD.bazel‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -144,6 +144,7 @@ pkg_filegroup(
144144
":lib_files",
145145
],
146146
prefix = "embedded",
147+
visibility = ["//visibility:public"],
147148
)
148149

149150
pkg_install(

‎deps/zlib.BUILD.bazel‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -145,6 +145,7 @@ pkg_filegroup(
145145
":lib_files",
146146
],
147147
prefix = "embedded",
148+
visibility = ["//visibility:public"],
148149
)
149150

150151
pkg_install(

‎omnibus/config/software/datadog-agent.rb‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -80,13 +80,13 @@
8080
if not windows_arch_i386? and ENV['WINDOWS_DDNPM_DRIVER'] and not ENV['WINDOWS_DDNPM_DRIVER'].empty?
8181
do_windows_sysprobe = "--windows-sysprobe"
8282
end
83-
command_on_repo_root "bazelisk run #{bazel_flags} -- //rtloader:install --destdir=\"#{install_dir}/bin/agent\""
83+
command_on_repo_root "bazelisk run #{bazel_flags} -- //rtloader:install --destdir=\"#{install_dir}"
8484
# Put the static rtloader library where it gets picked up by the go build linking to it
8585
command_on_repo_root "bazelisk run #{bazel_flags} -- //rtloader:install_static --destdir=\"#{project_dir}/rtloader/build/rtloader\""
8686
command "dda inv -- -e agent.build --exclude-rtloader --no-development --install-path=#{install_dir} --embedded-path=#{install_dir}/embedded #{do_windows_sysprobe} --flavor #{flavor_arg}", env: env, :live_stream => Omnibus.logger.live_stream(:info)
8787
command "dda inv -- -e systray.build", env: env, :live_stream => Omnibus.logger.live_stream(:info)
8888
else
89-
command_on_repo_root "bazelisk run #{bazel_flags} -- //rtloader:install --destdir='#{install_dir}/embedded'"
89+
command_on_repo_root "bazelisk run #{bazel_flags} -- //rtloader:install --destdir='#{install_dir}'"
9090
sh_ext = if linux_target? then "so" else "dylib" end
9191
command_on_repo_root "bazelisk run #{bazel_flags} -- //bazel/rules:replace_prefix" \
9292
" --prefix '#{install_dir}/embedded'" \

‎rtloader/BUILD.bazel‎

Lines changed: 36 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -228,7 +228,7 @@ pkg_files(
228228
name = "rtloader_three_pkg_files",
229229
srcs = [":datadog-agent-three"],
230230
prefix = select({
231-
"@platforms//os:windows": "",
231+
"@platforms//os:windows": "bin/agent",
232232
"//conditions:default": "lib",
233233
}),
234234
)
@@ -250,6 +250,10 @@ pkg_filegroup(
250250
":rtloader_with_symlinks",
251251
],
252252
}),
253+
prefix = select({
254+
"@platforms//os:windows": "",
255+
"//conditions:default": "embedded",
256+
}),
253257
visibility = ["//packages:__subpackages__"],
254258
)
255259

@@ -271,3 +275,34 @@ pkg_files(
271275
":rtloader_static": "libdatadog-agent-rtloader.a",
272276
},
273277
)
278+
279+
# Dev-only: installs rtloader + the full Python environment (CPython + deps) in one shot.
280+
# Used by the invoke tasks instead of separate per-dependency bazel run calls.
281+
282+
# OpenSSL shared libs only — exclude headers to avoid conflict with CPython's include/.
283+
pkg_filegroup(
284+
name = "openssl_libs_embedded",
285+
srcs = [
286+
"@openssl//:libcrypto_with_symlink",
287+
"@openssl//:libssl_with_symlink",
288+
],
289+
prefix = "embedded",
290+
)
291+
292+
pkg_filegroup(
293+
name = "python_env_files",
294+
srcs = [
295+
":all_files",
296+
":openssl_libs_embedded",
297+
"@bzip2//:all_files",
298+
"@cpython//:all_files",
299+
"@sqlite3//:all_files",
300+
"@xz//:all_files",
301+
"@zlib//:all_files",
302+
],
303+
)
304+
305+
pkg_install(
306+
name = "install_python_env",
307+
srcs = [":python_env_files"],
308+
)

‎tasks/agent.py‎

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@
3838
from tasks.libs.releasing.version import create_version_json
3939
from tasks.rtloader import clean as rtloader_clean
4040
from tasks.rtloader import install as rtloader_install
41+
from tasks.rtloader import install_with_bazel as rtloader_install_with_bazel
4142
from tasks.rtloader import make as rtloader_make
4243
from tasks.windows_resources import build_messagetable, build_rc, versioninfo_vars
4344

@@ -148,6 +149,7 @@ def build(
148149
agent_bin=None,
149150
run_on=None, # noqa: U100, F841. Used by the run_on_devcontainer decorator
150151
glibc=True,
152+
enable_bazel=False,
151153
):
152154
"""
153155
Build the agent. If the bits to include in the build are not specified,
@@ -159,11 +161,14 @@ def build(
159161
flavor = AgentFlavor[flavor]
160162

161163
if not exclude_rtloader and not flavor.is_iot():
162-
# If embedded_path is set, we should give it to rtloader as it should install the headers/libs
163-
# in the embedded path folder because that's what is used in get_build_flags()
164164
with gitlab_section("Install embedded rtloader", collapsed=True):
165-
rtloader_make(ctx, install_prefix=embedded_path, cmake_options=cmake_options)
166-
rtloader_install(ctx)
165+
if enable_bazel:
166+
bazel_embedded = rtloader_install_with_bazel(ctx)
167+
embedded_path = bazel_embedded
168+
python_home_3 = bazel_embedded
169+
else:
170+
rtloader_make(ctx, install_prefix=embedded_path, cmake_options=cmake_options)
171+
rtloader_install(ctx)
167172

168173
ldflags, gcflags, env = get_build_flags(
169174
ctx,

‎tasks/rtloader.py‎

Lines changed: 59 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
from invoke import task
1111
from invoke.exceptions import Exit
1212

13+
from tasks.libs.build.bazel import bazel
1314
from tasks.libs.common.utils import gitlab_section
1415

1516

@@ -71,15 +72,16 @@ def make(ctx, install_prefix=None, cmake_options=''):
7172
@task
7273
def clean(_):
7374
"""
74-
Clean up CMake's cache.
75+
Clean up CMake's cache and Bazel install artifacts under dev/.
7576
Necessary when the paths to some libraries found by CMake (for example Python) have changed on the system.
7677
"""
7778
dev_path = get_dev_path()
79+
embedded_path = os.path.join(dev_path, "embedded")
7880
include_path = os.path.join(dev_path, "include")
7981
lib_path = os.path.join(dev_path, "lib")
8082
rtloader_build_path = get_rtloader_build_path()
8183

82-
for p in [include_path, lib_path, rtloader_build_path]:
84+
for p in [embedded_path, include_path, lib_path, rtloader_build_path]:
8385
try:
8486
shutil.rmtree(p)
8587
print(f"Successfully cleaned '{p}'")
@@ -93,6 +95,61 @@ def install(ctx):
9395
run_make_command(ctx, "install")
9496

9597

98+
@task
99+
def install_with_bazel(ctx):
100+
"""
101+
Install rtloader, Python, and Python's shared-library dependencies using Bazel.
102+
103+
This is intended for local development alongside `agent.build`, as a way to leverage Bazel
104+
while we're still in a transitional stage. It installs an "embedded" environment which includes
105+
Python, under `dev/embedded`.
106+
107+
Returns the embedded path.
108+
"""
109+
dev_path = get_dev_path()
110+
111+
if sys.platform == "win32":
112+
bin_dir = os.path.join(os.path.dirname(os.path.abspath(dev_path)), "bin")
113+
114+
# Put static rtloader lib where the go build expects to find it
115+
bazel(
116+
ctx,
117+
"run",
118+
"//rtloader:install_static",
119+
"--",
120+
f"--destdir={os.path.join(get_rtloader_build_path(), 'rtloader')}",
121+
)
122+
123+
# Install rtloader and cpython where the Agent expects them at runtime
124+
bazel(ctx, "run", "//rtloader:install", "--", f"--destdir={os.path.dirname(bin_dir)}")
125+
bazel(ctx, "run", "@cpython//:install", "--", f"--destdir={bin_dir}")
126+
127+
return os.path.join(bin_dir, "embedded3")
128+
129+
embedded = os.path.join(dev_path, "embedded")
130+
131+
# Install rtloader + CPython + all Python deps
132+
bazel(ctx, "run", "//rtloader:install_python_env", "--", f"--destdir={dev_path}")
133+
134+
# Patch RPATH in all installed binaries/libraries so they find their
135+
# dependencies under <embedded>/lib at runtime.
136+
files_to_patch = [
137+
f
138+
for f in ctx.run(
139+
f"find {embedded} -type f \\( -name '*.so' -o -name '*.so.*' -o -name '*.dylib'"
140+
f" -o -name '*.pc' -o -path '*/bin/*' \\)",
141+
hide="out",
142+
)
143+
.stdout.strip()
144+
.split("\n")
145+
if f
146+
]
147+
if files_to_patch:
148+
bazel(ctx, "run", "//bazel/rules:replace_prefix", "--", "--prefix", embedded, *files_to_patch)
149+
150+
return embedded
151+
152+
96153
@task
97154
def test(ctx):
98155
with gitlab_section("Run rtloader tests", collapsed=True):

0 commit comments

Comments
 (0)