CC @envoyproxy/dependency-shepherds: Your approval is needed for changes made to (bazel/.*repos.*\.bzl)|(bazel/dependency_imports\.bzl)|(api/bazel/.*\.bzl)|(.*/requirements\.txt)|(.*\.patch).
envoyproxy/dependency-shepherds assignee is @RyanTheOptimist

k - possible workaround

$ bazel run @cmake-3.23.2-linux-x86_64//:bin/cmake
...

this isnt how we want to call it but i can get the target from one of the other rules - so we can make a //tools:cmake and //tools:ninja i reckon

Will you pursue this potential workaround further?

Will you pursue this potential workaround further?

im not immediately motivate to - i tried a bit further yesterday with mixed results

the workaround is definitely doable but its a toolchain and not a tool so it needs its env (configs and env vars) to work so less straightforward the way i was trying at least

possibly the above could be aliased in some way and then fed to the build rule

debugging the foreign cc targets in a bazel rule i also noticed that some of the bits were there to possibly use them directly in a genrule - discussions (rather than docs) suggest this should be possible

tbh tho - i have limited time to pursue it so much more up for reviewing any pr that retains all or part of the opentracing build as long as it does so without using the host cmake

feel free to pick/hack any parts from here or #27246 if this is a priority for you

Is the broken build visible anywhere in CI? What do I need to do to reproduce it? At the moment, the main branch seems to build fine?

trying rebuilding from scratch cc @daixiang0

I try to cherry-pick and try.

Another error, not about cmake:

root:[envoy]$ ./ci/run_envoy_docker.sh "ci/do_ci.sh bazel.dev.contrib"
No cmake here!
No remote cache is set, skipping setup remote cache.
ENVOY_SRCDIR=/source
ENVOY_BUILD_TARGET=//source/exe:envoy-static
ENVOY_BUILD_ARCH=x86_64
Setting test_tmpdir to /build/tmp.
Skip setting up Envoy Filter Example.
building using 12 CPUs
building for x86_64
clang toolchain with libc++ configured
bazel fastbuild build with contrib extensions and tests...
Building...
Building (type=fastbuild target=//contrib/exe:envoy-static debug=//contrib/exe:envoy-static.dwp name=envoy-contrib)...
ENVOY_BIN=contrib/exe/envoy-static
Starting local Bazel server and connecting to it...
DEBUG: Rule 'python3_10_x86_64-unknown-linux-gnu' indicated that a canonical reproducible form can be obtained by modifying arguments url = ["https://github.com/indygreg/python-build-standalone/releases/download/20220227/cpython-3.10.2+20220227-x86_64-unknown-linux-gnu-install_only.tar.gz"] and dropping ["urls"]
DEBUG: Repository python3_10_x86_64-unknown-linux-gnu instantiated at:
  /source/WORKSPACE:17:25: in <toplevel>
  /source/bazel/repositories_extra.bzl:17:31: in envoy_dependencies_extra
  /build/bazel_root/base/external/rules_python/python/repositories.bzl:533:26: in python_register_toolchains
Repository rule python_repository defined at:
  /build/bazel_root/base/external/rules_python/python/repositories.bzl:366:36: in <toplevel>
INFO: Analyzed target //contrib/exe:envoy-static (1009 packages loaded, 80702 targets configured).
INFO: Found 1 target...
ERROR: /source/contrib/vcl/source/BUILD:31:8: declared output 'contrib/vcl/source/external/libvppcom.a' was not created by genrule. This is probably because the genrule actually didn't create this output, or because the output was a directory and the genrule was run remotely (note that only the contents of declared file outputs are copied from genrules run remotely)
ERROR: /source/contrib/vcl/source/BUILD:31:8: declared output 'contrib/vcl/source/external/libvppinfra.a' was not created by genrule. This is probably because the genrule actually didn't create this output, or because the output was a directory and the genrule was run remotely (note that only the contents of declared file outputs are copied from genrules run remotely)
ERROR: /source/contrib/vcl/source/BUILD:31:8: declared output 'contrib/vcl/source/external/libsvm.a' was not created by genrule. This is probably because the genrule actually didn't create this output, or because the output was a directory and the genrule was run remotely (note that only the contents of declared file outputs are copied from genrules run remotely)
ERROR: /source/contrib/vcl/source/BUILD:31:8: declared output 'contrib/vcl/source/external/libvlibmemoryclient.a' was not created by genrule. This is probably because the genrule actually didn't create this output, or because the output was a directory and the genrule was run remotely (note that only the contents of declared file outputs are copied from genrules run remotely)
ERROR: /source/contrib/vcl/source/BUILD:31:8: declared output 'contrib/vcl/source/external/vppcom.h' was not created by genrule. This is probably because the genrule actually didn't create this output, or because the output was a directory and the genrule was run remotely (note that only the contents of declared file outputs are copied from genrules run remotely)
ERROR: /source/contrib/vcl/source/BUILD:31:8: Executing genrule //contrib/vcl/source:build failed: not all outputs were created or valid
Target //contrib/exe:envoy-static failed to build
ERROR: /source/contrib/exe/BUILD:23:16 Linking contrib/exe/envoy-static failed: not all outputs were created or valid
INFO: Elapsed time: 246.447s, Critical Path: 36.47s
INFO: 826 processes: 36 internal, 1 local, 789 processwrapper-sandbox.
FAILED: Build did NOT complete successfully

also now we still have some envoy_cmake, how to deal with them without cmake?

this pr was wip, and at very least will require to be rebased and probably updated

trying rebuilding from scratch cc @daixiang0

sorry, i was pinging you as you had highlighted that the build was broken - not so much about this PR

the PR i suggested picking when we spoke previously is this one #27246

I looked into it, and I think there are (at least) two alternative ways to fix this:

1. Ignore the BUILD (bazel) files in OpenTelemetry and just build using foreign_cc + cmake, like similar libraries. Make sure to delete the 3rd_party/BUILD file, otherwise it will get excluded by glob (since it stops at module boundaries). I have this basically working, but need to do some more verifications, since it changes the way the library is built and since this exposes an ABI, one needs to be a bit careful.
2. Generate the two files that CMake is used for by hand. Either, one could just "patch" the files with placeholders replaced with the hardcoded version or a preprocessor definition that could be patched in to the cc_library. Or do some regexing with sed over the CMakefile (without calling CMake). Maybe there would even be a nice way of doing this in Bazel. You need to replace the functionality of basically only these two lines https://github.com/opentracing/opentracing-cpp/blob/v1.5.1/CMakeLists.txt#L114-L115 which amounts to copying a file while replacing placeholders, which means you need to calculate/know OPENTRACING_VERSION_STRING ("1.5.1"), OPENTRACING_ABI_VERSION ("2") and whether OPENTRACING_BUILD_DYNAMIC_LOADING should be defined (yes, in our case always).

just build using foreign_cc + cmake, like similar libraries

this is what this PR attempts to do, altho in this case the cmake build is injected in the patch, rather than built separately

the problem i hit here is that it fails the asan test - afaict for ~unrelated reasons - it seems as tho cmake creates a throwaway function to test the compiler, and its this test that fails asan

we could move the build into envoy BUILD files - i suspect it wont make any difference - but worth a try

...enerate the two files that CMake is used for by hand. Either, one could just "patch" the files

not sure i follow entirely, but sounds complicated for somethign we are about to get rid of (even if we do keep it a little longer) - esp given there is a right way to do this

this is what this PR attempts to do, altho in this case the cmake build is injected in the patch, rather than built separately

OK, there are some things I don't understand there, and also the diff of a patch file is always a bit cumbersome to read. I tried something a bit different, maybe different enough to fix the next problem:

the problem i hit here is that it fails the asan test - afaict for ~unrelated reasons - it seems as tho cmake creates a throwaway function to test the compiler, and its this test that fails asan

OK, another thing I need to look into then. How do I start that build, or is that part of the bazel.dev CI target?

Also one question: If I run ./ci/run_envoy_docker.sh './ci/do_ci.sh bazel.dev', will it do incremental builds, or will it rebuild from scratch every time? Would I need to set up that remote build cache thing to be able to iterate quicker with that?

not sure i follow entirely, but sounds complicated for somethign we are about to get rid of

Especially if you no longer plan to update to newer versions, simply adding the generated version.h and config.h in the patch file and removing the genrule entirely from the opentracing BUILD file should be a very simple workaround. Should I submit a PR for that?

also the diff of a patch file is always a bit cumbersome to read.

yeah - i just built on what was there - in this case it has a BUILD file so that was being patched - probs not the cleanest way

will it do incremental builds,

yeah, there is a bit more to it than that, but basically yes

i would avoid the bazel.dev target tho - i think we car about the ones tested in CI

...simply adding the generated version.h and config.h in the patch file and removing the genrule entirely ...

this could work - try putting a PR together that does this and see if it can pass CI

Here's a first draft how it could look like, but I'm still running the build locally, so it may need some fixups: #27400

closing in favour of #27400