Skip to content

Conversation

@0xB10C
Copy link
Contributor

@0xB10C 0xB10C commented Nov 28, 2022

As mentioned in #26571, the task running the USDT interface tests fail when run in docker. cc7335e in #25528 added that the tests are run in a VM in Cirrus CI. Running them locally in docker containers might not work:

  • We use bcc as tracing toolkit which requires the kernel headers to compile the BPF bytecode. As docker containers use the hosts kernel and don't run their own, there is a potential for mismatches between kernel headers available in the container and the host kernel. This results in a failure loading the BPF byte code.
  • Privilges are required to load the BPF byte code into the kernel. Normally, the docker containers aren't run with these.
  • We currently use an untrusted third-party PPA to install the bpfcc-tools package on Ubuntu 22.04. Using this on a local dev system could be a security risk.

To not hinder the ASan + LSan + UBSan part of the CI task, the USDT tests are disabled on non-CirrusCI runs.

As mentioned in bitcoin#26571, the task running the USDT interface tests
fail when run in docker. cc7335e
in bitcoin#25528 added that the tests are run in a **VM** in Cirrus CI.
Running them locally in docker containers might not work:

- We use [bcc] as tracing toolkit which requires the kernel headers
  to compile the BPF bytecode. As docker containers use the hosts
  kernel and don't run their own, there is a potential for mismatches
  between kernel headers available in the container and the host
  kernel. This results in a failure loading the BPF byte code.
- Privilges are required to load the BPF byte code into the kernel.
  Normally, the docker containers aren't run with these.
- We currently use an untrusted third-party PPA to install the
  bpfcc-tools package on Ubuntu 22.04. Using this on a local dev
  system could be a security risk.

To not hinder the ASan + LSan + UBSan part of the CI task, the USDT
tests are disabled on non-CirrusCI runs.

[bcc]: https://github.com/iovisor/bcc
@DrahtBot
Copy link
Contributor

DrahtBot commented Nov 28, 2022

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.
A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #25797 (build: Add CMake-based build system by hebasto)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@DrahtBot DrahtBot added the Tests label Nov 28, 2022
@0xB10C
Copy link
Contributor Author

0xB10C commented Nov 28, 2022

@MarcoFalke @hebasto let me know if that moves things in the right direction and if it helps you run the task locally.

Copy link
Member

@maflcko maflcko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, lgtm

export DOCKER_NAME_TAG=ubuntu:22.04
export NO_DEPENDS=1
export GOAL="install"
export BITCOIN_CONFIG="--enable-usdt --enable-zmq --with-incompatible-bdb --with-gui=qt5 CPPFLAGS='-DARENA_DEBUG -DDEBUG_LOCKORDER' --with-sanitizers=address,integer,undefined CC=clang CXX=clang++"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does --enable-usdt fail if the bpfcc tools are missing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this only depends on the systemtap sys/sdt.h used to insert the tracepoints. BCC (bpfcc-tools on Ubuntu) is only needed for the functional tests. If not available, the tests are skipped.

@DrahtBot DrahtBot mentioned this pull request Nov 30, 2022
16 tasks
@maflcko maflcko merged commit 9e59d21 into bitcoin:master Dec 2, 2022
@0xB10C 0xB10C deleted the 2022-11-ci-only-run-usdt-tests-on-cirrus-ci branch December 2, 2022 10:29
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Dec 2, 2022
@bitcoin bitcoin locked and limited conversation to collaborators Dec 2, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants