• travis: WIP - build and run tests on os: osx #13816 (travis: build and run tests on os: osx by scravy)
• travis: Enable qt for all jobs #13515 (travis: Enable Qt build for Windows , i686 and arm Linux by ken2812221)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

WARNING: This is a debug build - may result in slower benchmarks.

Maybe it would make more sense to introduce an additional stage "benchmark" which does a non-debug build and perform the benchmark in there?

Also it would clearly separate it visually when looking at travis. The only thing that sucks about this approach that it requires an additional build = additional travis run time. But given the extensive caching being done this should be okay.

Maybe it would make more sense to introduce an additional stage "benchmark" which does a non-debug build and perform the benchmark in there?

Please see https://bitcoinperf.com/ for these. The one here is specifically designed to run everything in less than a 1/10 of a second. (scaling=0.001)

Ah, cool! Didn't know about this site.

Since this site does exist – what is the purpose of running the benchmark in the CI pipeline though? Is it to check that a commit didn't break the benchmark, i.e. to check that is runs successfully, not how fast actually?

I think as a contributor who's new to the project I would love to see a comment communicating that intent next to it (if this is the intent). Some of the lines in the .travis.yml are incomprehensible (for example I have no idea which steps require the before-install step of removing everything that contains /opt/python from the path). That should help maintain it.

Updated commit and OP based on your feedback

utACK fa7a3a1

ACK fa7a3a1

utACK fa7a3a1

utACK fa7a3a1

The one here is specifically designed to run everything in less than a 1/10 of a second. (scaling=0.001)

Very good idea.