[ci] Run functional tests using bitcoin-qt in one Travis job #12873

jamesob · 2018-04-03T20:41:07Z

As noted in #12843, we're currently missing bugs unique to bitcoin-qt during the Travis build because the functional test framework only uses bitcoind (and not bitcoin-qt).

Run a single Travis job under xvfb for framebuffer virtualization and using bitcoin-qt as the binary. I've also removed the NEED_XVFB configuration since our usage of xvfb is outdated per Travis' docs and the corresponding pre-build step is no longer necessary.

jnewbery · 2018-04-03T20:44:23Z

Concept ACK!

practicalswift · 2018-04-03T21:09:52Z

Concept ACK

Sjors · 2018-04-04T15:40:50Z

Concept ACK

jamesob · 2018-04-04T17:45:15Z

Closing and re-opening in an attempt to get Travis to kick off a build - I was flagged (and then unflagged) as abusing Travis.

fanquake · 2018-04-04T22:58:30Z

.travis.yml

Should remove this comment

Thanks! Fixed.

practicalswift · 2018-04-09T14:20:42Z

@jamesob Try force pushing a change instead (say after changing the commit message) - that should trigger a Travis rebuild. Closing and re-opening an issue will create a notification to followers to I think force pushing is a better way to wake up Travis :-)

jamesob · 2018-04-09T15:19:26Z

@practicalswift I'm really sorry about the spam, but force pushing isn't kicking off a Travis build for this PR (for reasons unknown to me). You can see above that @MarcoFalke had to do the same thing. I'll try opening a new PR next time I'd otherwise have to resort to an open/close. Sorry again for the trouble.

practicalswift · 2018-04-09T15:37:34Z

@jamesob I've now restarted the failing Travis build jobs :-)

practicalswift · 2018-04-09T15:41:04Z

@jamesob You may want to contact Travis CI support and make sure you're account has not been flagged on their side. That happens and they're usually fixing that quickly after receiving a ticket about it.

conscott · 2018-04-23T13:05:19Z

Concept ACK b42a408783b345d3f6c46e26a2d24dd00e1cff70

Seems one test is failing due to #12863.

For the actual bitcoin-qt job failing I tried to reproduce errors locally with Xvfd, manually setting BITCOIND to bitcoin-qt, but had no luck (all tests pass).

…print messages 80a5e59 [qa] Attach node index to test_node AssertionError and print messages (James O'Beirne) Pull request description: In the midst of fighting with #12873 it became apparent that there're a number of assertions and print statements which are emitted by test nodes but don't identify the node in question. This change makes debugging a bit easier by adding identifying information to non-logger test_node-related error messages. Tree-SHA512: 7cc86f2c81f4b3fdba15ec9a2d21a84c4b083629e845e82288087c3affbbdc5c68e74067621856cc97fe84fbc8cb4f5ca4977a51ef381e5d74515df8eb001239

jamesob · 2018-04-25T12:42:59Z

@conscott thanks for trying to reproduce locally. I've been doing same and while I'm unable to reproduce on my 16.04 host, I've been able to reproduce reliably on a 14.04 VM.

I think running the functional suite using the qt binary has uncovered a latent deadlock, perhaps due to 14.04's older default version of OpenSSL (1.0.1f) based on a tip from @TheBlueMatt.

Here's some relevant log output from the tests:

stderr:
bitcoin-qt: sync.cpp:101: void potential_deadlock_detected(const std::pair<void*, void*>&, const LockStack&, const LockStack&): Assertion `false' failed.
...
node0 2018-04-17T23:41:37.857938Z GUI: PaymentServer::LoadRootCAs: Loaded  145  root certificates 
node0 2018-04-17T23:41:38.262348Z POTENTIAL DEADLOCK DETECTED 
node0 2018-04-17T23:41:38.262376Z Previous lock order was: 
node0 2018-04-17T23:41:38.262386Z  (1) ppmutexOpenSSL[i]  util.cpp:107 
node0 2018-04-17T23:41:38.262403Z  (2) cs  ./addrman.h:508 
node0 2018-04-17T23:41:38.262416Z Current lock order is: 
node0 2018-04-17T23:41:38.262424Z  (2) cs  ./addrman.h:473 
node0 2018-04-17T23:41:38.262437Z  (1) ppmutexOpenSSL[i]  util.cpp:107

I'm going to continue to investigate.

maflcko · 2018-04-25T14:42:08Z

test/functional/test_framework/test_framework.py

Needs rebase due to conflict here

In particular, skip any assert_start_raises_init_error checks due to the way QT handles startup errors (presents the user with an error dialog which must be closed). In the future we may implement a better way of handling this.

As noted in bitcoin#12843, we're currently missing bugs unique to bitcoin-qt during the Travis build because the functional test framework only uses bitcoind (and not bitcoin-qt). Run a single Travis job under xvfb for framebuffer virtualization and using bitcoin-qt as the binary. Remove outdated NEED_XVFB Travis configuration.

DrahtBot · 2018-06-24T14:21:13Z

Needs rebase

maflcko · 2018-06-24T14:24:21Z

.travis.yml

    - if [ "$TRAVIS_EVENT_TYPE" = "cron" ]; then extended="--extended --exclude feature_pruning,feature_dbcrash"; fi
-    - if [ "$RUN_TESTS" = "true" ]; then DOCKER_EXEC test/functional/test_runner.py --combinedlogslen=4000 --coverage --quiet --failfast ${extended}; fi
+    - if [ "$RUN_TESTS_WITH_QT" = "true" ]; then
+          DOCKER_EXEC BITCOIND=$(DOCKER_EXEC find . -name 'bitcoin-qt' -executable) $FUNCTIONAL_TEST_CMD ${extended};


I believe you can just write BITCOIND=bitcoin-qt and it will look in the build dir automatically, as opposed to ./, what your find does.

laanwj · 2018-08-27T10:51:02Z

this has needed rebase for a long time
closing because of inactivity; let me know if you start work on this again and I'll reopen

…print messages Summary: 80a5e59 [qa] Attach node index to test_node AssertionError and print messages (James O'Beirne) Pull request description: In the midst of fighting with bitcoin/bitcoin#12873 it became apparent that there're a number of assertions and print statements which are emitted by test nodes but don't identify the node in question. This change makes debugging a bit easier by adding identifying information to non-logger test_node-related error messages. Tree-SHA512: 7cc86f2c81f4b3fdba15ec9a2d21a84c4b083629e845e82288087c3affbbdc5c68e74067621856cc97fe84fbc8cb4f5ca4977a51ef381e5d74515df8eb001239 Backport of Core PR13022 https://github.com/bitcoin/bitcoin/pull/13022/files Depends on D3914 Test Plan: `test_runner.py` Reviewers: #bitcoin_abc, deadalnix, Fabien Reviewed By: #bitcoin_abc, deadalnix, Fabien Subscribers: Fabien Differential Revision: https://reviews.bitcoinabc.org/D3915

…or and print messages 80a5e59 [qa] Attach node index to test_node AssertionError and print messages (James O'Beirne) Pull request description: In the midst of fighting with bitcoin#12873 it became apparent that there're a number of assertions and print statements which are emitted by test nodes but don't identify the node in question. This change makes debugging a bit easier by adding identifying information to non-logger test_node-related error messages. Tree-SHA512: 7cc86f2c81f4b3fdba15ec9a2d21a84c4b083629e845e82288087c3affbbdc5c68e74067621856cc97fe84fbc8cb4f5ca4977a51ef381e5d74515df8eb001239 fix 13022 Signed-off-by: pasta <[email protected]>

jnewbery added the Tests label Apr 3, 2018

jamesob force-pushed the 2018-04-03-travis-func-qt branch 2 times, most recently from 15e4bb0 to 76d6c0c Compare April 4, 2018 14:33

jamesob closed this Apr 4, 2018

jamesob reopened this Apr 4, 2018

jamesob force-pushed the 2018-04-03-travis-func-qt branch from 76d6c0c to 9be7c60 Compare April 4, 2018 15:02

jamesob force-pushed the 2018-04-03-travis-func-qt branch from 9be7c60 to 971e34a Compare April 4, 2018 17:38

jamesob closed this Apr 4, 2018

jamesob reopened this Apr 4, 2018

jamesob force-pushed the 2018-04-03-travis-func-qt branch from 971e34a to f9af997 Compare April 4, 2018 19:08

jamesob closed this Apr 4, 2018

jamesob reopened this Apr 4, 2018

jamesob force-pushed the 2018-04-03-travis-func-qt branch from f9af997 to 709bfae Compare April 4, 2018 20:01

jamesob closed this Apr 4, 2018

jamesob reopened this Apr 4, 2018

fanquake reviewed Apr 4, 2018

View reviewed changes

jamesob force-pushed the 2018-04-03-travis-func-qt branch from 709bfae to 0256aab Compare April 5, 2018 17:59

maflcko closed this Apr 5, 2018

maflcko reopened this Apr 5, 2018

jamesob force-pushed the 2018-04-03-travis-func-qt branch from 0256aab to 4d19340 Compare April 9, 2018 14:07

jamesob closed this Apr 9, 2018

jamesob reopened this Apr 9, 2018

maflcko reviewed Apr 25, 2018

View reviewed changes

test/functional/test_framework/test_framework.py Outdated

Copy link

Member

maflcko Apr 25, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs rebase due to conflict here

jamesob force-pushed the 2018-04-03-travis-func-qt branch from 15a2f8f to 2f83646 Compare April 25, 2018 15:00

jamesob mentioned this pull request Apr 26, 2018

Use thread names in logs and deadlock diagnostics #13099

Closed

maflcko mentioned this pull request May 8, 2018

qa: Remove unused option --srcdir #13188

Merged

jamesob mentioned this pull request May 11, 2018

Travis: Build tests on Ubuntu 18.04 with docker #13215

Merged

Make functional tests compatible with bitcoin-qt usage

85e2b57

In particular, skip any assert_start_raises_init_error checks due to the way QT handles startup errors (presents the user with an error dialog which must be closed). In the future we may implement a better way of handling this.

jamesob force-pushed the 2018-04-03-travis-func-qt branch 3 times, most recently from 384bb4f to 5fd09df Compare June 4, 2018 21:56

jamesob force-pushed the 2018-04-03-travis-func-qt branch from 5fd09df to 20323ee Compare June 4, 2018 22:09

DrahtBot added the Needs rebase label Jun 24, 2018

maflcko reviewed Jun 24, 2018

View reviewed changes

laanwj closed this Aug 27, 2018

laanwj removed the Needs rebase label Oct 24, 2019

bitcoin locked as resolved and limited conversation to collaborators Dec 16, 2021

[ci] Run functional tests using bitcoin-qt in one Travis job #12873

[ci] Run functional tests using bitcoin-qt in one Travis job #12873

Uh oh!

Conversation

jamesob commented Apr 3, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jnewbery commented Apr 3, 2018

Uh oh!

practicalswift commented Apr 3, 2018

Uh oh!

Sjors commented Apr 4, 2018

Uh oh!

jamesob commented Apr 4, 2018

Uh oh!

fanquake Apr 4, 2018

Choose a reason for hiding this comment

Uh oh!

jamesob Apr 5, 2018

Choose a reason for hiding this comment

Uh oh!

practicalswift commented Apr 9, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jamesob commented Apr 9, 2018

Uh oh!

practicalswift commented Apr 9, 2018

Uh oh!

practicalswift commented Apr 9, 2018

Uh oh!

conscott commented Apr 23, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jamesob commented Apr 25, 2018

Uh oh!

maflcko Apr 25, 2018

Choose a reason for hiding this comment

Uh oh!

DrahtBot commented Jun 24, 2018

Uh oh!

maflcko Jun 24, 2018

Choose a reason for hiding this comment

Uh oh!

laanwj commented Aug 27, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

jamesob commented Apr 3, 2018 •

edited

Loading

practicalswift commented Apr 9, 2018 •

edited

Loading

conscott commented Apr 23, 2018 •

edited

Loading