Skip to content

gitian: add statically built variant of bitcoind/bitcoin-cli for linux build#3914

Merged
laanwj merged 1 commit intobitcoin:masterfrom
laanwj:2014_03_statically_built_bitcoind
Apr 8, 2014
Merged

gitian: add statically built variant of bitcoind/bitcoin-cli for linux build#3914
laanwj merged 1 commit intobitcoin:masterfrom
laanwj:2014_03_statically_built_bitcoind

Conversation

@laanwj
Copy link
Copy Markdown
Member

@laanwj laanwj commented Mar 20, 2014

See #3803 and #3781.

This increases compatibility with older linux distributions.

  2943584 Mar 20 11:43 32/bitcoin-cli
  4376728 Mar 20 11:43 32/bitcoin-cli.static
  7748512 Mar 20 11:43 32/bitcoind
  9140856 Mar 20 11:43 32/bitcoind.static
 12164000 Mar 20 11:43 32/bitcoin-qt
  9740608 Mar 20 11:43 32/test_bitcoin
  8420256 Mar 20 11:43 32/test_bitcoin-qt
  3103728 Mar 20 11:53 64/bitcoin-cli
  4355112 Mar 20 11:53 64/bitcoin-cli.static
  7773680 Mar 20 11:53 64/bitcoind
  8890088 Mar 20 11:53 64/bitcoind.static
 12226032 Mar 20 11:53 64/bitcoin-qt
  9860528 Mar 20 11:53 64/test_bitcoin
  8457776 Mar 20 11:53 64/test_bitcoin-qt

A warning shown during build is:

 warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

Note I have not explicitly disabled hardening/-pie in this build. This makes that the 32-bit version still has a dependency on dynamic libc.so even though it is statically linked:

$ ldd 32/bitcoind.static
    statically linked
$ readelf -a 32/bitcoind.static |grep interpreter
      [Requesting program interpreter: /usr/lib/libc.so.1]

Not sure how much of an issue this is on true 32-bit systems. It needs to be tested. This is not the case for the 64-bit build:

$ ldd 64/bitcoind.static                                                       
    not a dynamic executable
$ readelf -a 64/bitcoind.static |grep interpreter
(nothing)

Edit: v2: -pie now explicitly disabled for 32-bit build as it was unusable otherwise.

Tested on:

  • Debian 7.2 64-bit: OK (tested by me)
  • Ubuntu 10.04.4 LTS 64-bit: OK (tested by me)
  • Ubuntu 10.04.4 LTS 32-bit: OK (tested by me, v2)
  • CentOS 6.5 64bit: OK (tested by alexeft on forums and anton000, v2)
  • CentOS 6.5 32bit: OK (tested by anton000, v2)

@laanwj
Copy link
Copy Markdown
Member Author

laanwj commented Mar 20, 2014

I've uploaded the result here (temporary download URI only!):

https://download.visucore.com/bitcoin/bitcoin-0.9.0-linux-static-2.tar.gz
https://download.visucore.com/bitcoin/bitcoin-0.9.0-linux-static-2.tar.gz.sig

@laanwj laanwj mentioned this pull request Mar 20, 2014
Closed
@laanwj
Copy link
Copy Markdown
Member Author

laanwj commented Mar 20, 2014

OK, whereas the 64-bit ones seem to work fine, the 32-bit static executables produced this way are absolutely broken. I have added a commit to disable -pie for 32-bit static executables, and updated the above downoad.

@theuni
Copy link
Copy Markdown
Member

theuni commented Mar 20, 2014

I'm still very hesitant on this approach. I'd be much more comfortable with picking a minimum required glibc version and wrapping symbols for newer versions.

@laanwj What's the minimum glibc you've got on any of those systems? I'll see if I can work up a POC for the sake of discussion.

@laanwj
Copy link
Copy Markdown
Member Author

laanwj commented Mar 20, 2014

Glibc 2.11 for Ubuntu 12.10, 2.12 for RH.
But there were also problems with libc++ symbols...

@anton000
Copy link
Copy Markdown

anton000 commented Mar 20, 2014

+1 thank you @laanwj for this!
Currently running fine as daemon on CentOS 6.5 64bit :)
Will try to fire up my 32bit centos box to test

edit: works fine on fully updated CentOS 6.5 32bit

@laanwj
Copy link
Copy Markdown
Member Author

laanwj commented Mar 21, 2014

Thanks for testing @anton000

@ghost
Copy link
Copy Markdown

ghost commented Mar 25, 2014

When will this be public? Need it to run 0.9 on debian.

@laanwj
Copy link
Copy Markdown
Member Author

laanwj commented Mar 25, 2014

When it's ready.

In the meantime you can use and help testing the executables here or build your own.

@ghost
Copy link
Copy Markdown

ghost commented Mar 25, 2014

already did - runs perfectly on debian wheezy 64 bit.

@theuni
Copy link
Copy Markdown
Member

theuni commented Mar 25, 2014

@laanwj May as well go ahead and use this, since it fixes real-world problems in the short-term. I'll keep looking into options for future releases... nothing I've come up with so far has been any more appealing.

@BitcoinPullTester
Copy link
Copy Markdown

BitcoinPullTester commented Mar 26, 2014

Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/ddcd1afc5fdd148cd56f257b40a12f70841bd1b3 for binaries and test log.
This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/
Contact BlueMatt on freenode if something looks broken.

@laanwj
Copy link
Copy Markdown
Member Author

laanwj commented Mar 26, 2014

Squashed into one commit, removed special-case for 32/64 bit (-static -pie is invalid on all architectures).

@theuni
Copy link
Copy Markdown
Member

theuni commented Mar 27, 2014

@laanwj Please see https://github.com/theuni/bitcoin/tree/libc-compat . That should fix for 2.13. If the approach works as intended, I can move forward with the other symbols as well.

@theuni
Copy link
Copy Markdown
Member

theuni commented Mar 28, 2014

Ok, hold on that. glibc behavior changed at some point, so some of my assumptions were wrong on the above. Looking again.

@theuni
Copy link
Copy Markdown
Member

theuni commented Mar 28, 2014

Ok, fresh new idea is pushed up to https://github.com/theuni/bitcoin/tree/libc-compat

To use: ./configure --with-glibc-back-compat or --with-glibc-back-compat=X where X is glibc version 2.X.

The idea is to eventually be able to only enable the work-arounds needed for a specific version. For now, there's only one enabled for 2.15.

@laanwj
Copy link
Copy Markdown
Member Author

laanwj commented Mar 28, 2014

@theuni so I can point my gitian build at that tree and generate testing executables (to try on the various platforms)? Just have to make sure I pass --with-glibc-back-compat ?

@theuni
Copy link
Copy Markdown
Member

theuni commented Mar 28, 2014

@laanwj Yes. Though, the only work-around included for now is for glibc 2.15, so it won't help on anything older than that. If the approach works and seems reasonable, I can begin adding more.

@theuni
Copy link
Copy Markdown
Member

theuni commented Mar 29, 2014

@laanwj Hold on that last one as well. I finally hit on a clean way of handling this, and the memcpy for 2.14 is resolved with it, so now the only concern is libstdc++.

If you're ok with waiting a week or so, I'll quit spamming here and just submit a PR that includes gitian descriptors once it's nice and tidy.

@laanwj
Copy link
Copy Markdown
Member Author

laanwj commented Mar 29, 2014

By all means keep 'spamming' to keep us informed as you're working on this, it's nice to know what people are up to.

@laanwj
Copy link
Copy Markdown
Member Author

laanwj commented Apr 8, 2014

@theuni Because of the SSL heartbeat vulnerability we'd like to spin a 0.9.1 as soon as possible. It would be useful to have a portable executable in it, so I'd like to pull this now. This solution has been extensively tested. Your symbol-filtering based solution can be used for next release, however it needs testing first.

@luke-jr
Copy link
Copy Markdown
Member

luke-jr commented Apr 8, 2014

@laanwj Huh? The heartbleed vuln doesn't affect Bitcoin Core afaik?

@laanwj
Copy link
Copy Markdown
Member Author

laanwj commented Apr 8, 2014

@luke-jr The vulnerability does not affect the bitcoin protocol but may affect auxilary usage of TLS/HTTPS in RPC SSL and payment request fetching.

@luke-jr
Copy link
Copy Markdown
Member

luke-jr commented Apr 8, 2014

Fetching too? I thought it was just server-side :(

@ghost
Copy link
Copy Markdown

ghost commented Apr 8, 2014

I think it would be good to release a 0.9.1 with the SSL bug patched.

laanwj added a commit that referenced this pull request Apr 8, 2014
@laanwj
Merge pull request #3914
dac4e2a 
ddcd1af gitian: add statically built variant of bitcoind/bitcoin-cli (Wladimir J. van der Laan)
@laanwj laanwj merged commit ddcd1af into bitcoin:master Apr 8, 2014
@laanwj laanwj deleted the 2014_03_statically_built_bitcoind branch April 9, 2014 14:02
@theuni theuni mentioned this pull request Apr 10, 2014
Merged
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@laanwj @theuni @anton000 @BitcoinPullTester @luke-jr