Updated Windows build doc for WSL/Xenial workaround #11438

sipsorcery · 2017-10-02T20:46:57Z

An update to the Windows build document that provides workarounds for the broken 64 bit mingw32 cross compiler on WSL/Xenial.

This update is an alternative to pull request #11437. While that pull request takes a valid approach by stating building on WSL should be avoided I think it is more useful to give Windows developers a workaround option.

The instructions have been tested on:

Ubuntu 14.04 and 64 bit mingw32 tool chain
Ubuntu 16.04 and 64 bit mingw32 tool chain
Ubuntu 17.04 and 64 bit mingw32 tool chain
Windows Subsystem for Linux (Windows 10 OS Build 15063.608) and 32 bit mingw32 tool chain
Windows Subsystem for Linux (Windows 10 OS Build 15063.608) and 64 bit mingw32 tool chain

Related items:

Serious incompatibility problems w/ newer mingw-64 on Ubuntu Serious incompatibility problems w/ newer mingw-64 on Ubuntu #8653
-fstack-protector-all triggers crashes in mingw-w64 5.3.1 -fstack-protector-all triggers crashes in mingw-w64 5.3.1 #8732
Windows build appears broken on WSL (buntu okay) Windows build appears broken on WSL (buntu okay) #10269
Compilation error for windows target [Docs] Update Windows build instructions for using WSL and Ubuntu 17.04 #11437

meshcollider · 2017-10-03T02:13:58Z

doc/build-windows.md

Use footnotes ([^1]) rather than * because * is used for formatting. (Same in other places).

meshcollider · 2017-10-03T02:14:36Z

doc/build-windows.md

stesp -> steps

meshcollider · 2017-10-03T03:39:24Z

doc/build-windows.md

Colon after [^1] :)

meshcollider · 2017-10-03T03:49:24Z

doc/build-windows.md

meshcollider · 2017-10-03T04:24:21Z

doc/build-windows.md

The trailing whitespace failure is the end of this line I think. Sorry for so many little reviews :) Remember to squash all your commits at the end

meshcollider · 2017-10-03T05:55:41Z

Gentle utACK 5e31c53
Will hopefully have some time to test this soon, and compare with #11437 properly

NicolasDorier · 2017-10-03T05:58:15Z

Concept ACK. thank you, this will save me so much time! Trying this.

molxyz · 2017-10-03T06:04:31Z

There's a misspelled word: sduo dpkg -i binutils_2.28-3ubuntu1_amd64.deb

sipsorcery · 2017-10-03T06:13:07Z

@molxyz have you got the latest version? That typo was fixed.

molxyz · 2017-10-03T07:11:28Z

@sipsorcery Ah yes.. sorry I forgot to refresh the page, got it now. This is great, thank you.

NicolasDorier · 2017-10-03T08:01:47Z

Still another problem on my WSL, probably unrelated though.

configure: error: No working boost sleep implementation found.

I think I give up with WSL, keeping my docker :/

sipsorcery · 2017-10-03T09:33:02Z

@NicolasDorier I did see that error once or twice and I believe it was around the same time as I was getting the error about the std::mutex header missing. I haven't seen it for a while though and I'd estimate I've re-installed and built bitcoin on WSL at least half a dozen times in the last week. My suggestion would be to either completely uninstall WSL using lxrun /uninstall /full and then reinstall or at the very least clean the dependencies, including boost, from your bitcoin source tree with git clean -x -f -d.

laanwj · 2017-10-04T13:57:50Z

doc/build-windows.md

Should mention early that 16.04 produces incorrect code, so that people don't get started with it and come to the conclusion later that it's broken.

There is no bad code, provided the mingw32 x-compiler on wsl/xenial is updated as per the instructions in the doc.

maflcko

Concept ACK. Some feedback and a nit.

maflcko · 2017-10-05T07:32:14Z

doc/build-windows.md

This should probably mention that building on WSL currently is by following the building on Xenial notes. (not the other way round)

maflcko · 2017-10-05T07:32:57Z

doc/build-windows.md

I don't think this is true, given that there are two different sections. One for trusty, one for zesty.

maflcko · 2017-10-05T07:33:32Z

doc/build-windows.md

I think you can write "17.04 or later", since 17.10 seems to work as well with the same steps.

maflcko · 2017-10-05T07:35:26Z

doc/build-windows.md

Might want to remove WSL here, since it will no longer be true when the user runs a different ubuntu version than xenial in their WSL.

Wouldn't it be better to leave it there and update the doc when WSL gets updated? I suspect the majority of devs reading this doc will be looking for WSL instructions.

maflcko · 2017-10-05T07:37:54Z

doc/build-windows.md

µnit: This installation step is the same for all three version. Might as well mention it only once, mention that no further steps are needed on trusty and that other versions require some workarounds: ...

maflcko · 2017-10-05T07:38:53Z

doc/build-windows.md

Maybe ot: For some reason GitHub doesn't insert links to the footnotes for me.

Looks like that footnote is not supported on github. According to a comment on this SO answer that format is used on GitLab. I'll use the alternative suggestion of anchor links.

This is great. I have tested on my WSL Ubuntu Xenial 16.04 several times and it works, but I have a question regarding this part:

sudo update-alternatives --config x86_64-w64-mingw32-g++

There are 2 choices for the alternative x86_64-w64-mingw32-g++ (providing /usr/bin/x86_64-w64-mingw32-g++).

Selection Path Priority Status

0 /usr/bin/x86_64-w64-mingw32-g++-win32 60 auto mode
1 /usr/bin/x86_64-w64-mingw32-g++-posix 30 manual mode
2 /usr/bin/x86_64-w64-mingw32-g++-win32 60 manual mode

The only choice is "1" to get bitcoin compiled successfully, the other choices cause errors in "make". Is there a reason to leave 3 choices? Sorry, I'm still new to this and the choices made me confused at first until I tried them all.
Thank you for doing this, I'm thrilled to be able to compile bitcoin on WSL again! :D

Note that "0" is the default and the options are 1 (posix) and 2 (win32).

maflcko · 2017-10-05T11:19:41Z

doc/build-windows.md

µnit: Can remove the "g++-mingw-w64-x86-64 mingw-w64-x86-64-dev"

maflcko · 2017-10-05T11:19:43Z

doc/build-windows.md

µnit: Can remove this line.

maflcko · 2017-10-05T17:03:24Z

Please squash your commits according to https://github.com/bitcoin/bitcoin/blob/master/CONTRIBUTING.md#squashing-commits

laanwj · 2017-10-11T09:49:33Z

Needs rebase for #11437.

sipsorcery · 2017-10-11T12:05:50Z

#11437 is largely mutually exclusive to this update. It steers devs away from using WSL or Ubuntu 16 due to the non-standard build steps required. This pull request provides those non-standard build steps.

NicolasDorier · 2017-10-13T13:13:47Z

I made a fresh ubuntu WSL 16.04, was unable to install boost, I tried all sort of incantation without success...

libboost-all-dev : Depends: libboost-tools-dev but it is not going to be installed
                    Depends: libboost-atomic-dev but it is not going to be installed
                    Depends: libboost-chrono-dev but it is not going to be installed
                    Depends: libboost-context-dev but it is not going to be installed
                    Depends: libboost-coroutine-dev but it is not going to be installed
                    Depends: libboost-date-time-dev but it is not going to be installed
                    Depends: libboost-exception-dev but it is not going to be installed
                    Depends: libboost-fiber-dev but it is not going to be installed
                    Depends: libboost-graph-dev but it is not going to be installed
                    Depends: libboost-graph-parallel-dev but it is not going to be installed
                    Depends: libboost-iostreams-dev but it is not going to be installed
                    Depends: libboost-locale-dev but it is not going to be installed
                    Depends: libboost-log-dev but it is not going to be installed
                    Depends: libboost-math-dev but it is not going to be installed
                    Depends: libboost-mpi-dev but it is not going to be installed
                    Depends: libboost-mpi-python-dev but it is not going to be installed
                    Depends: libboost-python-dev but it is not going to be installed
                    Depends: libboost-random-dev but it is not going to be installed
                    Depends: libboost-regex-dev but it is not going to be installed
                    Depends: libboost-serialization-dev but it is not going to be installed
                    Depends: libboost-signals-dev but it is not going to be installed
                    Depends: libboost-test-dev but it is not going to be installed
                    Depends: libboost-timer-dev but it is not going to be installed
                    Depends: libboost-type-erasure-dev but it is not going to be installed
                    Depends: libboost-wave-dev but it is not going to be installed

Not sure if related with this PR, but the instructions did not seem enough.

sipsorcery · 2017-10-14T03:26:52Z

@NicolasDorier what type of VM, Ubuntu 16 version and platform (x86 or x64)?

I have run the install steps using the latest source form the bitcoin git repo on VMWare Workstation with Ubuntu 16.04.3 Server x64 without any problems and resulting in working Windows executables.

I'm happy to check other permutations if you let me know what your configuration is.

NicolasDorier · 2017-10-15T07:18:54Z

This is x64.

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.3 LTS"

Problems are on boost... I can't seems to be able to fetch boost for some reason.

maflcko · 2017-10-15T08:17:34Z

Needs rebase on current master to solve merge conflicts.

sipsorcery · 2017-10-15T08:25:57Z

@MarcoFalke merge conflict is from #11437 which is largely mutually exclusive to this update.

It's a question of whether to steers devs away from building on WSL/ Ubuntu 16 using non-standard build steps required or only recommend Ubuntu 14 & 17 as per #11437.

maflcko · 2017-10-15T08:48:02Z

@sipsorcery Agree. Though, we can not merge your pull if there are merge conflicts (even if only minor).

I don't have WSL, so I can't help with testing and recommendations. How you solve the merge conflict is pretty much up to you. It doesn't matter if you use -Xtheirs or -Xours or solve it manually ...

sipsorcery · 2017-10-15T11:59:47Z

@MarcoFalke no worries, I've removed the conflict by pushing my change to the build-windows.md file on top of trunk.

sipa · 2017-10-16T06:58:41Z

The python tests don't need boost?

NicolasDorier · 2017-10-16T08:46:23Z

They don't but I need to compile bitcoind as a linux binary, which need boost.

sipa · 2017-10-16T08:50:04Z

@NicolasDorier Using the depends build will do all that.

sipsorcery · 2017-10-16T09:12:03Z

On a related note the Windows 10 Fall Creators Update gets released tomorrow (17 Oct 2017) and contains a major update to WSL. The main update is that WSL will now be deployed as a Windows Store App and users will need to choose from different Linux distros; currently Ubuntu (Xenial), openSUSE and SUSE Linux Enterprise Server.

Update: I've verified that the Windows bitcoin build on the Fall Creators Update & Ubuntu Windows Store App works successfully. The Ubuntu version hasn't changed from 16.04.3.

NicolasDorier · 2017-10-16T09:36:14Z

my understanding was that

cd depends
make HOST=x86_64-w64-mingw32

will just install the deps for making a windows binary. I was unaware it was also adding the libboost dependencies for linux build. Trying that now.

sipa · 2017-10-16T09:41:26Z

make HOST=x86_64-unknown-linux-gnu will do a build of dependencies for Linux build on x86_64.

NicolasDorier · 2017-10-16T09:44:40Z

Trying. Maybe this is something to add as exception of Xenial in the doc, since installing package with apt-get is not available?

NicolasDorier · 2017-10-16T13:16:38Z

Ok now

WARNING: 'automake-1.14' is missing on your system.

So make HOST=x86_64-unknown-linux-gnu does not run, I ran apt-get of autotools.

autotools-dev is already the newest version (20150820.1).

I think I will provide an easy to build image on docker hub based on trusty and document it. Would, requires Hyper-V for windows (windows 10 Enterprise edition) though.

sipsorcery · 2017-10-17T02:53:21Z

@NicolasDorier the automake error seems to be generated in the protobuf dependency build. The same error was mentioned in #10269. I'm guessing it's triggered when the apt packages line up in a certain way.

The suggested workaround, which also worked for me in the past, is:

make HOST=x86_64-unknown-linux-gnu AUTOMAKE=:

Edit: Also #11282.

NicolasDorier · 2017-10-18T07:34:09Z

autoreconf: running: : --add-missing --copy
Can't exec ":": No such file or directory at /usr/share/autoconf/Autom4te/FileUtils.pm line 326, <GEN2> line 5.
autoreconf: failed to run :: No such file or directory
funcs.mk:242: recipe for target '/mnt/c/Users/NicolasDorier/Documents/Sources/bitcoin-docker/bitcoin/depends/work/build/x86_64-unknown-linux-gnu/libevent/2.1.8-stable-b014fbf5633/.stamp_preprocessed' failed
make: *** [/mnt/c/Users/NicolasDorier/Documents/Sources/bitcoin-docker/bitcoin/depends/work/build/x86_64-unknown-linux-gnu/libevent/2.1.8-stable-b014fbf5633/.stamp_preprocessed] Error 1

Ha might be because too long path... I don't know. :(

I think this PR is definitively a way to the right direction, I hope other people can try it, my environment appear to be seriously fucked up.

rebroad · 2017-11-12T16:54:32Z

tested ACK

laanwj · 2017-11-13T11:48:45Z

@rebroad Thanks for testing

7383d77 Updated instructions for Windows 10 Fall Creators Update. (Aaron Clauson) e0fc4a7 Updated Windows build doc for WSL/Xenial workarounds. (Aaron Clauson) Pull request description: An update to the Windows build document that provides workarounds for the broken 64 bit mingw32 cross compiler on WSL/Xenial. This update is an alternative to pull request #11437. While that pull request takes a valid approach by stating building on WSL should be avoided I think it is more useful to give Windows developers a workaround option. The instructions have been tested on: - Ubuntu 14.04 and 64 bit mingw32 tool chain - Ubuntu 16.04 and 64 bit mingw32 tool chain - Ubuntu 17.04 and 64 bit mingw32 tool chain - Windows Subsystem for Linux (Windows 10 OS Build 15063.608) and 32 bit mingw32 tool chain - Windows Subsystem for Linux (Windows 10 OS Build 15063.608) and 64 bit mingw32 tool chain Related items: - Serious incompatibility problems w/ newer mingw-64 on Ubuntu #8653 - `-fstack-protector-all` triggers crashes in mingw-w64 5.3.1 #8732 - Windows build appears broken on WSL (buntu okay) #10269 - Compilation error for windows target #11437 Tree-SHA512: 7c937e37ed7120ae5dcf61aba50e5228a7ed6f729647c724b8f48e7cbbd81366c1a83a818618766a8fe0418425e05ba2eba2b14f2616621c58606585444f45fc

jky7 · 2017-11-13T18:03:51Z

@NicolasDorier I had the same issue, what ended up working was the following:
make HOST=x86_64-w64-mingw32 AUTOCONF=: AUTOHEADER=: AUTOMAKE=: ACLOCAL=:

also I have found that downloading the repo natively from inside WSL was less likely to break things in the build than downloading it in Windows and then working on it from WSL

meshcollider · 2017-11-15T09:43:12Z

Post-merge tested ACK on clean install of 64-bit Win10 Home 1709 build 16299.64

NicolasDorier · 2017-11-21T04:56:34Z

@JosephKY ha yes. There are some git config magic to do so git on Windows does not convert LF to CRLF on checkout. And that it convert CRLF to LF on check-in. I did that long time ago, now it works fine.

It was only core.autocrlf=input if I remember...

originally found in bitcoin/bitcoin#11438 and adopted for Merit Core

fanquake added Docs Windows labels Oct 2, 2017

meshcollider reviewed Oct 3, 2017

View reviewed changes

doc/build-windows.md Outdated

Copy link

Contributor

meshcollider Oct 3, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Colon after [^1] :)

meshcollider reviewed Oct 3, 2017

View reviewed changes

doc/build-windows.md Outdated

Copy link

Contributor

meshcollider Oct 3, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sudo

meshcollider reviewed Oct 3, 2017

View reviewed changes

sipsorcery force-pushed the wslbuilddoc branch from 831ec47 to 5e31c53 Compare October 3, 2017 04:52

laanwj reviewed Oct 4, 2017

View reviewed changes

maflcko reviewed Oct 5, 2017

View reviewed changes

sipsorcery force-pushed the wslbuilddoc branch from 15ed58f to 775783a Compare October 5, 2017 20:24

Updated Windows build doc for WSL/Xenial workarounds.

e0fc4a7

sipsorcery force-pushed the wslbuilddoc branch from afadea2 to e0fc4a7 Compare October 15, 2017 11:53

Updated instructions for Windows 10 Fall Creators Update.

7383d77

NicolasDorier mentioned this pull request Oct 23, 2017

"Windows Subsystem For Linux" for master not work #11549

Closed

rebroad mentioned this pull request Nov 11, 2017

Windows build instructions do not create .exe files #11633

Closed

sipsorcery mentioned this pull request Nov 11, 2017

Contradictory Windows build instructions #11659

Closed

laanwj merged commit 7383d77 into bitcoin:master Nov 13, 2017

laanwj mentioned this pull request Nov 13, 2017

Docs: Update WSL installation notes for Fall Creators update #11533

Merged

This was referenced Nov 14, 2017

Trouble Installing Bitcoin Core on Windows 10 64-Bit #11584

Closed

Windows build appears broken on WSL (buntu okay) #10269

Closed

Make with MinGW fails with recipe for target 'all-recursive' failed #10970

Closed

sipsorcery deleted the wslbuilddoc branch November 21, 2017 06:56

eugene-sy added a commit to meritlabs/merit that referenced this pull request Jan 6, 2018

fixes in docs for Ubuntu Xenial and WSL

ce593ae

originally found in bitcoin/bitcoin#11438 and adopted for Merit Core

eugene-sy mentioned this pull request Jan 6, 2018

Ubuntu Xenial and WSL documentation fixes. meritlabs/merit#175

Merged

instagibbs mentioned this pull request Apr 5, 2018

[Docs] Fix to strip out problematic Windows %PATH% ElementsProject/elements#331

Closed

bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021

Updated Windows build doc for WSL/Xenial workaround #11438

Updated Windows build doc for WSL/Xenial workaround #11438

Uh oh!

Conversation

sipsorcery commented Oct 2, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

meshcollider commented Oct 3, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NicolasDorier commented Oct 3, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

molxyz commented Oct 3, 2017

Uh oh!

sipsorcery commented Oct 3, 2017

Uh oh!

molxyz commented Oct 3, 2017

Uh oh!

NicolasDorier commented Oct 3, 2017

Uh oh!

sipsorcery commented Oct 3, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sipsorcery Oct 4, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maflcko left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Selection Path Priority Status

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maflcko commented Oct 5, 2017

Uh oh!

laanwj commented Oct 11, 2017

Uh oh!

sipsorcery commented Oct 11, 2017

Uh oh!

NicolasDorier commented Oct 13, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sipsorcery commented Oct 14, 2017

Uh oh!

NicolasDorier commented Oct 15, 2017

Uh oh!

sipsorcery commented Oct 2, 2017 •

edited

Loading

meshcollider commented Oct 3, 2017 •

edited

Loading

NicolasDorier commented Oct 3, 2017 •

edited

Loading

sipsorcery Oct 4, 2017 •

edited

Loading

NicolasDorier commented Oct 13, 2017 •

edited

Loading

sipsorcery commented Oct 16, 2017 •

edited

Loading

NicolasDorier commented Oct 16, 2017 •

edited

Loading

NicolasDorier commented Oct 16, 2017 •

edited

Loading

sipsorcery commented Oct 17, 2017 •

edited

Loading

jky7 commented Nov 13, 2017 •

edited

Loading