Fixes for porting to FreeBSD #688

hungte · 2013-04-19T15:08:18Z

A few commits to allow compiling on FreeBSD, verified on FreeBSD 9 with clang.

AtnNn · 2013-04-19T17:54:18Z

This is awesome! Does it build and run on FreeBSD? I'm going to try it out today.

mglukhovsky · 2013-04-20T01:23:03Z

@hungte, could you please fill out our contributor agreement form (www.rethinkdb.com/community/cla/)-- you only have to do it once. Thanks for the pull request!

mglukhovsky · 2013-04-20T02:37:08Z

Thanks, @hungte. I'll leave it to @AtnNn to finish reviewing the changes before we merge it in. We really appreciate the effort you've put forward, can't wait to include FreeBSD on the list of supported platforms!

hungte · 2013-04-20T02:42:15Z

atnnn> Yes it does. See my comment in commit 2cd2c77 - verified with 1.4.3 on FreeBSD 9, with pre-compiled web.
You need, however, to configure with CXX=clang and modify some settings in custom.mk as 2cd2c77 listed -
ALLOW_WARNINGS := 1
LIBRARY_PATHS += -lrt
CXX := ccache /usr/local/bin/clang -I/usr/local/include -D__STDC_LIMIT_MACROS

mglukhovsky> I'm a little concerned about the Travis build failure above - not sure if I've broken anything for Linux, or if it's temporary failure not related to my patches. Maybe I should get a Linux box and have a try...

mglukhovsky · 2013-04-20T02:46:13Z

@hungte, noted, @AtnNn will be able to help parse those unit test failures. You could try a Linux box, or building on a VM if you'd like to dig into yourself and rerun those unit tests.

hungte · 2013-04-20T04:07:16Z

The Travis failure seems not related to this patch since other people are seeing same failure.

Updated some build env changes so that LIBRARY_PATHS and -I/usr/local/include -D__STDC_LIMIT_MACROS are no longer required.

Step to test updated in commit 1539e00 description. Note there is no freebsd "ports" yet, so you still need to manually install all dependency stuff, or using precompiled web assets (which is a lot easier)

AtnNn · 2013-04-20T10:05:54Z

I got it to work. Thanks for your efforts :D

info: Running rethinkdb 1.4.3-940-gf8fb58-dirty (CLANG 3.1 ((branches/release_31 156863)))...                                                                                           
info: Running on FreeBSD 9.1-RELEASE amd64

It's nice to see all your commits, you seem to know what you are doing very well.

Here are the dependencies I had to install on FreeBSD 9.1:

pkg_add -r wget gcc48 protobuf protobuf-c v8 openjdk7 boost-all gmake bash clang
pkg_add -r node npm py-yaml
npm install -g less coffee-script handlebars

I tried with gcc48, but there seemed to be a lot of #include's missing, so I stuck with clang.

I had a few troubles getting the test suite to run, I put a few changes in 6fe0a1a.

I only saw four unit tests fail:

ClusteringBranch.Backfill:

src/unittest/clustering_branch.cc:246: Failure
Value of: store2.store.values[it->first]
  Actual: "7"
Expected: it->second
Which is: "12"

RPCDirectoryTest and RPCMailboxTest:

error: Error in src/arch/io/io_utils.cc at line 21:
error: Guarantee failed: [res == 0]  (errno 54 - Connection reset by peer) error returned by close(2)
error: Backtrace:
1: 0x18d6e53 <format_backtrace(bool)+51> at /usr/home/teapot/rethinkdb/build/debug_clang_notcmalloc/rethinkdb-unittest
2: 0x2046821 <report_fatal_error(char const*, int, char const*, ...)+625> at /usr/home/teapot/rethinkdb/build/debug_clang_notcmalloc/rethinkdb-unittest
3: 0x18a8c1a <scoped_fd_t::reset(int)+218> at /usr/home/teapot/rethinkdb/build/debug_clang_notcmalloc/rethinkdb-unittest
4: 0x18a6986 <scoped_fd_t::~scoped_fd_t()+22> at /usr/home/teapot/rethinkdb/build/debug_clang_notcmalloc/rethinkdb-unittest5: 0x18914e5 <scoped_fd_t::~scoped_fd_t()+21> at /usr/home/teapot/rethinkdb/build/debug_clang_notcmalloc/rethinkdb-unittest

And the SnapshotsTest took too long, so I killed it.

hungte · 2013-04-20T13:28:40Z

I've changed "Fix stack_t.ss_sp type difference for FreeBSD." to a better implementation by typeof.
Please re-check 8e5ca2a again.

AtnNn · 2013-04-22T23:28:39Z

@srh can you take a look at this pull request?

@hungte I would love to merge this, but I would like to get the tests to pass on FreeBSD first. I'll try to help but I'll be a little busy working on other things this week.

hungte · 2013-04-23T00:51:36Z

@AtnNn Of course - we should pass unit tests to make sure nothing is broken -- although maybe some failures are already in current master branch...
P.S: How to start the unit tests on FreeBSD? I tried make test-dep(s) but it simply failed.

AtnNn · 2013-04-23T00:58:40Z

To run the tests in debug mode, make unit DEBUG=1 or make DEBUG=1; ./build/debug*/rethinkdb-unittest.

I had to patch gtest to get the tests to build: 6fe0a1a

AtnNn · 2013-04-23T01:03:10Z

The failures in our next branch on travis are due to the python driver tests. All the unit tests currently pass.

coffeemug · 2013-04-23T02:26:53Z

@hungte -- wow, this rocks! Thank you for the contribution!

@AtnNn, @al3xandru -- if this makes it into 1.5, we should * definitely* add docs for it, and point it out in the changelog and announcement. I'm slating it for 1.5 for now, feel free to take out if it doesn't make it (but pretty please, could it make it? :-D)

coffeemug · 2013-04-30T02:06:50Z

@AtnNn -- our tests are now fixed (at least the errors that showed up here). Could we retest and merge?

AtnNn · 2013-04-30T02:15:19Z

@coffeemug the tests need to pass on both Travis and FreeBSD before we can safely merge this.

On Wednesday our sprint will be over and I'll take a closer look at why they are failing.

mglukhovsky · 2013-05-06T19:54:00Z

Hey @hungte, just wanted to let you know that we're super-excited about getting FreeBSD support in, but we're also in the midst of shipping 1.5. We're working to figure out what needs to be done to get tests pass. Thanks for your patience!

kudos · 2013-05-11T14:58:03Z

Hey @hungte, thanks for the great work here.

I'm not sure whether this issue is FreeBSD specific or not but it fails for me due to a missing directory (drivers/javascript/build). Creating it manually first solves the issue and I have it compiling successfully on 9.1.

Edit: Actually, just building again for a local Vagrant box and remembered some additional steps I needed, I had to symlink /usr/local/bin/bash to /bin/bash and use --lib-path /usr/local/lib

This is fine unless we do clone(2) with CLONE_THREAD flag.

hungte · 2013-05-15T08:54:10Z

Hi @kudos,

Added a commit to fix drivers/javascript/build ( c9606b3 ).

For /bin/bash, applying atnnn's patch may help (or after you rebase).

Do you remember where you failed without --lib-path /usr/local/lib? I added /usr/local/lib in one mk file but maybe there's somewhere else (ex, some drivers or modules that I didn't build) to be modified...

…nd MacOSX.

hungte · 2013-05-15T11:40:16Z

After rebasing to latest "next" branch, I keep seeing "no std::move' error that is probably caused by c++11 related stuff, or libc++. It's fixed on MacOSX, but I haven't found a good solution for FreeBSD yet.

So I revised the commits to work with gcc48 and made it built successfully again :)

kudos · 2013-05-15T11:59:20Z

The error I get on ./configure without using --lib-path looks like:

* Error: libprotobuf not found
/usr/bin/ld: cannot find -lprotobuf

I'll try building from your latest commit later today.

hungte · 2013-05-15T23:23:44Z

@coffeemug Yes it is (not searching /usr/local/lib by default).
I've already included that part in 10c54b7 .

jdoliner · 2013-05-16T17:56:37Z

What are we planning to do once we merge this in to make sure we don't break FreeBSD compilation? Can we get an automated test going for it?

AtnNn · 2013-05-16T17:59:47Z

Can get an automated test going for it?

Yes, good idea

Protoc-js does not always create folder for given output path. We need to create it explicitly in build procedure.

When using clang3.1/-std=c++0x, the casting of NULL will cause error: reinterpret_cast from 'nullptr_t' to 'memcached_value_t *' is not allowed. For c++11, NULL (nullptr) is defined as nullptr_t and should not be re-casted to other pointer types.

install(1) on FreeBSD does not allow reading file content from /dev/stdin, so we must create a real file in build directory before calling install.

Required by boost library.

The C++ ABI v1 on FreeBSD 9.x clang is different with GCC 4.x ABI v1.

Requires FreeBSD 9.1 (clang 3.1+) and the libc++ from /usr/src/lib/libc++.

For systems using ldconfig with different syntax (ex, FreeBSD or MacOSX), we need to fallback LD path to defaults.

Tested on FreeBSD 9.1 with following steps: # For dependency pkg_add -r wget protobuf protobuf-c v8 openjdk7 boost-all gmake pkg_add -r bash node npm py-yaml gcc48 npm install -g less coffee-script handlebars # To build with GCC48: CXX=gcc48 ./configure --ccache # Edit config.mk to add "ALLOW_WARNINGS := 1" # To build with system clang 3.1: CXX=clang ./configure --ccache # You have to install system libc++ by: # cd /usr/src/lib/libc++; sudo make install # Then edit config.mk to add "ALLOW_WARNINGS := 1" # To build gmake

hungte · 2013-05-17T09:47:11Z

Added some fixes so clang3.1/c++11 (need the libc++ in FreeBSD 9.1 source tree) won't fail to build.

AtnNn · 2013-05-17T16:18:16Z

@hungte I have a freebsd VM open, I'll try to help you with this branch today.

AtnNn · 2013-06-18T08:55:18Z

I rebased this onto next and added a few patches in the atnnn/freebsd branch: https://github.com/rethinkdb/rethinkdb/compare/atnnn;freebsd

It builds with

./configure CXX=g++47 --static none --without-tcmalloc --ccache CXXFLAGS='-std=c++11'
gmake

But fails to run

info: Loading data from directory /usr/home/user/rethinkdb/rethinkdb_data
Version: rethinkdb 1.6.0-50-gb60da6-dirty (GCC 4.7.3)
Version: rethinkdb 1.6.0-50-gb60da6-dirty (GCC 4.7.3)
Version: rethinkdb 1.6.0-50-gb60da6-dirty (GCC 4.7.3)
Version: rethinkdb 1.6.0-50-gb60da6-dirty (GCC 4.7.3)
Version: rethinkdb 1.6.0-50-gb60da6-dirty (GCC 4.7.3)
error: Error in src/arch/runtime/thread_pool.cc at line 325:
error: Error in src/arch/runtime/thread_pool.cc at line 325:
error: Segmentation fault from reading the address 0x5d.
Version: rethinkdb 1.6.0-50-gb60da6-dirty (GCC 4.7.3)
Version: rethinkdb 1.6.0-50-gb60da6-dirty (GCC 4.7.3)
error: Error in src/arch/runtime/thread_pool.cc at line 325:
error: Segmentation fault from reading the address 0x5d.
error: Backtrace:
error: Segmentation fault from reading the address 0x5d.
error: Backtrace:
error: Backtrace:
Segmentation fault: 11 (core dumped)

AtnNn · 2013-06-18T10:44:24Z

With DEBUG=1, the server runs and the web ui works. Simple operations like insert fail. Here are all the failing unit tests: https://gist.github.com/atnnn/dcbccf0bc39349d1820f

AtnNn · 2013-06-25T00:33:29Z

Someone who is more familiar with the code failing in these tests should take a look at this.

danielmewes · 2014-01-02T21:44:16Z

Unless some part of the networking isn't working properly, from what I can see these failure might just have to do with timing issues (heartbeat timeouts) in the tests, and the unit tests not being prepared for those.
@AtnNn Do you still have a FreeBSD VM running somewhere? I would find it interesting to re-test this with the current code base in next.

danielmewes · 2014-01-09T20:43:52Z

I merged next into @AtnNn's freebsd branch. If I compile in release mode, there now is a problem with hostname lookup. This is probably because a lot of that code has changed, and needs to be adapted to FreeBSD.
If I compile in debug mode, a newly created table never becomes available. I think this might have to do with the changed message hub, but that's just a hypothesis.

AtnNn · 2014-06-17T21:10:32Z

None of us use FreeBSD and we don't have the resources to keep this branch up-to-date.

@hungte hopefully we can resurrect it some day.

ghost assigned AtnNn Apr 20, 2013

hungte added 6 commits May 15, 2013 16:00

Use correct make program name (for systems calling make as "gmake").

dd1beff

Fix missing sockaddr_storage definition on FreeBSD.

094ef1d

Use correct alloca header file for FreeBSD.

c0f442a

Use getpid() for gettid() on FreeBSD.

461daa5

This is fine unless we do clone(2) with CLONE_THREAD flag.

Fix nftw failure (E_INVAL) on FreeBSD.

ecb3508

Fix dprintf compile error on FreeBSD.

bfe148d

hungte added 3 commits May 15, 2013 19:36

Select kqueue for timer on FreeBSD.

540c06b

Add FreeBSD to conditional compilation that applies to both FreeBSD a…

5244dea

…nd MacOSX.

Add missing va_list definition for gcc48/FreeBSD.

8dcfd39

hungte added 14 commits May 17, 2013 15:17

Fix stack_t.ss_sp type difference for FreeBSD.

7ac35c0

Fix file data syncing for FreeBSD.

31f5c80

Fix build error in javascript driver.

8573a54

Protoc-js does not always create folder for given output path. We need to create it explicitly in build procedure.

Fix GCC48 linking error on FreeBSD.

a2a5cab

Fix NULL casting error for clang.

96874eb

When using clang3.1/-std=c++0x, the casting of NULL will cause error: reinterpret_cast from 'nullptr_t' to 'memcached_value_t *' is not allowed. For c++11, NULL (nullptr) is defined as nullptr_t and should not be re-casted to other pointer types.

Add missing header for time_t in utils.hpp.

a35a94e

Fix "gmake install" on FreeBSD.

f8e93c9

install(1) on FreeBSD does not allow reading file content from /dev/stdin, so we must create a real file in build directory before calling install.

Support systems without python in /usr/bin (Ex, FreeBSD).

be171a4

Add -pthread for FreeBSD build procedure.

88997eb

Required by boost library.

Add FreeBSD build environment settings.

0e3380f

Support FreeBSD 9.x clang / libc++ ABI v1.

67f0a25

The C++ ABI v1 on FreeBSD 9.x clang is different with GCC 4.x ABI v1.

Support clang/c++11 on FreeBSD.

b2137c2

Requires FreeBSD 9.1 (clang 3.1+) and the libc++ from /usr/src/lib/libc++.

Fix configure default LD search path.

21e7e1a

For systems using ldconfig with different syntax (ex, FreeBSD or MacOSX), we need to fallback LD path to defaults.

AtnNn closed this Jun 17, 2014

AtnNn modified the milestones: backlog, wontfix Jul 30, 2014

danielmewes mentioned this pull request Apr 27, 2015

BSD targets for build system #4130

Closed

srh mentioned this pull request Aug 28, 2022

FreeBSD build fixes #7088

Merged

1 task

Fixes for porting to FreeBSD #688

Fixes for porting to FreeBSD #688

Uh oh!

Conversation

hungte commented Apr 19, 2013

Uh oh!

AtnNn commented Apr 19, 2013

Uh oh!

mglukhovsky commented Apr 20, 2013

Uh oh!

mglukhovsky commented Apr 20, 2013

Uh oh!

hungte commented Apr 20, 2013

Uh oh!

mglukhovsky commented Apr 20, 2013

Uh oh!

hungte commented Apr 20, 2013

Uh oh!

AtnNn commented Apr 20, 2013

Uh oh!

hungte commented Apr 20, 2013

Uh oh!

AtnNn commented Apr 22, 2013

Uh oh!

hungte commented Apr 23, 2013

Uh oh!

AtnNn commented Apr 23, 2013

Uh oh!

AtnNn commented Apr 23, 2013

Uh oh!

coffeemug commented Apr 23, 2013

Uh oh!

coffeemug commented Apr 30, 2013

Uh oh!

AtnNn commented Apr 30, 2013

Uh oh!

mglukhovsky commented May 6, 2013

Uh oh!

kudos commented May 11, 2013

Uh oh!

hungte commented May 15, 2013

Uh oh!

hungte commented May 15, 2013

Uh oh!

kudos commented May 15, 2013

Uh oh!

hungte commented May 15, 2013

Uh oh!

jdoliner commented May 16, 2013

Uh oh!

AtnNn commented May 16, 2013

Uh oh!

hungte commented May 17, 2013

Uh oh!

AtnNn commented May 17, 2013

Uh oh!

AtnNn commented Jun 18, 2013

Uh oh!

AtnNn commented Jun 18, 2013

Uh oh!

AtnNn commented Jun 25, 2013

Uh oh!

danielmewes commented Jan 2, 2014

Uh oh!

danielmewes commented Jan 9, 2014

Uh oh!

AtnNn commented Jun 17, 2014

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants