segfault in thread pool (again) #5348
Description
Looks like #4675
This happened on a machine in a new 3 node rethinkdb cluster, created today. I was loading data with rethinkdb-import, into a table with 3 shards x 2 replicas each. (This is the only table in the cluster other than system tables)
The rethinkdb process continued running and I had to kill -9 it.
2016-02-02T22:55:32.784320754 0.193153s notice: Recursively removing directory /var/lib/rethinkdb/tmp
2016-02-02T22:55:32.835829388 0.244673s notice: Initializing directory /var/lib/rethinkdb
2016-02-02T22:55:32.836004923 0.244836s notice: Running rethinkdb 2.2.4~0trusty (GCC 4.8.2)...
2016-02-02T22:55:32.885517341 0.294352s notice: Running on Linux 3.13.0-54-generic x86_64
2016-02-02T22:55:32.885585066 0.294416s notice: Loading data from directory /var/lib/rethinkdb
2016-02-02T22:55:33.709268076 1.118101s info: Automatically using cache size of 22820 MB
2016-02-02T22:55:33.719564033 1.128395s notice: Listening for intracluster connections on port 29015
2016-02-02T22:55:33.719572032 1.128403s warn: Attempted to join self, peer ignored
2016-02-02T22:55:33.719818891 1.128650s info: Attempting connection to 2 peers...
2016-02-02T22:55:33.725135147 1.133966s notice: Connected to server "aidata407_bu_us_archive_org_7fo" 687eceb2-ca51-44bc-a96d-79d564d7e079
2016-02-02T22:55:33.725203341 1.134034s notice: Connected to server "aidata407_us_archive_org_m9r" 8c913c4c-257a-4ad5-ad3a-66fd0716c891
2016-02-02T22:55:33.726888479 1.135719s notice: Listening for client driver connections on port 28015
2016-02-02T22:55:33.727045967 1.135877s notice: Listening for administrative HTTP connections on port 6680
2016-02-02T22:55:33.731470291 1.140302s notice: Listening on addresses: 127.0.0.1, 207.241.239.57, ::1, fe80::225:90ff:fe35:5c0a%4, fe80::225:90ff:fe35:5c0a%5
2016-02-02T22:55:33.731479857 1.140311s notice: Server ready, "aidata405_bu_us_archive_org_fyt" 0314742c-5387-41b2-bc32-20f2b4f52dd2
2016-02-02T23:09:32.841532394 840.250364s info: Table 8cb43727-8e09-428e-9997-09520d0c8087: Added replica on this server.
2016-02-03T01:02:55.776892691 7643.185725s error: Error in src/arch/runtime/thread_pool.cc at line 359:
2016-02-03T01:02:55.776938686 7643.185770s error: Segmentation fault from reading the address 0x18.
2016-02-03T01:02:55.776953667 7643.185785s error: Backtrace:
2016-02-03T01:02:55.784115036 7643.192947s error: Error in src/arch/runtime/thread_pool.cc at line 359:
2016-02-03T01:02:55.784172031 7643.193003s error: Segmentation fault from reading the address 0x18.
2016-02-03T01:02:55.784200069 7643.193031s error: Backtrace:
2016-02-03T01:02:55.869378707 7643.278210s error: Error in src/arch/runtime/thread_pool.cc at line 359:
2016-02-03T01:02:55.869424618 7643.278256s error: Segmentation fault from reading the address 0x18.
2016-02-03T01:02:55.869456081 7643.278287s error: Backtrace:
2016-02-03T01:02:55.929537425 7643.338369s error: Error in src/arch/runtime/thread_pool.cc at line 359:
2016-02-03T01:02:55.929573962 7643.338405s error: Segmentation fault from reading the address 0x18.
2016-02-03T01:02:55.929591193 7643.338422s error: Backtrace:
2016-02-03T01:02:55.971439042 7643.380271s error: Error in src/arch/runtime/thread_pool.cc at line 359:
2016-02-03T01:02:55.971477759 7643.380309s error: Segmentation fault from reading the address 0x18.
2016-02-03T01:02:55.971495115 7643.380326s error: Backtrace: