importprunedfunds + removeprunedfunds + listtransactions can cause OOM #13078
Description
I hit this while refactoring the wallet functional tests to remove the 'account' API.
To reproduce this, take the commit at 6f3b865 and run wallet_importprunedfunds.py. I've tried to pare that test down to a minimal failure scenario, but I haven't been able to reliably reproduce this with a shorter test yet.
What I know so far is that some combination of importprunedfunds, removeprunedfunds and listtransactions leads to the node OOM'ing. The failure occurs at line 107:
→ /wallet_importprunedfunds.py
2018-04-25T20:32:37.762000Z TestFramework (INFO): Initializing test directory /tmp/user/1000/testjke6yfkx
2018-04-25T20:32:38.143000Z TestFramework (INFO): Mining blocks...
terminate called without an active exception
2018-04-25T20:32:38.465000Z TestFramework (ERROR): Unexpected exception caught during testing
Traceback (most recent call last):
File "/home/ubuntu/bitcoin/test/functional/test_framework/authproxy.py", line 107, in _request
return self._get_response()
File "/home/ubuntu/bitcoin/test/functional/test_framework/authproxy.py", line 153, in _get_response
http_response = self.__conn.getresponse()
File "/usr/lib/python3.5/http/client.py", line 1197, in getresponse
response.begin()
File "/usr/lib/python3.5/http/client.py", line 297, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.5/http/client.py", line 266, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ubuntu/bitcoin/test/functional/test_framework/test_framework.py", line 147, in main
self.run_test()
File "./wallet_importprunedfunds.py", line 107, in run_test
assert not [tx for tx in self.nodes[1].listtransactions(include_watchonly=True) if tx['txid'] == txnid2]
File "/home/ubuntu/bitcoin/test/functional/test_framework/coverage.py", line 47, in __call__
return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
File "/home/ubuntu/bitcoin/test/functional/test_framework/authproxy.py", line 136, in __call__
response = self._request('POST', self.__url.path, postdata.encode('utf-8'))
File "/home/ubuntu/bitcoin/test/functional/test_framework/authproxy.py", line 111, in _request
self.__conn.request(method, path, postdata, headers)
File "/usr/lib/python3.5/http/client.py", line 1106, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python3.5/http/client.py", line 1151, in _send_request
self.endheaders(body)
File "/usr/lib/python3.5/http/client.py", line 1102, in endheaders
self._send_output(message_body)
File "/usr/lib/python3.5/http/client.py", line 934, in _send_output
self.send(msg)
File "/usr/lib/python3.5/http/client.py", line 877, in send
self.connect()
File "/usr/lib/python3.5/http/client.py", line 849, in connect
(self.host,self.port), self.timeout, self.source_address)
File "/usr/lib/python3.5/socket.py", line 711, in create_connection
raise err
File "/usr/lib/python3.5/socket.py", line 702, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
2018-04-25T20:32:38.473000Z TestFramework (INFO): Stopping nodes
2018-04-25T20:32:38.475000Z TestFramework.node1 (ERROR): Unable to stop node.
Traceback (most recent call last):
File "/home/ubuntu/bitcoin/test/functional/test_framework/test_node.py", line 187, in stop_node
self.stop()
File "/home/ubuntu/bitcoin/test/functional/test_framework/coverage.py", line 47, in __call__
return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
File "/home/ubuntu/bitcoin/test/functional/test_framework/authproxy.py", line 136, in __call__
response = self._request('POST', self.__url.path, postdata.encode('utf-8'))
File "/home/ubuntu/bitcoin/test/functional/test_framework/authproxy.py", line 106, in _request
self.__conn.request(method, path, postdata, headers)
File "/usr/lib/python3.5/http/client.py", line 1106, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python3.5/http/client.py", line 1141, in _send_request
self.putrequest(method, url, **skips)
File "/usr/lib/python3.5/http/client.py", line 974, in putrequest
raise CannotSendRequest(self.__state)
http.client.CannotSendRequest: Request-sent
Traceback (most recent call last):
File "./wallet_importprunedfunds.py", line 113, in <module>
ImportPrunedFundsTest().main()
File "/home/ubuntu/bitcoin/test/functional/test_framework/test_framework.py", line 170, in main
self.stop_nodes()
File "/home/ubuntu/bitcoin/test/functional/test_framework/test_framework.py", line 300, in stop_nodes
node.wait_until_stopped()
File "/home/ubuntu/bitcoin/test/functional/test_framework/test_node.py", line 214, in wait_until_stopped
wait_until(self.is_node_stopped, timeout=timeout)
File "/home/ubuntu/bitcoin/test/functional/test_framework/util.py", line 216, in wait_until
if predicate():
File "/home/ubuntu/bitcoin/test/functional/test_framework/test_node.py", line 205, in is_node_stopped
"Node returned non-zero exit code (%d) when stopping" % return_code)
AssertionError: [node 1] Node returned non-zero exit code (-6) when stopping
[node 1] Cleaning up leftover process
Tail of node1's debug.log:
2018-04-25T20:32:38.338225Z ThreadRPCServer method=removeprunedfunds user=__cookie__
2018-04-25T20:32:38.339174Z Received a POST request for / from 127.0.0.1:34026
2018-04-25T20:32:38.339214Z ThreadRPCServer method=listtransactions user=__cookie__
2018-04-25T20:32:38.339348Z Error: Out of memory. Terminating.