Skip to content

Conversation

@laanwj
Copy link
Member

@laanwj laanwj commented Sep 18, 2015

Fix a few rough edges discovered in use of the new libevent http server:;

init: Ignore SIGPIPE

Ignore SIGPIPE on all non-win32 OSes, otherwise an unexpectedly disconnecting
RPC client will terminate the application. This problem was introduced
with the libhttp-based RPC server.

Fixes #6660.

http: Disable libevent debug logging, if not explicitly enabled

Add a option "-debug=libevent" to enable libevent debugging for troubleshooting.
Libevent logging is redirected to our own log.

Addresses stdout spew in #6655.

rpc: Split option -rpctimeout into -rpcservertimeout and -rpcclienttimeout

The two timeouts for the server and client, are essentially different:

  • In the case of the server it should be a lower value to avoid clients
    clogging up connection slots
  • In the case of the client it should be a high value to accomedate slow
    responses from the server, for example for slow queries or when the
    lock is contended

Split the options into -rpcservertimeout and -rpcclienttimeout with
respective defaults of 30 and 900.

Should fix, or make it possible to fix timeout issues in #6655.

Ignore SIGPIPE on all non-win32 OSes, otherwise an unexpectedly disconnecting
RPC client will terminate the application. This problem was introduced
with the libhttp-based RPC server.

Fixes bitcoin#6660.
@jonasschnelli
Copy link
Contributor

utACK (plans to test this soon).
Not sure about the insides of getblocktemplate, but hows does the rpctimeout affect the long poll?

@laanwj
Copy link
Member Author

laanwj commented Sep 18, 2015

Not sure about the insides of getblocktemplate, but hows does the rpctimeout affect the long poll?

On the server side it doesn't affect that - it doesn't spend time waiting for input from the client.
On the client side, the request will time out if the reply takes longer than rpcclienttimeout.

@morcos
Copy link
Contributor

morcos commented Sep 18, 2015

Unfortunately I'm 0/2 on testing these. I couldn't recreate the SIGPIPE issue, in master or here, so thats's a start. But I still get what looks like a timeout from getblocktemplate_longpoll.py and I still get errors spewed to stdout when I don't have an IPv6 address.

EDIT: as per IRC:

  • SIGPIPE appears to be an OS X issue, which is why I couldn't recreate.
  • older versions of libevent don't support turning off the function call to turn off debugging

@laanwj
Copy link
Member Author

laanwj commented Sep 18, 2015

From IRC:

<morcos> regarding the rpc timeouts..  before 6695, you could fix the problem by starting the bitcoind's in the python tests with -rpctimeout=1500 (or whatever)
<morcos> doesn't that imply that somehow it was the timeout on the server end that was the problem?

Looks like the issue is not in bitcoin-cli but in the python http functionality that cannot cope with disconnects between requests (as said by @sdaftuar ). Which would be strange, but I'll take a look at it.

@laanwj
Copy link
Member Author

laanwj commented Sep 18, 2015

The rpcbind.py test hang in #6655 is caused by bitcoin-cli being unable to connect to IPv6 IPs. Not sure why. Also not sure whether this affects all libevent versions. But I can reproduce it with libevent 0x02001500 on Ubuntu 14.04 (which is relatively ancient).

@jonasschnelli
Copy link
Contributor

SIGPIPE appears to be an OS X issue, which is why I couldn't recreate.

I could reproduce it on debian as well. Use a bitcoin-cli command like getblocktemplate or getrawmempool and enter ctrl-C short after sending-off the command. Very hard to reproduce (requires a few tries).
With a script it would probably easy.

The SIGPIPE gets emitted when bitcoind starts to write to a closed socket (ctrl-c will terminate bitcoin-cli which ends up in writing to a closed socket on the server side).

@morcos
Copy link
Contributor

morcos commented Sep 18, 2015

@laanwj I know I said I didn't care, but I tracked down the silly warning messages. We need to call event_set_log_callback. I tried it and it works.

screen shot 2015-09-18 at 6 11 17 pm

@dcousens
Copy link
Contributor

concept ACK

@laanwj
Copy link
Member Author

laanwj commented Sep 19, 2015

@morcos Ah, that'd allow redirecting the messages to our own log. That makes sense in addition to what is done here.

Edit: done, the option is -debug=libevent now

@laanwj laanwj force-pushed the 2015_09_sigpipe_ignore branch from c2b12e5 to 897553e Compare September 19, 2015 11:30
@laanwj
Copy link
Member Author

laanwj commented Sep 19, 2015

The Python BadStatusLine on disconnect issue sounds like https://bugs.python.org/issue3566 "httplib persistent connections violate MUST in RFC2616 sec 8.1.4."

This was fixed in Python 3.5.

I've pushed a commit that should work around it.

@laanwj laanwj force-pushed the 2015_09_sigpipe_ignore branch from 958115d to c0db185 Compare September 19, 2015 16:24
@morcos
Copy link
Contributor

morcos commented Sep 20, 2015

ACK (modulo missing newlines)

Add a option "-debug=libevent" to enable libevent debugging for troubleshooting.
Libevent logging is redirected to our own log.
@laanwj laanwj force-pushed the 2015_09_sigpipe_ignore branch from c0db185 to 66391e9 Compare September 21, 2015 11:25
@laanwj
Copy link
Member Author

laanwj commented Sep 21, 2015

Fixed the newline issue (thanks @morcos), removed a spurious change in util.py

…meout

The two timeouts for the server and client, are essentially different:

- In the case of the server it should be a lower value to avoid clients
clogging up connection slots

- In the case of the client it should be a high value to accomedate slow
  responses from the server, for example for slow queries or when the
  lock is contended

Split the options into `-rpcservertimeout` and `-rpcclienttimeout` with
respective defaults of 30 and 900.
Python's httplib does not graciously handle disconnections from the http server, resulting in BadStatusLine errors.
See https://bugs.python.org/issue3566 "httplib persistent connections violate MUST in RFC2616 sec 8.1.4."

This was fixed in Python 3.5.

Work around it for now.
@laanwj laanwj force-pushed the 2015_09_sigpipe_ignore branch from 66391e9 to ddf98d1 Compare September 21, 2015 15:15
@laanwj laanwj merged commit ddf98d1 into bitcoin:master Sep 21, 2015
laanwj added a commit that referenced this pull request Sep 21, 2015
ddf98d1 Make RPC tests cope with server-side timeout between requests (Wladimir J. van der Laan)
2190ea6 rpc: Split option -rpctimeout into -rpcservertimeout and -rpcclienttimeout (Wladimir J. van der Laan)
8b2d6ed http: Disable libevent debug logging, if not explicitly enabled (Wladimir J. van der Laan)
5ce43da init: Ignore SIGPIPE (Wladimir J. van der Laan)
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bitcoind exits with SIGPIPE after Ctrl+C in bitcoin-cli

4 participants