Add -mempoolnotify to call an external script on mempool transactions #6072
Conversation
|
Do you really want to call a script several times per second? You're better
off polling getrawmempool, I would think.
Not saying polling is better than a callback, but the script-execution
mechanism is very inefficient. With zeromq notifications, things may be
different.
|
|
I think RPC pooling is not the way to go. Having an incremental notification is definitely the best option. I also think having a script-execution mechanism is not that good, but it is the existing solution, therefore I just followed the same implementation. As discussed in #1974, I would also prefer a named pipe. This way it should be easier to integrate with different MQs other than zeromq. |
|
Yes, we really just need to add a notification socket for local listeners that need high volume data like this. It also solves related problems such as notification storms during tx storms etc. |
|
I understand that you just want to continue the existing mechanism, that is
generally a very reasonable thing to do.
However, no matter what, forking off up to several dozen shell scripts per
second is just not a good idea. That has nothing to do with not wanting
notification, but just with the current notification mechanism sucks (for
high volume events, but also for other reasons). As jgarzik says: let's fix
that first.
|
|
In the case of staying with script-execution, bitcoind could just fork once when starting and then write to stdin of each script (instead of script arguments). |
|
Sure; piping a high volume stream of events to a single program is reasonable as well. The point is to avoid initiating fresh executions of a script upon each new event. This is the same problem as CGI per-hit execution in the mid-1990s. There is a large menu of solutions from which to choose. |
|
Piping sounds the best for me as a low term, cheap solution. It has no external dependency, can handle script crash (a plugin like system can't) and allows great integration in 3rd infrastructures. |
|
Sockets are the easiest, least complex solution. Getting pipes to work correctly on all platforms is not easy. |
|
In the case of using sockets, should bitcoind listen/accept connections or connect to a given host:port? |
|
@promag The standard pattern is a listen socket for bitcoind. Processes attach to a localhost port and receive events. If that process dies, the socket automatically closes. The admin is responsible for installing whatever watchdog processes. |
|
I think ZMQ could be the long term perspective of these *notify features (especially for rapid I/O stuff like mempool). ZMQ supports asynchronous I/O over sockets. It runs on POSIX bases OSs and windows. |
|
@jonasschnelli Agreed and #5303 does that. However, sockets do not require a library so sometimes that is chosen. |
|
Closing. Consensus is to NAK as-is, as execution storms etc. are possible. However, it is OK to provide this event signal via some other means, such as zmq or socket. |
|
Noticed another PR similar to this one #5328 |
5 participants
This pull request adds support for transactions hitting the memory pool notifications. It is similar to -blocknotify and -walletnotify.
Useful when bitcoind is configured without wallet support and one wants to be notified as soon as the transaction arrives.