rpc: provide per message stats for global traffic via new RPC 'getnetmsgstats' #29418

vasild · 2024-02-11T09:32:51Z

Introduce a new RPC, getnetmsgstats to retrieve traffic bytes and count of messages possibly broken down by:

direction (sent or received)
network (ipv4, tor, etc)
connection type (outbound-full-relay, block-relay-only, etc)
message type (verack, ping, etc)

Implements: #26337 Add per message stats to getnettotals rpc

Examples:

bitcoin-cli getnetmsgstats '["direction", "network", "connection_type", "message_type"]'

{
  "bytes": 5392756,
  "count": 32683
}

bitcoin-cli getnetmsgstats '["direction", "network", "connection_type"]'

{
  "sendcmpct": {
    "bytes": 975,
    "count": 31
  },
  "sendaddrv2": {
    "bytes": 816,
    "count": 28
  },
  "headers": {
    "bytes": 1786,
    "count": 28
  },
  "feefilter": {
    "bytes": 816,
    "count": 27
  },
  "version": {
    "bytes": 4843,
    "count": 37
  },
  "ping": {
    "bytes": 1104,
    "count": 36
  },
  "getaddr": {
    "bytes": 384,
    "count": 13
  },
  "block": {
    "bytes": 5128467,
    "count": 20365
  },
  "getheaders": {
    "bytes": 29436,
    "count": 28
  },
  "pong": {
    "bytes": 1040,
    "count": 34
  },
  "getdata": {
    "bytes": 1218925,
    "count": 20008
  },
  "sendheaders": {
    "bytes": 447,
    "count": 16
  },
  "wtxidrelay": {
    "bytes": 816,
    "count": 28
  },
  "addrv2": {
    "bytes": 301161,
    "count": 25
  },
  "verack": {
    "bytes": 816,
    "count": 28
  }
}

bitcoin-cli getnetmsgstats '["direction", "connection_type", "message_type"]'

{
  "i2p": {
    "bytes": 1311861,
    "count": 8226
  },
  "onion": {
    "bytes": 3932760,
    "count": 24272
  },
  "ipv6": {
    "bytes": 631265,
    "count": 3362
  },
  "ipv4": {
    "bytes": 1300106,
    "count": 7773
  }
}

bitcoin-cli getnetmsgstats

{
  "recv": {
    "i2p": {
      "manual": {
        "sendcmpct": {
          "bytes": 90,
          "count": 3
        },
        "sendaddrv2": {
          "bytes": 99,
          "count": 3
        },
        "headers": {
          "bytes": 309,
          "count": 3
        },
        "feefilter": {
          "bytes": 87,
          "count": 3
        },
        "version": {
          "bytes": 408,
          "count": 3
        },
        "ping": {
          "bytes": 87,
          "count": 3
        },
        "block": {
          "bytes": 432950,
          "count": 1794
        },
        "getheaders": {
          "bytes": 3150,
          "count": 3
        },
        "pong": {
          "bytes": 87,
          "count": 3
        },
        "wtxidrelay": {
          "bytes": 99,
          "count": 3
        },
        "addrv2": {
          "bytes": 68508,
          "count": 3
        },
        "verack": {
          "bytes": 99,
          "count": 3
        }
      }
    },
    "onion": {
      "block-relay-only": {
        "sendcmpct": {
          "bytes": 66,
          "count": 2
        },
        "sendaddrv2": {
          "bytes": 24,
          "count": 1
        },
        "headers": {
          "bytes": 106,
          "count": 1
        },
        "feefilter": {
          "bytes": 32,
          "count": 1
        },
        "version": {
          "bytes": 126,
          "count": 1
        },
        "ping": {
          "bytes": 64,
          "count": 2
        },
        "block": {
          "bytes": 478479,
          "count": 1896
        },
        "getheaders": {
          "bytes": 1053,
          "count": 1
        },
        "pong": {
          "bytes": 64,
          "count": 2
        },
        "sendheaders": {
          "bytes": 24,
          "count": 1
        },
        "wtxidrelay": {
          "bytes": 24,
          "count": 1
        },
        "verack": {
          "bytes": 24,
          "count": 1
        }
      },
      "outbound-full-relay": {
        "sendcmpct": {
          "bytes": 195,
          "count": 6
        },
        "sendaddrv2": {
          "bytes": 129,
          "count": 5
        },
        "headers": {
          "bytes": 527,
          "count": 5
        },
        "feefilter": {
          "bytes": 157,
          "count": 5
        },
        "version": {
          "bytes": 639,
          "count": 5
        },
        "ping": {
          "bytes": 189,
          "count": 6
        },
        "block": {
          "bytes": 1465688,
          "count": 5957
        },
        "getheaders": {
          "bytes": 5262,
          "count": 5
        },
        "pong": {
          "bytes": 189,
          "count": 6
        },
        "sendheaders": {
          "bytes": 24,
          "count": 1
        },
        "wtxidrelay": {
          "bytes": 129,
          "count": 5
        },
        "addrv2": {
          "bytes": 120693,
          "count": 9
        },
        "verack": {
          "bytes": 129,
          "count": 5
        }
      }
    },
    "ipv6": {
      "manual": {
        "sendcmpct": {
          "bytes": 120,
          "count": 4
        },
        "sendaddrv2": {
          "bytes": 132,
          "count": 4
        },
        "headers": {
          "bytes": 412,
          "count": 4
        },
        "feefilter": {
          "bytes": 116,
          "count": 4
        },
        "version": {
          "bytes": 544,
          "count": 4
        },
        "ping": {
          "bytes": 116,
          "count": 4
        },
        "block": {
          "bytes": 138074,
          "count": 583
        },
        "getheaders": {
          "bytes": 4200,
          "count": 4
        },
        "pong": {
          "bytes": 116,
          "count": 4
        },
        "wtxidrelay": {
          "bytes": 132,
          "count": 4
        },
        "addrv2": {
          "bytes": 93436,
          "count": 4
        },
        "verack": {
          "bytes": 132,
          "count": 4
        }
      },
      "outbound-full-relay": {
        "version": {
          "bytes": 126,
          "count": 1
        }
      }
    },
    "ipv4": {
      "outbound-full-relay": {
        "sendcmpct": {
          "bytes": 66,
          "count": 2
        },
        "sendaddrv2": {
          "bytes": 24,
          "count": 1
        },
        "headers": {
          "bytes": 106,
          "count": 1
        },
        "feefilter": {
          "bytes": 32,
          "count": 1
        },
        "version": {
          "bytes": 252,
          "count": 2
        },
        "ping": {
          "bytes": 32,
          "count": 1
        },
        "block": {
          "bytes": 313988,
          "count": 1255
        },
        "getheaders": {
          "bytes": 1053,
          "count": 1
        },
        "pong": {
          "bytes": 32,
          "count": 1
        },
        "sendheaders": {
          "bytes": 24,
          "count": 1
        },
        "wtxidrelay": {
          "bytes": 24,
          "count": 1
        },
        "addrv2": {
          "bytes": 40,
          "count": 1
        },
        "verack": {
          "bytes": 24,
          "count": 1
        }
      }
    }
  },
  "sent": {
    "i2p": {
      "manual": {
        "sendcmpct": {
          "bytes": 90,
          "count": 3
        },
        "sendaddrv2": {
          "bytes": 99,
          "count": 3
        },
        "headers": {
          "bytes": 66,
          "count": 3
        },
        "feefilter": {
          "bytes": 87,
          "count": 3
        },
        "version": {
          "bytes": 544,
          "count": 4
        },
        "ping": {
          "bytes": 87,
          "count": 3
        },
        "getaddr": {
          "bytes": 99,
          "count": 3
        },
        "getheaders": {
          "bytes": 3150,
          "count": 3
        },
        "pong": {
          "bytes": 87,
          "count": 3
        },
        "getdata": {
          "bytes": 105382,
          "count": 1789
        },
        "sendheaders": {
          "bytes": 99,
          "count": 3
        },
        "wtxidrelay": {
          "bytes": 99,
          "count": 3
        },
        "verack": {
          "bytes": 99,
          "count": 3
        }
      }
    },
    "onion": {
      "block-relay-only": {
        "sendcmpct": {
          "bytes": 33,
          "count": 1
        },
        "sendaddrv2": {
          "bytes": 24,
          "count": 1
        },
        "headers": {
          "bytes": 25,
          "count": 1
        },
        "version": {
          "bytes": 127,
          "count": 1
        },
        "ping": {
          "bytes": 64,
          "count": 2
        },
        "getheaders": {
          "bytes": 1053,
          "count": 1
        },
        "pong": {
          "bytes": 64,
          "count": 2
        },
        "getdata": {
          "bytes": 115932,
          "count": 1884
        },
        "sendheaders": {
          "bytes": 24,
          "count": 1
        },
        "wtxidrelay": {
          "bytes": 24,
          "count": 1
        },
        "verack": {
          "bytes": 24,
          "count": 1
        }
      },
      "outbound-full-relay": {
        "sendcmpct": {
          "bytes": 162,
          "count": 5
        },
        "sendaddrv2": {
          "bytes": 129,
          "count": 5
        },
        "headers": {
          "bytes": 122,
          "count": 5
        },
        "feefilter": {
          "bytes": 157,
          "count": 5
        },
        "version": {
          "bytes": 771,
          "count": 6
        },
        "ping": {
          "bytes": 221,
          "count": 7
        },
        "getaddr": {
          "bytes": 129,
          "count": 5
        },
        "getheaders": {
          "bytes": 5262,
          "count": 5
        },
        "pong": {
          "bytes": 189,
          "count": 6
        },
        "getdata": {
          "bytes": 363478,
          "count": 5932
        },
        "sendheaders": {
          "bytes": 129,
          "count": 5
        },
        "wtxidrelay": {
          "bytes": 129,
          "count": 5
        },
        "addrv2": {
          "bytes": 231,
          "count": 4
        },
        "verack": {
          "bytes": 129,
          "count": 5
        }
      }
    },
    "ipv6": {
      "manual": {
        "sendcmpct": {
          "bytes": 120,
          "count": 4
        },
        "sendaddrv2": {
          "bytes": 132,
          "count": 4
        },
        "headers": {
          "bytes": 88,
          "count": 4
        },
        "feefilter": {
          "bytes": 116,
          "count": 4
        },
        "version": {
          "bytes": 544,
          "count": 4
        },
        "ping": {
          "bytes": 116,
          "count": 4
        },
        "getaddr": {
          "bytes": 132,
          "count": 4
        },
        "getheaders": {
          "bytes": 4200,
          "count": 4
        },
        "pong": {
          "bytes": 116,
          "count": 4
        },
        "getdata": {
          "bytes": 35492,
          "count": 584
        },
        "sendheaders": {
          "bytes": 99,
          "count": 3
        },
        "wtxidrelay": {
          "bytes": 132,
          "count": 4
        },
        "verack": {
          "bytes": 132,
          "count": 4
        }
      },
      "outbound-full-relay": {
        "version": {
          "bytes": 254,
          "count": 2
        }
      }
    },
    "ipv4": {
      "outbound-full-relay": {
        "sendcmpct": {
          "bytes": 33,
          "count": 1
        },
        "sendaddrv2": {
          "bytes": 24,
          "count": 1
        },
        "headers": {
          "bytes": 25,
          "count": 1
        },
        "feefilter": {
          "bytes": 32,
          "count": 1
        },
        "version": {
          "bytes": 508,
          "count": 4
        },
        "ping": {
          "bytes": 32,
          "count": 1
        },
        "getaddr": {
          "bytes": 24,
          "count": 1
        },
        "getheaders": {
          "bytes": 1053,
          "count": 1
        },
        "pong": {
          "bytes": 32,
          "count": 1
        },
        "getdata": {
          "bytes": 76531,
          "count": 1231
        },
        "sendheaders": {
          "bytes": 24,
          "count": 1
        },
        "wtxidrelay": {
          "bytes": 24,
          "count": 1
        },
        "verack": {
          "bytes": 24,
          "count": 1
        }
      }
    }
  }
}

Previous incarnations of this:
#27534 rpc: add 'getnetmsgstats', new rpc to view network message statistics (Thank you, @satsie!)
#28926 rpc: add 'getnetmsgstats' RPC (Thank you, @willcl-ark!)

DrahtBot · 2024-02-11T09:32:54Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/29418.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	waketraindev
Concept ACK	epiccurious

If your review is incorrectly listed, please copy-paste  into the comment that the bot should ignore.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#34162 (net: Avoid undershooting in GetAddressesUnsafe by fjahr)
#34049 (rpc: Disallow captures in RPCMethodImpl by ajtowns)
#30951 (net: option to disallow v1 connection on ipv4 and ipv6 peers by stratospher)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

LLM Linter (✨ experimental)

Possible typos and grammar issues:

getnettotals counts also bytes that are not accounted at any message -> getnettotals also counts bytes not accounted for by any message [clarifies awkward phrasing: "accounted at" is nonstandard; "accounted for by" or "accounted for in" is correct and clearer]

^{drahtbot_id_5_m}

DrahtBot · 2024-02-11T10:51:04Z

🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the
documentation.

Possibly this is due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.

Leave a comment here, if you need help tracking down a confusing failure.

_{Debug: https://github.com/bitcoin/bitcoin/runs/21448984768}

epiccurious · 2024-02-11T22:26:43Z

Concept ACK 91c90d759503aa3fa15b330bba2148698cdc271d.

DrahtBot · 2024-04-04T14:24:37Z

Needs rebase if still relevant

0xB10C · 2024-04-09T16:20:24Z

just fyi as this came up: You could count up message stats with the inbound and outbound message tracepoints. However, this does only gives you stats from the points on where you start hooking into the tracepoints, might not be as user friendly (RPC is easier to use than the tracepoints), and the tracepoints currently only work on Linux.

While I personally don't need it, I see why some node operators need traffic statistics. E.g. How much of my (limited monthly) traffic is used up by inbound connections?

DrahtBot · 2024-04-23T06:28:45Z

Converted to draft due to failing CI and inactivity

vasild · 2024-05-09T13:40:37Z

91c90d7595...6bfda84b92: rebase, pick changes from #29421 and adjust the newly added test.

…e constant b3efb48 protocol: make message types constexpr (Vasil Dimov) 2fa9de0 net: make the list of known message types a compile time constant (Vasil Dimov) Pull request description: Turn the `std::vector` to `std::array` because it is cheaper and allows us to have the number of the messages as a compile time constant: `ALL_NET_MESSAGE_TYPES.size()` which can be used in future code to build other `std::array`s with that size. --- This change is part of #29418 but it makes sense on its own and would be good to have it, regardless of the fate of #29418. Also, if this is merged, that would reduce the size of #29418, thus the current standalone PR. ACKs for top commit: achow101: ACK b3efb48 jonatack: ACK b3efb48 maflcko: utACK b3efb48 🎊 willcl-ark: ACK b3efb48 Tree-SHA512: 6d3860c138c64514ebab13d97ea67893e2d346dfac30a48c3d9bc769a1970407375ea4170afdb522411ced306a14a9af4eede99e964d1fb1ea3efff5d5eb57af

vasild · 2024-05-23T09:51:33Z

6bfda84b92...b5189f543c: rebase due to conflicts (part of this PR was merged via #29421)

vasild · 2025-01-24T16:16:28Z

a46cbdd478...db442f3d6e: do #29418 (comment)

vasild · 2025-03-20T17:38:29Z

db442f3d6e...626cc06c69: rebase due to conflicts

src/rpc/net.cpp

vasild · 2025-05-21T09:00:09Z

626cc06c69...528b4d6b9c: rebase and do #29418 (comment)

test/functional/rpc_net.py

Before this change only per-peer stats were gathered. They are lost when the peer disconnects. So, collect the traffic stats globally in `CConnman`, broken down by * direction (sent or received, (2)) * network of the peer (IPv4, IPv6, Tor, I2P, CJDNS (5)) * connection type (inbound, full outbound, feeler, etc, (6)) * message type (verack, ping, etc, (36))

vasild · 2025-09-09T08:03:00Z

528b4d6b9c...87d3d640e6: rebase due to conflicts and fix a typo in comment

Introduce a new RPC `getnetmsgstats` that renders the global traffic stats from `CConnman`. Co-authored-by: Vasil Dimov <[email protected]>

Aggregation can be by either one of direction, network, connection type or message type. For example if the following ``` { "ipv4": { "ping": 3 }, "ipv6": { "ping": 4 } } ``` is aggregated by network, then the result will be ``` { "ping": 7 } ```

Co-authored-by: Vasil Dimov <[email protected]>

vasild · 2025-09-10T07:57:12Z

87d3d640e6...7d4e82c0bb: remove leading \n from RPC help

waketraindev · 2025-12-29T01:19:29Z

Concept ACK

Useful netstats for tracking. Normally, I'd sum and aggregate peer stats. This approach simplifies that.

waketraindev · 2025-12-29T16:30:51Z

ACK 7d4e82c

lgtm. Tested on master and with v30.1 in my node monitoring dashboard.

DrahtBot added the RPC/REST/ZMQ label Feb 11, 2024

vasild mentioned this pull request Feb 11, 2024

rpc: add 'getnetmsgstats' RPC #28926

Closed

DrahtBot added the CI failed label Feb 11, 2024

vasild force-pushed the getnetmsgstats branch from bc156d8 to 91c90d7 Compare February 11, 2024 12:45

DrahtBot removed the CI failed label Feb 11, 2024

vasild mentioned this pull request Feb 11, 2024

net: make the list of known message types a compile time constant #29421

Merged

This was referenced Feb 11, 2024

net, net_processing: additional and consistent disconnect logging #28521

Merged

Introduce 'getblockfileinfo' RPC command #27770

Closed

Prune locks #19463

Closed

This was referenced Feb 11, 2024

log, refactor: Allow log macros to accept context arguments #29256

Open

Stratum v2 Template Provider (take 3) #29432

Closed

DrahtBot added the CI failed label Mar 12, 2024

DrahtBot mentioned this pull request Mar 13, 2024

scripted-diff: Use LogInfo over LogPrintf #29641

Merged

achow101 requested a review from mzumsande April 9, 2024 15:52

DrahtBot marked this pull request as draft April 23, 2024 06:28

vasild force-pushed the getnetmsgstats branch from 91c90d7 to 6bfda84 Compare May 9, 2024 13:40

DrahtBot removed the CI failed label May 9, 2024

DrahtBot added the Needs rebase label May 21, 2024

vasild force-pushed the getnetmsgstats branch from 6bfda84 to b5189f5 Compare May 23, 2024 09:50

vasild marked this pull request as ready for review May 23, 2024 09:51

DrahtBot removed the Needs rebase label May 23, 2024

vasild force-pushed the getnetmsgstats branch from a46cbdd to db442f3 Compare January 24, 2025 16:15

This was referenced Mar 10, 2025

[IBD] specialize block serialization #31868

Draft

Replace libevent with our own HTTP and socket-handling implementation #32061

Open

DrahtBot added the Needs rebase label Mar 20, 2025

vasild force-pushed the getnetmsgstats branch from db442f3 to 626cc06 Compare March 20, 2025 17:38

DrahtBot removed the Needs rebase label Mar 20, 2025

DrahtBot added the CI failed label May 18, 2025

maflcko reviewed May 20, 2025

View reviewed changes

src/rpc/net.cpp Outdated Show resolved Hide resolved

vasild force-pushed the getnetmsgstats branch from 626cc06 to 528b4d6 Compare May 21, 2025 08:59

DrahtBot removed the CI failed label May 21, 2025

DrahtBot reviewed Jun 25, 2025

View reviewed changes

test/functional/rpc_net.py Outdated Show resolved Hide resolved

DrahtBot mentioned this pull request Jun 27, 2025

fuzz: Make process_message(s) more deterministic #32822

Merged

DrahtBot added the Needs rebase label Jul 25, 2025

vasild force-pushed the getnetmsgstats branch from 528b4d6 to 87d3d64 Compare September 9, 2025 08:02

DrahtBot removed the Needs rebase label Sep 9, 2025

satsie and others added 3 commits September 10, 2025 09:54

rpc: visualize global CConnman stats in a new RPC getnetmsgstats

b9e530e

Introduce a new RPC `getnetmsgstats` that renders the global traffic stats from `CConnman`. Co-authored-by: Vasil Dimov <[email protected]>

test: add functional tests for the new getnetmsgstats RPC

7d4e82c

Co-authored-by: Vasil Dimov <[email protected]>

vasild force-pushed the getnetmsgstats branch from 87d3d64 to 7d4e82c Compare September 10, 2025 07:56

This was referenced Dec 7, 2025

net: Waste less time in socket handling #34025

Merged

rpc: Disallow captures in RPCMethodImpl #34049

Open

DrahtBot mentioned this pull request Dec 29, 2025

net: Avoid undershooting in GetAddressesUnsafe #34162

Open

rpc: provide per message stats for global traffic via new RPC 'getnetmsgstats' #29418

Are you sure you want to change the base?

rpc: provide per message stats for global traffic via new RPC 'getnetmsgstats' #29418

Conversation

vasild commented Feb 11, 2024

Uh oh!

DrahtBot commented Feb 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage & Benchmarks

Reviews

Conflicts

LLM Linter (✨ experimental)

Uh oh!

DrahtBot commented Feb 11, 2024

Uh oh!

epiccurious commented Feb 11, 2024

Uh oh!

DrahtBot commented Apr 4, 2024

Uh oh!

0xB10C commented Apr 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DrahtBot commented Apr 23, 2024

Uh oh!

vasild commented May 9, 2024

Uh oh!

vasild commented May 23, 2024

Uh oh!

vasild commented Jan 24, 2025

Uh oh!

vasild commented Mar 20, 2025

Uh oh!

Uh oh!

vasild commented May 21, 2025

Uh oh!

Uh oh!

vasild commented Sep 9, 2025

Uh oh!

vasild commented Sep 10, 2025

Uh oh!

waketraindev commented Dec 29, 2025

Uh oh!

waketraindev commented Dec 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

DrahtBot commented Feb 11, 2024 •

edited

Loading

0xB10C commented Apr 9, 2024 •

edited

Loading