fix: several improvements to how AddrMan works#781
Conversation
673d7e1 to
dc1cb67
Compare
dc1cb67 to
8427273
Compare
|
Pushed 8427273:
|
|
Also, it would be great to implement a rate-limiting for addrv2 messages. |
I Agree. Bitcoin Core has a strict rate limit of accepting only one address every ten seconds from a peer to avoid eclipsing. |
|
While you're at it, maybe also make WARN floresta_wire::p2p_wire::node: PeerNotFoundAtAddress(178.250.189.42, 38333)
ERROR bdk_floresta: Failed to manually disconnect from peer 178.250.189.42:38333It claims the peer is not found even though I'm literally connected to it. |
jaoleal
left a comment
There was a problem hiding this comment.
Complementing with some suggestions
| good_peers_by_service: HashMap::new(), | ||
| peers_by_service: HashMap::new(), | ||
| max_size: max_size.unwrap_or(MAX_ADDRESSES), | ||
| reachable_networks, |
There was a problem hiding this comment.
Isnt interesting to have a default for this and it being the addresses we support ?
There was a problem hiding this comment.
Knowing this isn't really the address_man business, and wouldn't help much. So I would say no
This method is used for the |
1703ed1 to
724469e
Compare
|
Pushed 724469e
This was annoying to rebase |
|
I had to simulate the behaviour of the Bitcoin Core isRoutable() function in Rust-Bitcoin for a target in bitcoinfuzz. You might find it useful to see the additional verification that it has, which might be missing in Floresta, to ensure that it behaves equally to Core and does not waste time connecting to a non-routeable address. |
fadc867 to
3124891
Compare
|
After a little 1v1 against CI, pushed 3124891
|
The BIP demands nodes to ignore any message with more than 1_000 addresses. We currently may receive and send packages bigger than that. This commit fixes it by enforcing this limit.
We don't currently push peers added through CLI to our AddrMan, therefore we don't learn about them to use in the future. This commit fixes it, by calling `addr_man.push` with any peer that have been added from the CLI
Some networks might not be reachable for use, usually because they require a specific proxy that we might not have. Therefore, addresses for that net are always unreachable for us. This commit adds a method to filter for unreachable addresses, so they are never added to our address manager.
We try to keep a good view of the network, knowing which peers are working a ready to connect. To accieve this, we keep track of the lastest connections we've made, so we know what peers should be online and responsive. However, after a certain time from the last connection, we assume that the information we have is too old to be counted. But our current code is too aggressive in what it considers outdated, and will drop this info for peers we've connected just a few minutes ago. This commit fixes this by preserving this information for one day (24 hours) before considering it stale and moving that peer to the unknown state.
The address manager implements some methods like `is_private` and `is_good_peer` to help with peer selection. However, they are closer to the LocalAddress struct, so I've moved them to LocalAddress and made them public.
To avoid being eclipsed with an address spam attack, we limit the rate of addrv2 messages a peer can send us to one every 10 seconds.
DNS seeds are trusted to deliver recently seen peers, that are very likely to still be alive. This commit adds those peers directly to the "good" peers table, making it easier for our node to start up peers
Before connecting to an address, we mark it as `Failed`, in case we never hear from that peer again, and only after handshake we mark as `Connected`. However, before this commit, we would remove then from the `good_peers` bucket, and only re-add after disconnecting. That meant we wouldn't send their addresses in `AddrV2` messages, and they wouldn't count towards the `enough_peers` function. This commit fixes this by adding `Connected` peers back to the good peers bucket
Before this commit we would add duplicate addresses to each one of the per-service bucket. This commit checks whether the address already existed there, and only adds if not
JoseSK999
left a comment
There was a problem hiding this comment.
One last nit, if it makes sense to you
| impl FromStr for LocalAddress { | ||
| fn from_str(s: &str) -> Result<Self, Self::Err> { | ||
| LocalAddress::try_from(s) | ||
| } | ||
| type Err = std::net::AddrParseError; | ||
| } | ||
|
|
||
| // Note that, since we can't know the network we are operating in, this code | ||
| // can't know what's the default port. Therefore, it will only work if you give | ||
| // a SocketAddr, i.e. <IP:PORT> | ||
| impl TryFrom<&str> for LocalAddress { | ||
| fn try_from(value: &str) -> Result<Self, Self::Error> { | ||
| let split = value.split(':').collect::<Vec<_>>(); | ||
| let address = split[0].parse::<Ipv4Addr>()?; | ||
| let port = if let Some(port) = split.get(1) { | ||
| port.parse().unwrap_or(8333) | ||
| } else { | ||
| 8333 | ||
| let address = value.parse::<SocketAddr>()?; | ||
| let ip = match address { | ||
| SocketAddr::V4(ipv4) => AddrV2::Ipv4(*ipv4.ip()), | ||
| SocketAddr::V6(ipv6) => AddrV2::Ipv6(*ipv6.ip()), | ||
| }; | ||
|
|
||
| Ok(LocalAddress::new( | ||
| AddrV2::Ipv4(address), | ||
| ip, | ||
| SystemTime::now() | ||
| .duration_since(UNIX_EPOCH) | ||
| .unwrap() | ||
| .as_secs(), |
There was a problem hiding this comment.
Optional nit: we are now repeating this same logic 9 times in the file. A helper would be good:
fn time_since_unix() -> u64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
}c5b38b6 to
9125089
Compare
|
Pushed db86233
|
|
Missed linting |
There was a problem hiding this comment.
| .unwrap_or_default() |
There was a problem hiding this comment.
Azure is so annoying
100%
This method returns the time elapsed since the unix epoch. This logic is reused all over addr_man, so it is better to just have it somewhere and use it.
db86233 to
21e316e
Compare
|
ACK 21e316e |
|
BOOM |
In getfloresta#793 we reduced the `DNS_SEED_REQUEST_INTERVAL` to two minutes. This tells how long do we wait before re-doing a request do the DNS seeds. The goal was to make sure we had enough peers to keep going. However, it doesn't seem like it worked as intended since DNS servers are currently bad at giving utreexo peers (and on signet they are also bad at giving CBS peers). This forces our node to retry DNS requests every two minutes. But since most users don't have their own recursive resolver (I don't), the previous result is almost always cached, so you get basically the same answer several times. This pollutes stdout for no good reason. Since getfloresta#781 we now are super aggressive about asking for addresses and checking whether they are alive and learning about their capabilities and getfloresta#865 will make it even better, since we can try several feeler connections at the same time. So I think DNS seeds should be only used to get us some starting peers, and then we build our local network view from the P2P net ourselves.
In getfloresta#793 we reduced the `DNS_SEED_REQUEST_INTERVAL` to two minutes. This tells how long do we wait before re-doing a request do the DNS seeds. The goal was to make sure we had enough peers to keep going. However, it doesn't seem like it worked as intended since DNS servers are currently bad at giving utreexo peers (and on signet they are also bad at giving CBS peers). This forces our node to retry DNS requests every two minutes. But since most users don't have their own recursive resolver (I don't), the previous result is almost always cached, so you get basically the same answer several times. This pollutes stdout for no good reason. Since getfloresta#781 we now are super aggressive about asking for addresses and checking whether they are alive and learning about their capabilities and getfloresta#865 will make it even better, since we can try several feeler connections at the same time. So I think DNS seeds should be only used to get us some starting peers, and then we build our local network view from the P2P net ourselves.
In getfloresta#793 we reduced the `DNS_SEED_REQUEST_INTERVAL` to two minutes. This tells how long do we wait before re-doing a request do the DNS seeds. The goal was to make sure we had enough peers to keep going. However, it doesn't seem like it worked as intended since DNS servers are currently bad at giving utreexo peers (and on signet they are also bad at giving CBS peers). This forces our node to retry DNS requests every two minutes. But since most users don't have their own recursive resolver (I don't), the previous result is almost always cached, so you get basically the same answer several times. This pollutes stdout for no good reason. Since getfloresta#781 we now are super aggressive about asking for addresses and checking whether they are alive and learning about their capabilities and getfloresta#865 will make it even better, since we can try several feeler connections at the same time. So I think DNS seeds should be only used to get us some starting peers, and then we build our local network view from the P2P net ourselves.
In getfloresta#793 we reduced the `DNS_SEED_REQUEST_INTERVAL` to two minutes. This tells how long do we wait before re-doing a request do the DNS seeds. The goal was to make sure we had enough peers to keep going. However, it doesn't seem like it worked as intended since DNS servers are currently bad at giving utreexo peers (and on signet they are also bad at giving CBS peers). This forces our node to retry DNS requests every two minutes. But since most users don't have their own recursive resolver (I don't), the previous result is almost always cached, so you get basically the same answer several times. This pollutes stdout for no good reason. Since getfloresta#781 we now are super aggressive about asking for addresses and checking whether they are alive and learning about their capabilities and getfloresta#865 will make it even better, since we can try several feeler connections at the same time. So I think DNS seeds should be only used to get us some starting peers, and then we build our local network view from the P2P net ourselves.
0ceeb59 chore(wire): increase the dns seed grace period time (Davidson Souza) Pull request description: ### Description and Notes In #793 we reduced the `DNS_SEED_REQUEST_INTERVAL` to two minutes. This tells how long do we wait before re-doing a request do the DNS seeds. The goal was to make sure we had enough peers to keep going. However, it doesn't seem like it worked as intended since DNS servers are currently bad at giving utreexo peers (and on signet they are also bad at giving CBS peers). This forces our node to retry DNS requests every two minutes. But since most users don't have their own recursive resolver (I don't), the previous result is almost always cached, so you get basically the same answer several times. This pollutes stdout for no good reason. Since #781 we now are super aggressive about asking for addresses and checking whether they are alive and learning about their capabilities and #865 will make it even better, since we can try several feeler connections at the same time. So I think DNS seeds should be only used to get us some starting peers, and then we build our local network view from the P2P net ourselves. ACKs for top commit: luisschwab: ACK 0ceeb59 JoseSK999: ACK 0ceeb59 Tree-SHA512: a3529e147ca18ba18eab7b59918b17521a70addf9de7c37750612e58b6aea9f3173869672454126f516b2e38bd3d7f597d5e3731b850d2b86201982b8e14eae7
Description and Notes
Fixes #853
This PR improves our
AddrMan, and also makes our node more aggressive on requesting for new addresses. With these changes:AddrManAddrMan