Send IDONTWANT on publish to avoid downloading data we already have#6513
Send IDONTWANT on publish to avoid downloading data we already have#6513mergify[bot] merged 3 commits intosigp:unstablefrom
IDONTWANT on publish to avoid downloading data we already have#6513Conversation
jimmygchen
added
ready-for-merge
jxs
left a comment
jxs
left a comment
There was a problem hiding this comment.
Thanks for starting this JImmy, left a comment.
Meanwhile the spec is clear in stating that IDONTWANT's should be sent for received messages:
In more specific terms, a new control message is introduced: IDONTWANT. It's primarily intended to notify mesh peers that the node already received a message and there is no need to send its duplicate.
and:
When the peer receives the first message instance it immediately broadcasts (not queue for later piggybacking) IDONTWANT with the messageId to all its mesh peers. This could be performed prior to the message validation to further increase the effectiveness of the approach.
can we forward this conversation to the spec and open an issue there proposing to also update the spec?
jimmygchen
added
waiting-on-author
jxs
left a comment
jxs
left a comment
There was a problem hiding this comment.
Re-reading on your comment Jimmy,
though, and in most cases we won't be publishing the same messages as other, so perhaps we could also consider adding a send_idontwant bool flag to the behavour.publish function, and only set it to true when publishing blobs
I think this makes sense, we probably should offer this option to users
jimmygchen
added
ready-for-review
I had a chat with Age regarding this and we decided not to add this flag. The Let me know what you think! |
jxs
left a comment
jxs
left a comment
There was a problem hiding this comment.
LGTM thanks Jimmy!
I had a chat with Age regarding this and we decided not to add this flag.
The idontwant_message_size_threshold should limit the amount of IDONTWANT being sent to peers - if a message exceeds idontwant_message_size_threshold, it's probably worth sending an IDONTWANT.
Let me know what you think!
Yeah makes sense
jimmygchen
added
ready-for-merge
|
@mergify queue |
✅ The pull request has been merged automaticallyDetailsThe pull request has been merged automatically at 6a8d13e |
…sigp#6513) * Send `IDONTWANT` on publish to avoid downloading data we already have. * Merge branch 'unstable' into send-idontwant-on-publish * Move broadcast of `IDONTWANT` to after publishing.
5 participants
Issue Addressed
With the introduction of
getBlobsV1optimisation, nodes may be publishing blobs if they retrieved it from the EL but haven't received it from gossip. In these scenarios, the node already have the blobs and can save download bandwidth by sending outIDONTWANTto peers on the blobs we publish, so that it avoids downloading them from gossip.This would apply to all messages > 1000 bytes (
idontwant_message_size_threshold) though, and in most cases we won't be publishing the same messages as other, so perhaps we could also consider adding asend_idontwantbool flag to thebehavour.publishfunction, and only set it to true when publishing blobs?