-
Notifications
You must be signed in to change notification settings - Fork 24.5k
Closed
Description
Typically you need to send a cluster forget to each node in a cluster to delete a node. If you don't do this fast enough, the node will be re-added through gossip. Ideally you just need to forget a node and it will eventually be forgotten throughout the cluster.
This is part of #8948.
Possible ways to implement this:
- Append the contents of the clusterBlacklistNodes (node id and timestamp) to all cluster ping and pong messages. A ping extension (mechanism introduced for hostnames) can be used for this. (The blacklist expire timestamp needs to be gossiped too, so that we don't end up with the node being re-forgotten and re-added to the blacklist by gossip after the blacklist TTL is passed.) This increases the size of the cluster bus messages while any node is blacklisted (1 minute), but if the blacklist is short and the ping extension is small, this might be acceptable.
- Append the contents of the clusterBlacklistNodes (node id and timestamp) to every pong message which is a response to a ping, but only when we detect that the ping's gossip section contains a forgotten node, i.e. a node in our blacklist. This minimizes the extra data sent over the cluster bus, but it is enough to make sure a node is forgotten throughout the cluster?
- When a CLUSTER FORGET command is received, broadcast this to the cluster, for example in a pong with the forgotten node in a ping extension. This is done only once and by the node which receives the command. If a client sends CLUSTER FORGET to all nodes in a cluster, there will be NxN pongs sent in a short period of time.
@madolson et.al. any preference among the above or other ideas?
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels