From: torrust/torrust-demo#14
Relates to: #1033

We are having problems with the tracker demo. The logs contain many errors validating the connection ID. It looks like the client doesn't implement the protocol correctly because it's not sending the connection ID received from the connect request. Since the client is making many requests, this produces a lot of new ERROR records in the logs, ultimately depressing tracker performance.

Solution Overview:

1. Hierarchical Counting Bloom Filters:

   • Individual IP Layer: Use one CBF to track individual IP addresses. This will allow for fine-grained detection of misbehaving IPs.

   • Subnet Layer: Implement another level of CBFs where each filter covers a subnet range. This allows for detecting patterns of misbehavior across a subnet without penalizing neighboring IPs inappropriately.

Implementation Details:

• Individual IP CBF:

  • Each IP address is hashed into this filter.
  • When an error occurs, increment the count associated with that IP's hash in the CBF.
  • If the count exceeds a threshold, you can take action against that specific IP (e.g., rate limiting or temporary banning).

• Subnet CBF:

  • Instead of hashing individual IPs, hash the subnet address (e.g., the network part of an IP address).
  • When multiple IPs within a subnet misbehave, their errors are aggregated into this subnet's count.
  • If the count for a subnet exceeds a different, higher threshold, you can apply measures at the subnet level, like rate limiting traffic from that subnet.

Advantages:

• Granularity: This approach gives you both detailed control over individual IPs and the ability to detect broader patterns of misbehavior within subnets.

• Performance: CBFs are efficient in terms of memory usage and speed, allowing for quick lookups and updates even with large datasets.

• Flexibility: You can adjust the thresholds for individual IPs and subnets separately, allowing for different levels of tolerance based on your policy or observed behavior patterns.

• False Positives: While CBFs have a small chance of false positives, by using multiple levels (individual and subnet), you can mitigate the impact. For example, if an IP is flagged at both levels, it's more likely to be a true positive.

Challenges:

• Configuration: You need to decide on the size of the CBFs, the number of hash functions, and the error thresholds for both individual IPs and subnets. This requires some experimentation or simulation to find the right balance between false positives, memory usage, and effectiveness.

• Complexity: Managing two layers of CBFs introduces additional complexity in terms of implementation and maintenance.

• False Positives at Subnet Level: If a subnet contains both misbehaving and well-behaving IPs, the well-behaving ones might suffer from the actions taken against the subnet.

Implementation Steps:

1. Decide on the Subnet Size: Determine what constitutes a subnet for your purposes (e.g., /24, /16, etc.).

2. Initialize CBFs:

   • Create one CBF for individual IPs.
   • Create another CBF for subnets, where each bucket represents a subnet.

3. Error Handling Logic:

   • When an error occurs:
     • Hash the IP to update the individual IP CBF.
     • Extract the subnet from the IP and hash it to update the subnet CBF.

4. Action Protocol:

   • If an individual IP's count exceeds a threshold, apply rate limiting or other measures to that IP.
   • If a subnet's count exceeds a higher threshold, consider similar measures but at the subnet level.

5. Decay Mechanism: Implement a decay or aging process for counts to ensure that past behavior doesn't indefinitely affect current interactions unless the behavior persists.

By employing this hierarchical approach with Counting Bloom Filters, you can effectively manage IP-based errors at different levels of granularity, protecting your network's performance while minimizing the impact on innocent IPs.

Originally posted by @da2ce7 in torrust/torrust-demo#14 (comment)