It is likely that you are developing some form of (web\/RESTful) API, and in case it is publicly-facing (or even when it\u2019s internal), you normally want to rate-limit it somehow. That is, to limit the number of requests performed over a period of time, in order to save resources and protect from abuse.<\/p>\n
This can probably be achieved on web-server\/load balancer level with some clever configurations, but usually you want the rate limiter to be client-specific (i.e. each client of your API sohuld have a separate rate limit), and the way the client is identified varies. It\u2019s probably still possible to do it on the load balancer, but I think it makes sense to have it on the application level.<\/p>\n
I\u2019ll use spring-mvc for the example, but any web framework has a good way to plug an interceptor.<\/p>\n
So here\u2019s an example of a spring-mvc interceptor:<\/p>\n
@Component\r\npublic class RateLimitingInterceptor extends HandlerInterceptorAdapter {\r\n\r\n private static final Logger logger = LoggerFactory.getLogger(RateLimitingInterceptor.class);\r\n \r\n @Value(\"${rate.limit.enabled}\")\r\n private boolean enabled;\r\n \r\n @Value(\"${rate.limit.hourly.limit}\")\r\n private int hourlyLimit;\r\n\r\n private Map<String, Optional<SimpleRateLimiter>> limiters = new ConcurrentHashMap<>();\r\n \r\n @Override\r\n public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler)\r\n throws Exception {\r\n if (!enabled) {\r\n return true;\r\n }\r\n String clientId = request.getHeader(\"Client-Id\");\r\n \/\/ let non-API requests pass\r\n if (clientId == null) {\r\n return true;\r\n }\r\n SimpleRateLimiter rateLimiter = getRateLimiter(clientId);\r\n boolean allowRequest = limiter.tryAcquire();\r\n \r\n if (!allowRequest) {\r\n response.setStatus(HttpStatus.TOO_MANY_REQUESTS.value());\r\n }\r\n response.addHeader(\"X-RateLimit-Limit\", String.valueOf(hourlyLimit));\r\n return allowRequest;\r\n }\r\n \r\n private SimpleRateLimiter getRateLimiter(String clientId) {\r\n if (limiters.containsKey(clientId)) {\r\n return limiters.get(clientId);\r\n } else {\r\n synchronized(clientId.intern()) {\r\n \/\/ double-checked locking to avoid multiple-reinitializations\r\n if (limiters.containsKey(clientId)) {\r\n return limiters.get(clientId);\r\n }\r\n \r\n SimpleRateLimiter rateLimiter = createRateLimiter(clientId);\r\n \r\n limiters.put(clientId, rateLimiter);\r\n return rateLimiter;\r\n }\r\n }\r\n }\r\n\t\r\n\t@PreDestroy\r\n\tpublic void destroy() {\r\n\t\t\/\/ loop and finalize all limiters\r\n\t}\r\n}<\/pre>\nThis initializes rate-limiters per client on demand. Alternatively, on startup you could just loop through all registered API clients and create a rate limiter for each. In case the rate limiter doesn\u2019t allow more requests (tryAcquire() returns false), then raturn \u201cToo many requests\u201d and abort the execution of the request (return \u201cfalse\u201d from the interceptor).
<\/div> <\/div><\/p>\nThis sounds simple. But there are a few catches. You may wonder where the SimpleRateLimiter<\/code> above is defined. We\u2019ll get there, but first let\u2019s see what options do we have for rate limiter implementations.<\/p>\nThe most recommended one seems to be the guava RateLimiter<\/a>. It has a straightforward factory method that gives you a rate limiter for a specified rate (permits per second). However, it doesn\u2019t accomodate web APIs very well, as you can\u2019t initilize the RateLimiter with pre-existing number of permits. That means a period of time should elapse before the limiter would allow requests. There\u2019s another issue \u2013 if you have less than one permits per second (e.g. if your desired rate limit is \u201c200 requests per hour\u201d), you can pass a fraction (hourlyLimit \/ secondsInHour), but it still won\u2019t work the way you expect it to, as internally there\u2019s a \u201cmaxPermits\u201d field that would cap the number of permits to much less than you want it to. Also, the rate limiter doesn\u2019t allow bursts \u2013 you have exactly X permits per second, but you cannot spread them over a long period of time, e.g. have 5 requests in one second, and then no requests for the next few seconds. In fact, all of the above can be solved, but sadly, through hidden fields that you don\u2019t have access to. Multiple feature requests exist for years now, but Guava just doesn\u2019t update the rate limiter, making it much less applicable to API rate-limiting.<\/p>\nUsing reflection, you can tweak the parameters and make the limiter work. However, it\u2019s ugly, and it\u2019s not guaranteed it will work as expected. I have shown here<\/a> how to initialize a guava rate limiter with X permits per hour, with burstability and full initial permits. When I thought that would do, I saw that tryAcquire()<\/code> has a synchronized(..)<\/code> block. Will that mean all requests will wait for each other when simply checking whether allowed to make a request? That would be horrible.<\/p>\nSo in fact the guava RateLimiter is not meant for (web) API rate-limiting. Maybe keeping it feature-poor is Guava\u2019s way for discouraging people from misusing it?<\/p>\n
That\u2019s why I decided to implement something simple myself, based on a Java Semaphore. Here\u2019s the naive implementation<\/a>:<\/p>\npublic class SimpleRateLimiter {\r\n private Semaphore semaphore;\r\n private int maxPermits;\r\n private TimeUnit timePeriod;\r\n private ScheduledExecutorService scheduler;\r\n\r\n public static SimpleRateLimiter create(int permits, TimeUnit timePeriod) {\r\n SimpleRateLimiter limiter = new SimpleRateLimiter(permits, timePeriod);\r\n limiter.schedulePermitReplenishment();\r\n return limiter;\r\n }\r\n\r\n private SimpleRateLimiter(int permits, TimeUnit timePeriod) {\r\n this.semaphore = new Semaphore(permits);\r\n this.maxPermits = permits;\r\n this.timePeriod = timePeriod;\r\n }\r\n\r\n public boolean tryAcquire() {\r\n return semaphore.tryAcquire();\r\n }\r\n\r\n public void stop() {\r\n scheduler.shutdownNow();\r\n }\r\n\r\n public void schedulePermitReplenishment() {\r\n scheduler = Executors.newScheduledThreadPool(1);\r\n scheduler.schedule(() -> {\r\n semaphore.release(maxPermits - semaphore.availablePermits());\r\n }, 1, timePeriod);\r\n\r\n }\r\n}<\/pre>\nIt takes a number of permits (allowed number of requests) and a time period. The time period is \u201c1 X\u201d, where X can be second\/minute\/hour\/daily \u2013 depending on how you want your limit to be configured \u2013 per second, per minute, hourly, daily. Every 1 X a scheduler replenishes the acquired permits. There is no control for bursts (a client can spend all permits with a rapid succession of requests), there is no warm-up functionality, there is no gradual replenishment. Depending on what you want, this may not be ideal, but that\u2019s just a basic rate limiter that is thread-safe and doesn\u2019t have any blocking. I wrote a unit test to confirm that the limiter behaves properly, and also ran performance tests against a local application to make sure the limit is obeyed. So far it seems to be working.<\/p>\n
Are there alternatives? Well, yes \u2013 there are libraries like RateLimitJ<\/a> that uses Redis to implement rate-limiting. That would mean, however, that you need to setup and run Redis. Which seems like an overhead for \u201csimply\u201d having rate-limiting.<\/p>\nOn the other hand, how would rate-limiting work properly in a cluster of application nodes? Application nodes probably need some database or gossip protocol to share data about the per-client permits (requests) remaining? Not necessarily. A very simple approach to this issue would be to assume that the load balancer distributes the load equally among your nodes. That way you would just have to set the limit on each node to be equal to the total limit divided by the number of nodes. It won\u2019t be exact, but you rarely need it to be \u2013 allowing 5-10 more requests won\u2019t kill your application, allowing 5-10 less won\u2019t be dramatic for the users.<\/p>\n
That, however, would mean that you have to know the number of application nodes. If you employ auto-scaling (e.g. in AWS), the number of nodes may change depending on the load. If that is the case, instead of configuring a hard-coded number of permits, the replenishing scheduled job can calculate the \u201cmaxPermits\u201d on the fly, by calling an AWS (or other cloud-provider) API to obtain the number of nodes in the current auto-scaling group. That would still be simpler than supporting a redis deployment just for that.<\/p>\n
Overall, I\u2019m surprised there isn\u2019t a \u201ccanonical\u201d way to implement rate-limiting (in Java). Maybe the need for rate-limiting is not as common as it may seem. Or it\u2019s implemented manually \u2013 by temporarily banning API clients that use \u201ctoo much resources\u201d.<\/p>\n
\n