Currently Spring Cloud Gateway is second the most popular Spring Cloud project just after Spring Cloud Netflix (in terms of a number of stars on GitHub). It has been created as a successor of Zuul proxy in the Spring Cloud family. This project provides an API Gateway for microservices architecture, and is built on top of reactive Netty and Project Reactor. It is designed to provide a simple, but effective way to route to APIs and address such popular concerns as security, monitoring/metrics, and resiliency.
Spring Cloud Gateway offers you many features and configuration options. Today I’m going to focus on the single one, but very interesting aspect of gateway configuration – rate limiting. A rate limiter may be defined as a way to control the rate of traffic sent or received on the network. We can also define a few types of rate limiting. Spring Cloud Gateway currently provides a Request Rate Limiter, which is responsible for restricting each user to N requests per second.
When using RequestRateLimiter with Spring Cloud Gateway we may leverage Redis. Spring Cloud implementation uses token bucket algorithm to do rate limiting. This algorithm has a centralized bucket host where you take tokens on each request, and slowly drip more tokens into the bucket. If the bucket is empty, it rejects the request.

1. Dependencies

We will test our sample application against Spring Cloud Gateway rate limiting under higher traffic. First, we need to include some dependencies. Of course, Spring Cloud Gateway starter is required. For handling rate limiter with Redis we also need to add dependency to spring-boot-starter-data-redis-reactive starter. Other dependencies are used for test purposes. Module mockserver provided within Testcontainers. It is responsible for mocking a target service. In turn, the library mockserver-client-java is used for integration with mockserver container during the test. And the last library junit-benchmarks is used for a benchmarking test method and running the test concurrently.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-data-redis-reactive</artifactId>
</dependency>
<dependency>
   <groupId>org.projectlombok</groupId>
   <artifactId>lombok</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-test</artifactId>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>org.testcontainers</groupId>
   <artifactId>mockserver</artifactId>
   <version>1.12.3</version>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>org.mock-server</groupId>
   <artifactId>mockserver-client-java</artifactId>
   <version>3.10.8</version>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>com.carrotsearch</groupId>
   <artifactId>junit-benchmarks</artifactId>
   <version>0.7.2</version>
   <scope>test</scope>
</dependency>

The sample application is built on top of Spring Boot 2.2.1.RELEASE and uses Spring Cloud Hoxton.RC2 Release Train.

<parent>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-parent</artifactId>
   <version>2.2.1.RELEASE</version>
</parent>

<properties>
   <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
   <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
   <java.version>11</java.version>
   <spring-cloud.version>Hoxton.RC2</spring-cloud.version>
</properties>

<dependencyManagement>
   <dependencies>
      <dependency>
         <groupId>org.springframework.cloud</groupId>
         <artifactId>spring-cloud-dependencies</artifactId>
         <version>${spring-cloud.version}</version>
         <type>pom</type>
         <scope>import</scope>
      </dependency>
   </dependencies>
</dependencyManagement>

2. Implementation

Request rate limiting is realized using a Spring Cloud Gateway component called GatewayFilter. Each instance of this filter is constructed in a specific factory. Filter is of course responsible for modifying requests and responses before or after sending the downstream request. Currently, there are 30 available built-in gateway filter factories.
The GatewayFilter takes an optional keyResolver parameter and parameters specific to the rate limiter implementation (in that case an implementation using Redis). Parameter keyResolver is a bean that implements the KeyResolver interface. It allows you to apply different strategies to derive the key for limiting requests. Parameter keyResolver is a bean that implements the KeyResolver interface. It allows you to apply different strategies to derive the key for limiting requests. Following Spring Cloud Gateway documentation:

The default implementation of KeyResolver is the PrincipalNameKeyResolver which retrieves the Principal from the ServerWebExchange and calls Principal.getName(). By default, if the KeyResolver does not find a key, requests will be denied. This behavior can be adjusted with the spring.cloud.gateway.filter.request-rate-limiter.deny-empty-key (true or false) and spring.cloud.gateway.filter.request-rate-limiter.empty-key-status-code properties.

Since, we have discussed some theoretical aspects of Spring Cloud Gateway rate limiting we may proceed to the implementation. First, let’s define the main class and very simple KeyResolver bean, that is always equal to one.

@SpringBootApplication
public class GatewayApplication {

   public static void main(String[] args) {
      SpringApplication.run(GatewayApplication.class, args);
   }

   @Bean
   KeyResolver userKeyResolver() {
      return exchange -> Mono.just("1");
   }
}

Assuming we have the following configuration and a target application running on port 8091 we may perform some test calls. You may set two properties for customizing the process. The redis-rate-limiter.replenishRate decides how many requests per second a user is allowed to send without any dropped requests. This is the rate that the token bucket is filled. The second property redis-rate-limiter.burstCapacity is the maximum number of requests a user is allowed to do in a single second. This is the number of tokens the token bucket can hold. Setting this value to zero will block all requests.

server:
  port: ${PORT:8085}

spring:
  application:
    name: gateway-service
  redis:
    host: 192.168.99.100
    port: 6379
  cloud:
    gateway:
      routes:
      - id: account-service
        uri: http://localhost:8091
        predicates:
        - Path=/account/**
        filters:
        - RewritePath=/account/(?.*), /$\{path}
      - name: RequestRateLimiter
          args:
            redis-rate-limiter.replenishRate: 10
            redis-rate-limiter.burstCapacity: 20

Now, if you call the endpoint exposed by the gateway you get the following response. It includes some specific headers, which are prefixed by x-ratelimit. Header x-ratelimit-burst-capacity indicates to burstCapacity value, x-ratelimit-replenish-rate indicates to replenishRate value, and the most important x-ratelimit-remaining, which shows you the number of requests you may send in the next second.

If you exceed the number of allowed requests Spring Cloud Gateway return response with code HTTP 429 - Too Many Requests, and will not process the incoming request.

3. Testing Spring Cloud Gateway rate limiting

We have the Spring Boot test that uses two Docker containers provided by Testcontainers: MockServer and Redis. Because the exposed port is generated dynamically we need to set gateway properties in @BeforeClass method before running the test. Inside the init method we also use MockServerClient to define mock service on the mock server container. Our test method is running concurrently in six threads and is repeated 600 times.

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
@RunWith(SpringRunner.class)
public class GatewayRateLimiterTest {

    private static final Logger LOGGER = LoggerFactory.getLogger(GatewayRateLimiterTest.class);

    @Rule
    public TestRule benchmarkRun = new BenchmarkRule();

    @ClassRule
    public static MockServerContainer mockServer = new MockServerContainer();
    @ClassRule
    public static GenericContainer redis = new GenericContainer("redis:5.0.6").withExposedPorts(6379);

    @Autowired
    TestRestTemplate template;

    @BeforeClass
    public static void init() {
        System.setProperty("spring.cloud.gateway.routes[0].id", "account-service");
        System.setProperty("spring.cloud.gateway.routes[0].uri", "http://192.168.99.100:" + mockServer.getServerPort());
        System.setProperty("spring.cloud.gateway.routes[0].predicates[0]", "Path=/account/**");
        System.setProperty("spring.cloud.gateway.routes[0].filters[0]", "RewritePath=/account/(?<path>.*), /$\\{path}");
        System.setProperty("spring.cloud.gateway.routes[0].filters[1].name", "RequestRateLimiter");
        System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.redis-rate-limiter.replenishRate", "10");
        System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.redis-rate-limiter.burstCapacity", "20");
        System.setProperty("spring.redis.host", "192.168.99.100");
        System.setProperty("spring.redis.port", "" + redis.getMappedPort(6379));
        new MockServerClient(mockServer.getContainerIpAddress(), mockServer.getServerPort())
                .when(HttpRequest.request()
                        .withPath("/1"))
                .respond(response()
                        .withBody("{\"id\":1,\"number\":\"1234567890\"}")
                        .withHeader("Content-Type", "application/json"));
    }

    @Test
    @BenchmarkOptions(warmupRounds = 0, concurrency = 6, benchmarkRounds = 600)
    public void testAccountService() {
        ResponseEntity<Account> r = template.exchange("/account/{id}", HttpMethod.GET, null, Account.class, 1);
        LOGGER.info("Received: status->{}, payload->{}, remaining->{}", r.getStatusCodeValue(), r.getBody(), r.getHeaders().get("X-RateLimit-Remaining"));
    }

}

Let’s take a look at the test result. After starting gateway allows a user to send max 20 requests in a single second. After exceeding this value it starts to return HTTP 429.

After dropping some incoming requests gateway starts to accept them in the next second. But this time it allows to process only 10 requests, which is equal to replenishRate parameter value.

The source code is available in GitHub repository: https://github.com/piomin/sample-spring-cloud-gateway.git.