In this article I’m going to describe two features of Spring Cloud Gateway: retrying based on GatewayFilter pattern and timeout handling based on a global configuration. In some previous articles in this series I have described rate limiting based on Redis, and a circuit breaker pattern built with Resilience4J. For more details about those two features you may refer to the following blog posts:

• Rate Limiting In Spring Cloud Gateway with Redis
• Circuit Breaking in Spring Cloud Gateway with Resilience4J

Example

We use the same repository as for two previous articles about Spring Cloud Gateway. The address of repository is https://github.com/piomin/sample-spring-cloud-gateway.git. The test class dedicated for the current article is GatewayRetryTest.

Implementation and testing

As you probably know most of the operations in Spring Cloud Gateway are realized using filter pattern, which is an implementation of Spring Framework GatewayFilter. Here, we can modify incoming requests and outgoing responses before or after sending the downstream request.
The same as for examples described in my two previous articles about Spring Cloud Gateway we will build JUnit test class. It leverages Testcontainers MockServer for running mock exposing REST endpoints.
Before running the test we need to prepare a sample route containing Retry filter. When defining this type of GatewayFilter we may set multiple parameters. Typically you will use the following three of them:

• retries – the number of retries that should be attempted for a single incoming request. The default value of this property is 3
• statuses – the list of HTTP status codes that should be retried, represented by using org.springframework.http.HttpStatus enum name.
• backoff – the policy used for calculating Spring Cloud Gateway timeout between subsequent retry attempts. By default this property is disabled.

Let’s start from the simplest scenario – using default values of parameters. In that case we just need to set a name of GatewayFilter for a route – Retry.

@ClassRule
public static MockServerContainer mockServer = new MockServerContainer();

@BeforeClass
public static void init() {
   System.setProperty("spring.cloud.gateway.routes[0].id", "account-service");
   System.setProperty("spring.cloud.gateway.routes[0].uri", "http://192.168.99.100:" + mockServer.getServerPort());
   System.setProperty("spring.cloud.gateway.routes[0].predicates[0]", "Path=/account/**");
   System.setProperty("spring.cloud.gateway.routes[0].filters[0]", "RewritePath=/account/(?<path>.*), /$\\{path}");
   System.setProperty("spring.cloud.gateway.routes[0].filters[1].name", "Retry");
   MockServerClient client = new MockServerClient(mockServer.getContainerIpAddress(), mockServer.getServerPort());
   client.when(HttpRequest.request()
      .withPath("/1"), Times.exactly(3))
      .respond(response()
         .withStatusCode(500)
         .withBody("{\"errorCode\":\"5.01\"}")
         .withHeader("Content-Type", "application/json"));
   client.when(HttpRequest.request()
      .withPath("/1"))
      .respond(response()
         .withBody("{\"id\":1,\"number\":\"1234567891\"}")
         .withHeader("Content-Type", "application/json"));
   // OTHER RULES
}

Our test method is very simple. It is just using Spring Framework TestRestTemplate to perform a single call to the test endpoint.

@Autowired
TestRestTemplate template;

@Test
public void testAccountService() {
   LOGGER.info("Sending /1...");
   ResponseEntity r = template.exchange("/account/{id}", HttpMethod.GET, null, Account.class, 1);
   LOGGER.info("Received: status->{}, payload->{}", r.getStatusCodeValue(), r.getBody());
   Assert.assertEquals(200, r.getStatusCodeValue());
}

Before running the test we will change a logging level for Spring Cloud Gateway logs, to see the additional information about the retrying process.

logging.level.org.springframework.cloud.gateway.filter.factory: TRACE

Since we didn’t set any backoff policy the subsequent attempts were replied without any delay. As you see on the picture below, a default number of retries is 3, and the filter is trying to retry all HTTP 5XX codes (SERVER_ERROR).

Now, let’s provide a little more advanced configuration. We can change the number of retries and set the exact HTTP status code for retrying instead of the series of codes. In our case a retried status code is HTTP 500, since it is returned by our mock endpoint. We can also enable backoff retrying policy beginning from 50ms to max 500ms. The factor is 2 what means that the backoff is calculated by using formula prevBackoff * factor. A formula is becoming slightly different when you set property basedOnPreviousValue to false – firstBackoff * (factor ^ n). Here’s the appropriate configuration for our current test.

@ClassRule
public static MockServerContainer mockServer = new MockServerContainer();

@BeforeClass
public static void init() {
   System.setProperty("spring.cloud.gateway.routes[0].id", "account-service");
   System.setProperty("spring.cloud.gateway.routes[0].uri", "http://192.168.99.100:" + mockServer.getServerPort());
   System.setProperty("spring.cloud.gateway.routes[0].predicates[0]", "Path=/account/**");
   System.setProperty("spring.cloud.gateway.routes[0].filters[0]", "RewritePath=/account/(?<path>.*), /$\\{path}");
   System.setProperty("spring.cloud.gateway.routes[0].filters[1].name", "Retry");
   System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.retries", "10");
   System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.statuses", "INTERNAL_SERVER_ERROR");
   System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.backoff.firstBackoff", "50ms");
   System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.backoff.maxBackoff", "500ms");
   System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.backoff.factor", "2");
   System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.backoff.basedOnPreviousValue", "true");
   MockServerClient client = new MockServerClient(mockServer.getContainerIpAddress(), mockServer.getServerPort());
   client.when(HttpRequest.request()
      .withPath("/1"), Times.exactly(3))
      .respond(response()
         .withStatusCode(500)
         .withBody("{\"errorCode\":\"5.01\"}")
         .withHeader("Content-Type", "application/json"));
   client.when(HttpRequest.request()
      .withPath("/1"))
      .respond(response()
         .withBody("{\"id\":1,\"number\":\"1234567891\"}")
         .withHeader("Content-Type", "application/json"));
   // OTHER RULES
}

If you run the same test one more time with a new configuration the logs look a little different. I have highlighted the most important differences in the picture below. As you see the current number of retries 10 only for HTTP 500 status. After setting a backoff policy the first retry attempt is performed after 50ms, the second after 100ms, the third after 200ms etc.

We have already analyzed the retry mechanism in Spring Cloud Gateway. Timeouts is another important aspect of request routing. With Spring Cloud Gateway we may easily set a global read and connect timeout. Alternatively, we may also define them for each route separately. Let’s add the following property to our test route definition. It sets a global timeout on 100ms. Now, our test route contains a test Retry filter with newly added global read timeout on 100ms.

System.setProperty("spring.cloud.gateway.httpclient.response-timeout", "100ms");

Alternatively, we may set timeout per single route. If we would prefer such a solution here a line we should add to our sample test.

System.setProperty("spring.cloud.gateway.routes[1].metadata.response-timeout", "100");

Then we define another test endpoint available under context path /2 with 200ms delay. Our current test method is pretty similar to the previous one, except that we are expecting HTTP 504 as a result.

@Test
public void testAccountServiceFail() {
   LOGGER.info("Sending /2...");
   ResponseEntity<Account> r = template.exchange("/account/{id}", HttpMethod.GET, null, Account.class, 2);
   LOGGER.info("Received: status->{}, payload->{}", r.getStatusCodeValue(), r.getBody());
   Assert.assertEquals(504, r.getStatusCodeValue());
}

Let’s run our test. The result is visible in the picture below. I have also highlighted the most important parts of the logs. After several failed retry attempts the delay between subsequent attempts has been set to the maximum backoff time – 500ms. Since the downstream service is delayed 100ms, the visible interval between retry attempts is around 600ms. Moreover, Retry filter by default handles IOException and TimeoutException, what is visible in the logs (exceptions parameter).

Summary

The current article is the last in series about traffic management in Spring Cloud Gateway. I have already described the following patterns: rate limiting, circuit breaker, fallback, failure retries and timeouts handling. That is only a part of Spring Cloud Gateway features. I hope that my articles help you in building API gateway for your microservices in an optimal way.