Spring Microservices Security Best Practices
In this article, I’ll describe several best practices for building microservices with Spring Boot and Spring Security. I’m going to focus only on the aspects related to security. If you are interested in the general list of best practices for building microservices with Spring Boot read my article Spring Boot Best Practices for Microservices. On the other hand, if you plan to run your applications on Kubernetes, you might be interested in the article Best Practices For Microservices on Kubernetes.
Before we start with a list of security “golden rules”, let’s analyze a typical microservices architecture. We will focus on components important for building a secure solution.
The picture visible below illustrates a typical microservices architecture built with Spring Cloud. There is an API gateway built on top of Spring Cloud Gateway. Since it is an entry point to our system, we will enable some important security mechanisms on it. There are several microservices hidden behind the gateway. There is also a discovery server, which allows localizing IP addresses using the name of services. And finally, there are some components that do not take part in communication directly. It is just a proposition of a few selected tools. You may choose other solutions providing the same features. Vault is a tool for securely storing and accessing secrets. Keycloak is an open-source identity and access management solution. Spring Cloud Config Server provides an HTTP API for external configuration. It may integrate with several third-party tools including Vault.
Let’s begin. Here’s our list of Spring Security best practices.
1. Enable rate limiting on the API gateway
An API gateway is an important pattern in microservice-based architecture. It acts as a single entry point into the whole system. It is responsible not only for request routing but also for several other things including security. Consequently, one of the most essential components of security we should enable on the gateway is rate limiting. It protects your API against DoS attacks, which can tank a server with unlimited HTTP requests.
Spring provides its own implementation of the API gateway pattern called Spring Cloud Gateway. On the other hand, Spring Cloud Gateway comes with a built-in implementation of a rate-limiting component. To sum up, you just need to include a single dependency to build a gateway application. Then you have to provide some configuration settings to enable rate limiting for a single route.
In order to enable a rate limiter on a gateway, we need to use a component called RequestRateLimiter GatewayFilter factory. It uses a RateLimiter implementation to determine if the current request is allowed to proceed. Otherwise, the gateway returns a status of HTTP 429 - Too Many Requests. The RequestRateLimiter implementation uses Redis as a backend.
Let’s take a look at a typical configuration of routes handled by Spring Cloud Gateway. There are three parameters that can be used to configure the rate limiter: replenishRate, burstCapacity, and requestedTokens. The replenishRate property is how many requests per second a single user may send. The burstCapacity property is the maximum number of requests a user can send in a single second. With the requestedTokens property, we may set the cost of a single token.
spring:
cloud:
gateway:
routes:
- id: account-service
uri: http://localhost:8090
predicates:
- Path=/account/**
filters:
- RewritePath=/account/(?<path>.*), /$\{path}
- name: RequestRateLimiter
args:
redis-rate-limiter.replenishRate: 1
redis-rate-limiter.burstCapacity: 60
redis-rate-limiter.requestedTokens: 15Then we should define a KeyResolver bean. A rate limiter defines all parameters per a single key returned by the resolver.
@Bean
KeyResolver authUserKeyResolver() {
return exchange -> ReactiveSecurityContextHolder.getContext()
.map(ctx -> ctx.getAuthentication().getPrincipal().toString());
}For more details, you may refer to the article Secure Rate Limiting with Spring Cloud Gateway.
2. Generate and propagate certificates dynamically
Should we use SSL in microservice-to-microservice communication? Of course yes. But the question is how will you handle certificates used by your microservices. There are several best practices related to SSL certificate management. For example, you should not issue certificates for long time periods. You should also automatically renew or refresh them.
There are some tools that can help in following best practices. One of the most popular of them is Vault from Hashicorp. It provides the PKI secrets engine, which is responsible for generating dynamic X.509 certificates. The simplest way to try Vault is to run it locally on a Docker container.
$ docker run --cap-add=IPC_LOCK -d --name vault -p 8200:8200 vaultThen, we may enable and configure the PKI engine in some simple steps. You can do it using Vault CLI or UI. For some more detailed information about it read my article SSL with Spring WebFlux and Vault PKI. For now, let’s enable PKI with TTL and then configure CA using CLI as shown below.
$ vault secrets enable pki
$ vault secrets tune -max-lease-ttl=8760h pki
$ vault write pki/root/generate/internal \
common_name=my-website.com \
ttl=8760hIn the next step, we will use Spring VaultTemplate to issue a certificate dynamically. The fragment of code visible below shows how to create a certificate request with 12h TTL and localhost as a Common Name. Firstly, let’s build such a request using the VaultCertificateRequest object. Then we will invoke the issueCertificate method on the VaultPkiOperations object. The generated CertificateBundle contains both a certificate and a private key.
private CertificateBundle issueCertificate() throws Exception {
VaultPkiOperations pkiOperations = vaultTemplate.opsForPki("pki");
VaultCertificateRequest request = VaultCertificateRequest.builder()
.ttl(Duration.ofHours(12))
.commonName("localhost")
.build();
VaultCertificateResponse response = pkiOperations
.issueCertificate("default", request);
CertificateBundle certificateBundle = response.getRequiredData();
return certificateBundle;
}Finally, we just need to use the method for generating a certificate in Vault on runtime. The default behavior of our web server needs to be overridden. To do that, we need to create a Spring @Component that implements WebServerFactoryCustomizer. Depending on the web server we need to customize a different WebServerFactory. Typically, for Spring MVC it is Tomcat and Netty for Spring WebFlux. Inside the customize method, we are generating a certificate and store it inside the keystore (cert + private key) and truststore (CA).
@Component
public class GatewayServerCustomizer implements
WebServerFactoryCustomizer<NettyReactiveWebServerFactory> {
@SneakyThrows
@Override
public void customize(NettyReactiveWebServerFactory factory) {
String keyAlias = "vault";
CertificateBundle bundle = issueCertificate();
KeyStore keyStore = bundle.createKeyStore(keyAlias);
String keyStorePath = saveKeyStoreToFile("server-key.pkcs12", keyStore);
Ssl ssl = new Ssl();
ssl.setEnabled(true);
ssl.setClientAuth(Ssl.ClientAuth.NEED);
ssl.setKeyStore(keyStorePath);
ssl.setKeyAlias(keyAlias);
ssl.setKeyStoreType(keyStore.getType());
ssl.setKeyPassword("");
ssl.setKeyStorePassword("123456");
X509Certificate caCert = bundle.getX509IssuerCertificate();
log.info("CA-SerialNumber: {}", caCert.getSerialNumber());
KeyStore trustStore = KeyStore.getInstance("pkcs12");
trustStore.load(null, null);
trustStore.setCertificateEntry("ca", caCert);
String trustStorePath = saveKeyStoreToFile("server-trust.pkcs12", trustStore);
ssl.setTrustStore(trustStorePath);
ssl.setTrustStorePassword("123456");
ssl.setTrustStoreType(trustStore.getType());
factory.setSsl(ssl);
factory.setPort(8443);
}
}3. Use SSL in microservices communication
Since using SSL on the edge of a microservices-based system is obvious, inter-service communication is sometimes considered to be non-secure. My recommendation is always the same. Better use SSL or not 🙂 But we should think about securing at least components that store sensitive data. One of them will probably be a config server. That’s obviously one of the Spring Security best practices. For Spring Boot microservices we can use a component called Spring Cloud Config Server. Since it is built on top of Spring MVC we may easily enable a secure connection on the server-side.
server:
port: ${PORT:8888}
ssl:
enabled: true
client-auth: need
key-store: classpath: server-key.jks
key-store-password: 123456
key-alias: configserver
trust-store: classpath: server-trust.jks
trust-store-password: 123456On the client side, we use a component called Spring Cloud Config Client. Since it is responsible for connecting with the server, we also need to handle SSL there. To do that we need to override the RestTemplate SSL configuration on the ConfigServicePropertySourceLocator bean. The fragment of code visible below uses a self-signed certificate, but we can easily implement here a strategy described in the previous section.
@Configuration
public class SSLConfigServiceBootstrapConfiguration {
@Autowired
ConfigClientProperties properties;
@Bean
public ConfigServicePropertySourceLocator configServicePropertySourceLocator() throws Exception {
final char[] password = "123456".toCharArray();
final ClassPathResource resource = new ClassPathResource("account.jks");
SSLContext sslContext = SSLContexts.custom()
.loadKeyMaterial(resource.getFile(), password, password)
.loadTrustMaterial(resource.getFile(), password, new TrustSelfSignedStrategy()).build();
CloseableHttpClient httpClient = HttpClients.custom()
.setSSLContext(sslContext)
.setSSLHostnameVerifier((s, sslSession) -> true)
.build();
HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory(httpClient);
ConfigServicePropertySourceLocator configServicePropertySourceLocator = new ConfigServicePropertySourceLocator(properties);
configServicePropertySourceLocator.setRestTemplate(new RestTemplate(requestFactory));
return configServicePropertySourceLocator;
}
}In the latest version of Spring Cloud Config, we can enable TLS traffic encryption in configuration. We just need to define the right settings using properties with a prefix
spring.cloud.config.tls.*.
What about encrypting communication between applications and a discovery server? You can choose between several available discovery servers supported in Spring Cloud. But let’s assume we use Eureka. Similarly to Spring Cloud Config, we use a high-level client to communicate with a server. So, in that case, we need to define a bean DiscoveryClientOptionalArgs, and also override SSL settings on the HTTP client there. The Eureka client uses the Jersey HTTP client, so we need to create an instance of EurekaJerseyClientBuilder to override the SSL configuration.
@Bean
public DiscoveryClient.DiscoveryClientOptionalArgs discoveryClientOptionalArgs() throws Exception {
DiscoveryClient.DiscoveryClientOptionalArgs args = new DiscoveryClient.DiscoveryClientOptionalArgs();
final char[] password = "123456".toCharArray();
final ClassPathResource resource = new ClassPathResource("account.jks");
SSLContext sslContext = SSLContexts.custom()
.loadKeyMaterial(resource.getFile(), password, password)
.loadTrustMaterial(resource.getFile(), password, new TrustSelfSignedStrategy()).build();
EurekaJerseyClientBuilder builder = new EurekaJerseyClientBuilder();
builder.withClientName("account-client");
builder.withMaxTotalConnections(10);
builder.withMaxConnectionsPerHost(10);
builder.withCustomSSL(sslContext);
args.setEurekaJerseyClient(builder.build());
return args;
}Finally, we may configure HTTPS in communication between microservices. Since we use RestTemplate or WebClient instances directly on the client side it is relatively easy to implement secure communication in that case.
4. Keep configuration data encrypted
The current one of the best practices for Spring microservices security is related to a configuration server. We should encrypt at least sensitive data like passwords or secrets stored there. Spring Cloud Config Server provides a built-in mechanism for that. But we can also use Vault as a backend store for Spring Cloud Config Server, where all data is encrypted by default.
We will start with a default encrypt mechanism provided by Spring Cloud Config Server. Firstly, we need to enable it in the configuration properties.
spring:
cloud:
config:
server:
encrypt:
enabled: trueThen, we have to configure a key store responsible for encrypting our sensitive data.
encrypt:
keyStore:
location: classpath:/config.jks
password: 123456
alias: config
secret: 123456Finally, we can set encrypted data instead of plain string with {cipher} prefix.
spring:
application:
name: account-service
security:
user:
password: '{cipher}AQBhpDVYHANrg59OGY7ioSbMdOrH7ZA0vfa2VqIvfxJK5vQp...'Alternatively, you can use it as a configuration data backend. To do that you should enable a Spring profile called vault.
spring.profiles.active=vaultThen, we may add an example secret.
$ vault write secret/hello value=world
$ vault read secret/hello5. Restrict access to the API resources
In the previous sections, we discussed such topics as authentication, traffic, and data encryption. But another important aspect of securing your applications is authorization and access to the API resources. If you think about web app authorization, the first approach that probably comes to your mind is OAuth 2.0 or OpenID Connect. OAuth 2.0 is the industry-standard protocol for authorization. Of course, it is supported by Spring Security. There are also multiple OAuth2 providers you can integrate your application with. One of them is Keycloak. I will use it in the example in this article. Firstly, let’s run Keycloak on a Docker container. By default, it exposes API and a web console on the port 8080.
$ docker run -d --name keycloak -p 8888:8080 \
-e KEYCLOAK_USER=spring \
-e KEYCLOAK_PASSWORD=spring123 \
jboss/keycloakWe are going to enable and configure OAuth 2.0 support on the API gateway. Besides spring-cloud-starter-gateway dependency, we need to include spring-boot-starter-oauth2-client and spring-cloud-starter-security to activate the TokenRelay filter. Then we have to provide the Spring Security configuration settings for the OAuth2 client.
spring:
security:
oauth2:
client:
provider:
keycloak:
token-uri: http://localhost:8080/auth/realms/master/protocol/openid-connect/token
authorization-uri: http://localhost:8080/auth/realms/master/protocol/openid-connect/auth
userinfo-uri: http://localhost:8080/auth/realms/master/protocol/openid-connect/userinfo
user-name-attribute: preferred_username
registration:
keycloak-with-test-scope:
provider: keycloak
client-id: spring-with-test-scope
client-secret: c6480137-1526-4c3e-aed3-295aabcb7609
authorization-grant-type: authorization_code
redirect-uri: "{baseUrl}/login/oauth2/code/keycloak"
keycloak-without-test-scope:
provider: keycloak
client-id: spring-without-test-scope
client-secret: f6fc369d-49ce-4132-8282-5b5d413eba23
authorization-grant-type: authorization_code
redirect-uri: "{baseUrl}/login/oauth2/code/keycloak"In the last step, we need to configure the Spring Security filter. Since Spring Cloud Gateway is built on top of Spring WebFlux, we need to annotate the configuration bean with @EnableWebFluxSecurity. Inside the filterChain method we are going to enable authorization for all the exchanges. We will also set OAuth2 as a default login method and finally disable CSRF.
@Configuration
@EnableWebFluxSecurity
public class SecurityConfig {
@Bean
public SecurityWebFilterChain filterChain(ServerHttpSecurity http) {
http.authorizeExchange(exchanges -> exchanges.anyExchange().authenticated())
.oauth2Login(withDefaults());
http.csrf().disable();
return http.build();
}
}As I mentioned before, we will have a token relay pattern between the gateway and microservices. A Token Relay is where an OAuth2 consumer acts as a Client and forwards the incoming token to outgoing resource requests. So, now let’s enable global method security and OAuth2 resources server for the downstream services.
@Configuration
@EnableWebSecurity
@EnableGlobalMethodSecurity(prePostEnabled = true)
public class SecurityConfig extends WebSecurityConfigurerAdapter {
protected void configure(HttpSecurity http) throws Exception {
http.authorizeRequests(authorize -> authorize.anyRequest().authenticated())
.oauth2ResourceServer(OAuth2ResourceServerConfigurer::jwt);
}
}After that, it is possible to configure role-based access using @PreAuthorize and @PostAuthorize. Let’s take a look at the implementation of the REST controller class. It is a single ping method. That method may be accessed only by the client with the TEST scope. For more implementation details you may refer to the article Spring Cloud Gateway OAuth2 with Keycloak.
@RestController
@RequestMapping("/callme")
public class CallmeController {
@PreAuthorize("hasAuthority('SCOPE_TEST')")
@GetMapping("/ping")
public String ping() {
SecurityContext context = SecurityContextHolder.getContext();
Authentication authentication = context.getAuthentication();
return "Scopes: " + authentication.getAuthorities();
}
}6. Dynamically generate credentials to the external systems
Does your application connect to external systems like databases or message brokers? How do you store the credentials used by your application? Of course, we can always encrypt sensitive data, but if we work with many microservices having separate databases it may not be a very comfortable solution. Here comes Vault with another handy mechanism. Its database secrets engine generates database credentials dynamically based on configured roles. We may also take advantage of dynamically generated credentials for RabbitMQ, Nomad, and Consul.
Firstly, let’s enable the Vault database engine, which is disabled by default.
$ vault secrets enable databaseLet’s assume our application connects to the Postgres database. Therefore, we need to configure a Vault plugin for PostgreSQL database and then provide connection settings and credentials.
$ vault write database/config/postgres \
plugin_name=postgresql-database-plugin \
allowed_roles="default" \
connection_url="postgresql://{{username}}:{{password}}@localhost:5432?sslmode=disable" \
username="postgres" \
password="postgres123456"
Then we need to create a database role. The name of the role should be the same as the name passed in field allowed_roles in the previous step. We also have to set a target database name and SQL statement that creates users with privileges.
$ vault write database/roles/default db_name=postgres \
creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}';GRANT SELECT, UPDATE, INSERT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO \"{{name}}\";" \
default_ttl="1h" max_ttl="24h"
Thanks to Spring Cloud Vault project we can easily integrate any Spring Boot application with the Vault databases engine. Two dependencies need to be included in Maven pom.xml to enable that support. Of course, we also need dependencies for the JPA and Postgres driver.
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-vault-config</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-vault-config-databases</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
</dependency>The only thing we have to do is to configure integration with Vault via Spring Cloud Vault. The following configuration settings should be placed in bootstrap.yml (no application.yml). You may consider running the application on Nomad.
spring:
cloud:
vault:
uri: http://localhost:8200
token: ${VAULT_TOKEN}
postgresql:
enabled: true
role: default
backend: database
datasource:
url: jdbc:postgresql://localhost:5432/posgtresThe important part of the configuration visible above is under the property spring.cloud.vault.postgresql. Following Spring Cloud documentation “Username and password are stored in spring.datasource.username and spring.datasource.password so using Spring Boot will pick up the generated credentials for your DataSource without further configuration”. For more details about integration between Spring Cloud and Vault database engine, you may refer to my article Secure Spring Cloud Microservices with Vault and Nomad.
7. Always be up to date
This one of the best practices may be applied anywhere not only as a rule to Spring microservices security. We usually use open-source libraries in our applications, so it is important to include the latest versions of them. They may contain critical updates for publicly disclosed vulnerabilities contained within a project’s dependencies. There are also several dependency scanners like Snyk or OWASP.
Final thoughts
That is my private list of best practices for Spring Boot microservices security. Of course, most of them are not related just to a single framework, and we apply them for any other framework or toolkit. Do you have your own list of Spring Security best practices? Don’t afraid to share it in the comments.
18 COMMENTS