Java HTTP Server and Virtual Threads
In this article, you will learn how to create an HTTP server with Java and use virtual threads for handling incoming requests. We will compare this solution with an HTTP server that uses a standard thread pool. Our test will compare memory usage in both scenarios under a heavy load of around 200 concurrent requests.
If you like articles about Java you can also read my post about unknown and useful Java features. It is not my first article about virtual threads. I have already written about Java 19 virtual threads and support for them in the Quarkus framework in this article.
Source Code
If you would like to try it by yourself, you may always take a look at my source code. In order to do that you need to clone my GitHub repository. After that, you should follow my instructions.
Prerequisites
In order to do the exercise on your laptop you need to have JDK 19+ and Maven installed.
Enable Virtual Threads
Even if you have Java 19 that’s not all. Since virtual threads are still a preview feature in Java 19 we need to enable it during compilation. With Maven we need to enable preview features using maven-compiler-plugin as shown below.
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.10.1</version>
<configuration>
<release>19</release>
<compilerArgs>
--enable-preview
</compilerArgs>
</configuration>
</plugin>
</plugins>
</build>Create HTTP Server with Virtual Threads
We don’t need much to create an HTTP or even HTTPS server with Java. In Java API, an object called HttpServer allows us to achieve it very easily. Once we will create the server we can override a default thread executor with the setExecutor method. No matter which type of executor we choose, there is one requirement that must be fulfilled by our server. It needs to be able to handle 200 requests simultaneously. Therefore for standard Java threads, we will create a pool with a maximum size of 200. For virtual threads, there is no sense to create any pools. They do not consume many resources since they are related directly to the OS.
Let’s take a look at the fragment of code visible below. That’s our method for creating an HTTP server. It will listen on 8080 port (1) under the /example context path (2). The SimpleDelayedHandler object handles all incoming requests. Depending on the value of the withLock variable, it will simulate delay without locking (false) or with ReentrantLock (true). In order to simplify the exercise, we can switch between standard (4) and virtual threads executor (3) using the single boolean parameter. After setting all required parameters, we can start the server (5).
private static void runServer(boolean virtual, boolean withLock)
throws IOException {
HttpServer httpServer = HttpServer
.create(new InetSocketAddress(8080), 0); // (1)
httpServer.createContext("/example",
new SimpleDelayedHandler(withLock)); // (2)
if (virtual) {
httpServer.setExecutor(
Executors.newVirtualThreadPerTaskExecutor()
); // (3)
} else {
httpServer.setExecutor(
Executors.newFixedThreadPool(200)
); // (4)
}
httpServer.start(); // (5)
}Then, we need to call the runServer method from the main method. We will test 4 scenarios depending on the value of two input arguments. We will discuss it in the next section.
public static void main(String[] args) throws IOException {
runServer(true, false);
}After running the server you can make a test call using the following command:
$ curl http://localhost:8080/exampleBuild Test Scenarios
As mentioned before, we will run four test scenarios. In the first two of them, we just compare the performance of the HTTP server with the standard thread pool and with virtual threads. We will simulate the processing time with the Thread.sleep method. In the next two scenarios, we will simulate the usage of the workers’ pool (1). For example, it can be something similar to using a JDBC connection pool in the REST app. There are 50 workers handling 200 requests (2). Those workers will also delay the thread execution with the Thread.sleep method, but this time they will lock the thread at the beginning of execution and unlock it at the end.
Depending on the value of the withLock input argument we will use the workers’ pool (3) or we will just sleep the thread (4). In both cases, we will finally return the response Ping_ and incremented number (5) represented by the AtomicLong object. Here’s the implementation of our handler.
public class SimpleDelayedHandler implements HttpHandler {
private final List<SimpleWork> workers =
new ArrayList<>(); // (1)
private final int workersCount = 50;
private final boolean withLock;
AtomicLong id = new AtomicLong();
public SimpleDelayedHandler(boolean withLock) {
this.withLock = withLock;
if (withLock) {
for (int i = 0; i < workersCount; i++) { // (2)
workers.add(new SimpleWork());
}
}
}
@Override
public void handle(HttpExchange t) throws IOException {
String response = null;
if (withLock) {
response = workers
.get((int) (id.incrementAndGet() % workersCount))
.doJob();
} else {
try {
Thread.sleep(200);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
response = "Ping_" + id.incrementAndGet();
}
t.sendResponseHeaders(200, response.length());
OutputStream os = t.getResponseBody();
os.write(response.getBytes());
os.close();
}
}Here’s the implementation of our worker. As you it also sleeps the thread (this time for 100 milliseconds). However, during that time it locks the object. Since we have 50 worker objects in the pool only 50 threads may use it at the same time. Others will wait until the lock will be released.
public class SimpleWork {
AtomicLong id = new AtomicLong();
ReentrantLock lock = new ReentrantLock();
public String doJob() {
String response = null;
lock.lock();
try {
Thread.sleep(100);
response = "Ping_" + id.incrementAndGet();
} catch (InterruptedException e) {
throw new RuntimeException();
} finally {
lock.unlock();
}
return response;
}
}Load Test for Java Virtual vs Standard Threads
Let’s begin with the first scenario. We will test standard threads without any locking workers simulation.
public static void main(String[] args) throws IOException {
runServer(false, false);
}We can make some warmup tests as shown below. I’m using the siege tool for load testing. We can define the number of concurrent threads and the number of repetitions.
In the right test, we will simulate 200 concurrent requests.
$ siege http://localhost:8080/example -c 200 -r 500Let’s switch to the profiler view. Here you can see heap memory usage during the test. The usage is around 300 MB, while the reservation is more than 500 MB.
Let’s take a look at the telemetry view. As you see there are ~200 running threads.
Now, we will run the same test for the HTTP server using virtual threads. Let’s restart the application with the following arguments:
public static void main(String[] args) throws IOException {
runServer(true, false);
}Let’s switch to the profiler view once again. Here you can see heap memory usage during the test. You can compare it to the previous results. Now the usage is around 180 MB, while the reservation is around 300 MB.
Here’s the telemetry view. There are just some (~10) platform threads that “carry” virtual threads.
Here’s the visualization of the thread pool from the beginning of the test. As you see there are just some platform threads (CarrierThreads) and a lot of short-lived virtual threads.
Locks with Virtual Threads
In the end, let’s make the same checks, but this time with our worker objects pool that uses ReentrantLock to synchronize threads. Firstly, we will start the app with the following arguments to test standard threads.
public static void main(String[] args) throws IOException {
runServer(false, true);
}In fact, for standard threads, the main difference is in thread pool visualization. As you see, now many threads waiting for the lock to release. Our workers’ pool became a bottleneck for the app.
It doesn’t have any impact on RAM usage in comparison to the previous test for standard Java threads.
And finally the last scenario. Now, we will do the same check for virtual threads.
public static void main(String[] args) throws IOException {
runServer(true, true);
}Here are the results for memory usage.
In thread pool visualization we have just some “carrier” threads. As you see they are not “locked”.
In the “Thread Monitor” view there are a lot of virtual threads that wait a moment until the lock is released.
Of course, you can clone my GitHub repo and make your own tests. I was using JProfiler for memory and threads visualization.
Final Thoughts
Java virtual threads are really long-awaited feature. Since they are still in the preview status in Java 19 we need to wait for their wide adoption in the most popular Java libraries. Unfortunately, even Java 19 is not an LTS and if you are working for one of those companies that only use LTS versions you will have to wait for Java 21 which should be released in September 2023. Nevertheless, virtual threads can reduce the effort of writing, maintaining, and observing especially for high-throughput concurrent applications. We can use them as simply as the standard Java threads. The aim of this article was to show you how you can start with virtual threads to build your own solution, for example, an HTTP server. Then you can easily compare the difference in performance between standard and virtual threads.
8 COMMENTS