[Translation] Performance Battle: Web MVC vs Webflux

1. Introduction

While learning reactive programming recently, I suddenly wondered: is a new programming paradigm necessarily better than the traditional one? In what aspects does reactive programming improve performance?

The following content is translated from: Spring Benchmark – Web MVC vs Webflux

The author roughly compared the performance characteristics of traditional Spring MVC and WebFlux. Since the comparison is just a simple hello world endpoint, real-world business request complexity would be much higher.

The original blog post is from August 2022, which is still quite valuable for reference. If you use Notion, you can bookmark it using this link.

2. Main Content

In the Java world, every process is designed as a CPU thread-based concept, implemented through blocking operations and imperative code. Therefore, the original Java web servers adopted a thread-per-request approach, following the Servlet specification. However, thread-based programming has some limitations because a CPU can only handle a limited number of threads at a time, around ten thousand.

In today's business environment, some internet-connected business applications need to support million-level throughput, which has become a critical requirement. If we continue with the thread-per-request approach, the CPU needs to actively run a large number of threads to handle requests. But cloud hosts typically provide 1-2 CPUs, or even less than 1 CPU. Therefore, asynchronous event loop-based, non-blocking web servers emerged, handling concurrency with fewer threads and scaling with fewer hardware resources.

I've always wanted to try building Java web service applications using asynchronous web servers, but until now I haven't had the opportunity because at my workplace, we don't handle such high traffic.

So I did a quick experiment comparing the performance differences between thread-per-request engines and asynchronous web servers. I used Spring Web MVC and Spring WebFlux for a simple, quick comparison. Spring Web MVC is based on Servlet web servers, while WebFlux is based on asynchronous Netty.

Here are the framework versions, libraries, and tools I used for the comparison:

• JDK: 18.0.1 64-bit Temurin Eclipse Adoptium
• org.springframework.boot:spring-boot-starter-parent version: 2.7.2
• io.gatling:gatling-maven-plugin version: 4.2.4
• io.gatling.highcharts:gatling-charts-highcharts version: 3.8.3
• visualvm: 2.1.4

I ran the tests on a machine with an Intel 12th Gen Core i7 processor (20 CPU threads).

3. Project Structure

First, create a parent project to hold common library versions.

<project ....>
    <modelVersion>4.0.0</modelVersion>
    <packaging>pom</packaging>
    <modules>
        <module>web</module>
        <module>webflux</module>
        <module>gatling</module>
    </modules>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.2</version>
        <relativePath/>
    </parent>
    <groupId>org.example</groupId>
    <artifactId>spring-web-vs-webflux</artifactId>
    <version>1.0-SNAPSHOT</version>
    <properties>
        <maven.compiler.source>18</maven.compiler.source>
        <maven.compiler.target>18</maven.compiler.target>
        <java.version>18</java.version>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
    </dependencies>
</project>

Then we create a simple subproject for Spring Web MVC with a basic main class and an endpoint.

Next, we create another simple subproject for Spring WebFlux with similar structure.

WebFlux and Web MVC will both use their default application properties. As you can see, WebFlux provides the same annotation mechanism for controller routing without needing to implement Mono and the functional way of specifying routes.

We're not doing JSON processing here because I want to limit the comparison to HTTP engine processing itself.

Finally, we create a Gatling subproject for performance testing with a simulation class that injects 10,000 users, each calling the endpoint 100 times, resulting in 1,000,000 HTTP requests. We won't use any pauses, sleep times, or execution time limits, sending requests to the server as fast as possible.

4. Testing Section

The test execution results are shown below.

Spring Web MVC:

Spring WebFlux:

The Gatling test took slightly longer on WebFlux. Note that this doesn't mean WebFlux runs slower.

From the table above, Web Servlet can complete more requests (the OK column) because it has more threads (Spring defaults to 200 threads) to handle incoming requests. WebFlux can handle fewer requests. However, if you look at response times, overall WebFlux provides faster response times compared to Web MVC. This is because context switching in CPU threads is slower, while event loop programming is faster. Even with fewer threads, the gap between Web Servlet and WebFlux is only 5%. Additionally, all KOs are due to connection refused, meaning the queue is full. With more threads available to fetch requests from the queue, Web MVC has higher throughput. But when the CPU switches between different threads, this results in longer processing times to complete requests compared to asynchronous WebFlux. WebFlux has lower throughput because incoming requests are immediately rejected when the queue is full. Increasing the queue size might allow WebFlux to handle more throughput.

Startup times for both are roughly similar.

Spring Web MVC startup time:

Spring WebFlux startup time:

Below is data showing JVM utilization for Spring Web MVC:

Below is data showing JVM utilization for Spring WebFlux:

From the JVM utilization tables above, WebFlux has more stable CPU peaks compared to Spring Web MVC. WebFlux also uses less heap size compared to Spring Web MVC.

WebFlux also uses far fewer threads (20 CPU threads * 2 = 40) compared to Spring Web MVC (219 threads).

For Web Servlet MVC, many NIO threads are created to handle requests:

WebFlux uses a minimal number of threads for request processing:

5. Repeated Tests

I repeated the tests with the same specifications as the first test. Then for the third time, I reduced the Web Servlet MVC thread count to roughly the same as WebFlux. Here's a summary of the results:

Overall, WebFlux can handle almost the same throughput with fewer resources compared to Web Servlet. WebFlux response times are also faster than Web Servlet. With larger queue configuration for WebFlux, it might be possible to reduce the number of rejected connections.

The third test shows that if we have a machine with a high CPU count (remember I'm using a 12th Gen Intel processor), fewer threads will provide better performance for Web Servlet. But it will also require more heap memory because more requests will be stored in the queue. But on cloud hosts, CPUs are very limited. Therefore, for Web Servlet, fewer threads doesn't necessarily mean faster or better than WebFlux (asynchronous web server). Further testing is needed to verify this.

From the third test, the number of NIO threads was reduced:

6. Summary

In general, using an asynchronous web server doesn't necessarily mean faster or higher throughput than RPS/Servlet-based web servers. It means being able to handle roughly the same throughput with fewer resources.

TechEmpower has done better benchmarks, showing that other asynchronous web servers (like Vert.x) have better/higher throughput performance. For Spring WebFlux, although it's built on Netty (same as Vert.x), performance seems lower than what asynchronous web servers can typically achieve.

As you can see, Spring-Tomcat is faster than Spring WebFlux, although the gap isn't very large.

But asynchronous web servers have a problem. The programming style is no longer the imperative style Java was designed for. Debugging asynchronous code becomes difficult. Perhaps when Java Loom is officially adopted by many TPR-based web servers, we'll look at the benchmark results again.

7. Extended Section

The following content is no longer part of the original blog.

In the comparison results above, WebFlux seems to be at a disadvantage, not bringing better performance, and in TechEmpower tests, WebFlux ranks at 375. But I found some different perspectives on Reddit.

Original post: Why so much hate for Webflux?

Some highly upvoted answers: