Backpressure in Client-Server Applications

2026-01-16T00:00:00+00:00

Backpressure, in client-server applications, is accomplished when a client adjusts its transmission of messages in response to a server that has slowed its processing of messages.

Stuck in Traffic</h2>
Ideally, when a networked client connects to a server, messages are sent at maximum speed and responses are likewise returned quickly. To handle more client traffic we scale horizontally by adding additional server instances and our long-tail latency metrics are stable over time. However, in most real world applications the server is not a monolith unto itself and will make internal requests to relational databases and external APIs with unpredictable performance characteristics.
Timeouts are essential for achieving resiliency. When a timeout is reached, the client disconnects and potentially tries again until it gives up. The absence of sensible timeouts results in resource consumption (e.g., network sockets, memory, threads) and conditions resembling a traffic jam. In a healthy system both the client and server will monitor connections and proactively timeout.
Constantly connecting and disconnecting, in the face of server instability, has a cost. On average, it takes longer to establish a network connection, and negotiate a secure socket, than to issue requests and responses. An alternative is for the client to respond to backpressure from the server by keeping the connection open and only sending data when the server is ready to accept it. Instead of a chaotic traffic jam, the stream of messages is like a cargo train, with slow-downs anticipated.

Backpressure</h2>
The Transmission Control Protocol (TCP) considers backpressure as part of the specification. “Understanding TCP Protocol and Backpressure</a> ”¹ does a nice job summarizing the key ideas.

TCP implements flow control to ensure that the sender does not overwhelm the receiver with data. It uses a sliding window mechanism to control the number of unacknowledged segments that can be sent at a time.</li>
The receiver advertises its window size, indicating the amount of data it can currently accept.</li>
The sender adjusts the rate of transmission based on the receiver’s window size, ensuring efficient data transfer.</li> </ul> </blockquote>
In practice, for backpressure to work effectively, both the client and the server should adhere to some specific design principles:

Client & Server

Connection should be long-lived (e.g., HTTP keep-alive, websocket, etc.)</li> </ul> </li>
Client

Networking libraries should indicate when the socket is writable</li>
Timeouts should be tuned to allow for backpressure</li> </ul> </li>
Server

Separate thread-pools should handle,

Accepting network connections</li>
Sending and receiving data over established connections</li> </ul> </li>
Slow or idle clients should be detected and pruned</li> </ul> </li> </ul>
Let us consider an asynchronous, event-driven application architecture. On the server a single thread can effectively handle the job of accepting network connections. Once connected, another thread, or pool of threads, can read requests and send responses. These "worker threads" must take care not block on I/O or mutexes. We read the request data, make asynchronous calls to internal services, and during those asynchronous calls our worker threads may service other requests. Once our asynchronous work is complete, and if the client is still connected, we write a response. This could be backpressure at work, but how can we know for certain? Unfortunately, most of us use software frameworks that make it difficult to know what is going on beneath the surface.
An Experiment</h1>
For this article we created a project at https://github.com/travishaagen/blog-backpressure</a>. It's only dependency is Netty</a>, which is an event-driven abstraction over Java's internal sockets and byte buffers. To prove to ourselves that backpressure is occurring, the application establishes a single websocket connection. The client has a Netty event-loop with a single thread. Likewise, the server has a single-threaded connection acceptor and a single threaded worker-pool, which attempts to write to a bounded queue of size 1. A consumer thread reads from the bounded queue at a slow rate. When the queue is full, we cannot write to it, so we stop reading data. When we stop reading, the client is informed that the server cannot accept more data, which causes the client to stop writing. This is backpressure.
Every network connection establishes a Channel</a>. A `Channel</code> has a method called isWritable()</code> with the following documentation,`
`Returns true</code> if and only if the I/O thread will perform the requested write operation immediately. Any write requests made when this method returns false</code> are queued until the I/O thread is ready to process the queued write requests.`WriteBufferWaterMark</a> can be used to configure on which condition the write buffer would cause this channel to change writability. </blockquote> In our WebSocketClient</code> you can see that, in a loop, we increment a 64-bit integer and only write it to the channel when writable. The client is detecting and adapting to a full write buffer. while (!group.isShuttingDown()) { // will not write, when the write buffer watermark is full if (ch.isWritable()) { var buf = allocator.buffer(Long.BYTES, Long.BYTES); buf.writeLong(writeCounter++); var frame = new BinaryWebSocketFrame(buf); ch.writeAndFlush(frame); } else { // we're in a spin-loop, so yield to other threads Thread.yield(); } } </code></pre> On the server, to control our experiment, we create a bounded queue of size 1. var consumerQueue = new LinkedBlockingQueue<Long>(1); </code></pre> We then pass the queue to a "consumer thread" which reads from it at a slow pace, to simulate a backlog of work. while (true) { var value = consumerQueue.take(); // simulate consuming the queue more slowly than the incoming event rate Thread.sleep(0, 250_000); } </code></pre> We also pass the queue to our WebSocketFrameHandler</code>. It reads from the Channel</code> and will spin in a loop until it can write to the queue. It does not perform any additional reads until the data is written, which causes backpressure. var msg = frame.content().readLong(); if (!consumerQueue.offer(msg)) { while (!consumerQueue.offer(msg)) { // yield to other threads while we try to write to the bounded queue Thread.yield(); } } </code></pre> When we run the application we'll see an output similar to the following. The Wrote</code> lines show the number of messages written to the socket and Consumed</code> shows the count of messages read by the server's "consumer thread". Note that, to save space, only the last consecutive Wrote</code>/Consumed</code> lines are shown below. % ./gradlew runApp > Task :runApp Consumed: 0 Wrote: 100000 Consumed: 60000 Wrote: 120000 Consumed: 85000 Wrote: 150000 Consumed: 105000 Wrote: 170000 Consumed: 125000 Wrote: 190000 </code></pre> When we observe the loopback</code> interface in Wireshark</a> and set the filter to tcp.window_size == 0</code> we see that the TCP window drops to zero multiple times. This is what causes the client to stop writing. Final Thoughts</h2> Backpressure is an elegant concept in computer networking. As the above experiment shows, both the client and server need to actively control reading and writing socket data for it to work properly. Websockets seem particularly well suited to applications that seek to control backpressure, because it's easy to conceive of inbound messages feeding into one queue and outbound messages into another. We hope that this discussion will assist developers in thinking about backpressure and how well their libraries and frameworks support it. For further reading we suggest “Applying Back Pressure When Overloaded</a> ”² by Martin Thompson. References</h2> Kumar, A. (2023, August 6). Understanding TCP Protocol and Backpressure. Sum Of Bytes. https://sumofbytes.com/blog/understanding-tcp-protocol-and-backpressure/</a></li> Thompson, M. (2012, May 19). Applying Back Pressure When Overloaded. Mechanical Sympathy. https://mechanical-sympathy.blogspot.com/2012/05/apply-back-pressure-when-overloaded.html</a></li> </ol>

Static Exceptions for Flow Control

2026-01-02T00:00:00+00:00

Static Exceptions, when used for flow control, can dramatically reduce garbage generation and latency in Java and Kotlin applications.

TL;DR</h2>
To create high performance exceptions for flow control in the JVM, simply extend the following `StaticException</code> class. Then store the Exception</code> instance in a static final</code> field and throw it as needed. They're fast, because the call stack doesn't need to be traversed and since we're creating reusable objects, there's less for the garbage collector to do.`
`Java:`
public abstract class StaticException extends RuntimeException { private static final StackTraceElement[] emptyStackTrace = new StackTraceElement[0]; public StaticException(String message) { super(message, null, false, true); setStackTrace(emptyStackTrace); } @Override public synchronized Throwable fillInStackTrace() { return this; } } </code></pre> Kotlin:
abstract class StaticException( message: String, ) : RuntimeException(message, null, false, true) { companion object { private val emptyStackTrace = arrayOf<StackTraceElement>() } init { stackTrace = emptyStackTrace } override fun fillInStackTrace(): Throwable { return this } } </code></pre>
Example Use Case</h2>
In a web application, conditions that trigger HTTP error responses can happen anywhere in the call chain. For example, the client's bearer token has expired, so we return `401 Unauthorized</code>. A request is malformed, so validation logic returns 400 Bad Request</code>. Likewise, deeper in the call chain, an asynchronous API call times out, so we return 503 Service Unavailable</code>. None of these error conditions are particularly unexpected, and aside from the timeout, may not even warrant logging.`
`For JVM applications it can be convenient to throw an exception that signifies "respond with an error". When the exception is caught, we build the appropriate HTTP response. An example response exception might be,`
public class ResponseException extends RuntimeException { public final int statusCode; public ResponseException(String message, int statusCode) { super(message); this.statusCode = statusCode; } } </code></pre>
Which could be caught in our web controller logic, or a framework mechanism such as a Spring @ExceptionHandler</a>, where the appropriate HTTP response status header and payload body would be constructed,
`{ "code": 400, "message": "Bad Request" } </code></pre>`
Discussion</h2>
We're using exceptions for flow control. Unfortunately, from a performance perspective, exceptions are expensive. The bulk of the cost comes from `Throwable.fillInStackTrace()</code> which traverses the``call stack</a>, and is exacerbated by massively long call stacks created by libraries and frameworks that we use to make life easier.`
`Fortunately, there is a solution. We can preinitialize exceptions and either generate the call stack once, or provide an empty stack. I first learned about this technique from Norman Maurer's blog post entitled “The hidden performance costs of instantiating Throwables</a>”¹. Norman is one of the creators of Netty</a> which is an abstraction over Java's internal sockets and byte buffers. It's a technique that has been used sparingly in Netty itself.`
`However, if you do it wrong, it can lead to problems. Let's look at this constructor for Throwable</code>,`
protected Throwable( String message, Throwable cause, boolean enableSuppression, boolean writableStackTrace ) { if (writableStackTrace) { fillInStackTrace(); } else { stackTrace = null; } detailMessage = message; this.cause = cause; if (!enableSuppression) { suppressedExceptions = null; } if (jfrTracing) { ThrowableTracer.traceThrowable(getClass(), message); } } </code></pre> It's most essential that we set enableSuppression</code> to false</code>. Otherwise, our own code or magical side effects from libraries could invoke the addSuppressed</code> method, and potentially add objects to the suppressedExceptions</code> collection every time we throw the reusable exception. The collection would grow infinitely. public final synchronized void addSuppressed(Throwable exception) { if (exception == this) throw new IllegalArgumentException(SELF_SUPPRESSION_MESSAGE, exception); Objects.requireNonNull(exception, NULL_CAUSE_MESSAGE); if (suppressedExceptions == null) // Suppressed exceptions not recorded return; if (suppressedExceptions == SUPPRESSED_SENTINEL) suppressedExceptions = new ArrayList<>(1); suppressedExceptions.add(exception); } </code></pre> This happened to me (:faceplant:</code>). It turns out that Project Reactor</a> invokes addSuppressed</code> in a class called FluxOnAssembly</a> every time an Exception</code> passes through. Benchmarks</h2> Norman included some Java Microbenchmark Harness (JMH)</a> results in the beforementioned blog post</a>. I also ran across “Why Consuming Stack Traces is Noticeably Slower in Java 11 Compared to Java 8: JMH Benchmark Results.</a>”² It makes some interesting points and I didn't realize that Java 11 introduced a change that was meant to improve average performance by lazily traversing the call stack for logging. Because of this lazy traversal, I included some additional *AndGetStacktrace</code> JMH benchmarks below. Benchmark Mode Cnt Score Error Units staticException thrpt 5 1799330562.898 ± 3931377.639 ops/s staticExceptionAndGetStacktrace thrpt 5 105525288.622 ± 404717.391 ops/s newException thrpt 5 1529637.197 ± 2700.762 ops/s newExceptionAndGetStacktrace thrpt 5 328081.037 ± 1212.503 ops/s </code></pre> To summarize, Throwing and catching a reusable StaticException</code> reached 1,799,330,563 operations/second</li> Reusable StaticException</code> followed by a call to e.getStackTrace()</code> was 105,525,289 operations/second</li> Throwing a new RuntimeException</code> every time was 1,529,637 operations/second</li> And RuntimeException</code> with e.getStackTrace()</code> was 328,081 operations/second</li> </ol> Comparing the flow control static</code> use case #1 with new</code> #3 shows 1000x better performance for StaticException</code>. Both the static</code> and new</code> cases are seriously hampered whenever you choose to log the stacktrace. The result for staticExceptionAndGetStacktrace</code> is surprising, because the only additional operation is calling clone()</code> on our empty StackTraceElement[]</code> array. Long call stacks were not simulated with the benchmarks either. Above benchmark code available at https://github.com/travishaagen/blog-static-exceptions-for-flow-control</a> Final Thoughts</h2> Throwing exceptions has a cost in the JVM. For optimal performance, exceptions should be thrown sparingly. When a failure condition is not unexpected, your application's latency and throughput can benefit from static exceptions. References</h2> Maurer, N. (2013, November 9). The hidden performance costs of instantiating Throwables. The Thoughts of Norman Maurer. http://normanmaurer.me/blog/2013/11/09/The-hidden-performance-costs-of-instantiating-Throwables/</a></li> Why Consuming Stack Traces is Noticeably Slower in Java 11 Compared to Java 8: JMH Benchmark Results. (2025, November 26). javaspring.net. https://www.javaspring.net/blog/consuming-stack-traces-noticeably-slower-in-java-11-than-java-8/</a></li> </ol>

Travis Haagen's Blog

Backpressure in Client-Server Applications

Static Exceptions for Flow Control