Tomcat Forgives a Blocking Call. My Event Loop Didn't.

A 5 millisecond blocking call. Nobody would call that slow. It passes code review, it looks fine in a trace, and if I saw it in a dashboard I'd move on.

Put it on a Tomcat worker thread and the service does 17,189 requests/sec while an unrelated endpoint answers in 0 ms.

Put the identical call on a Netty event loop and the service does 549 requests/sec while that same unrelated endpoint climbs to 217 ms.

Same call. Same machine. Same JVM. The only difference is which thread it's allowed to occupy.

Why the two models fail differently

Tomcat runs a pool of worker threads — 200 by default. A request gets a thread for its whole lifetime, and if it blocks, that thread parks. I've spent one worker out of 200. The other 199 keep serving. Blocking here is wasteful, but it's paid for out of a large budget, and the budget is the point.

Netty runs a handful of event loops — typically 2× the core count. Each connection is pinned to one loop at accept time and stays there. One loop serves many connections by never waiting: it reads what's ready, hands it to a handler, and moves on.

Block on that loop and it stops moving on. Every connection assigned to it waits — not just the one being served. Not requests to the same endpoint: every connection on that loop, whatever it was asking for.

That's the part that surprises people. It isn't that the slow endpoint gets slower. It's that endpoints with nothing to do with it get slower, and the hunt for the bug starts in the wrong service.

The experiment

Real Netty 4.1.115 and real embedded Tomcat 10.1.34, two servers with identical endpoints:

• /slow — sleeps for the block duration, standing in for a JDBC call someone left in a reactive handler
• /fast — returns a constant, never blocks, shares no state with /slow
• /async — does 5 ms of properly non-blocking work, for the experiment further down

A load generator hammers /slow at a given concurrency. Meanwhile a single prober calls /fast every 50 ms and records latency. /fast is the innocent bystander — it stands for every other endpoint in the service.

Netty gets 4 event loops. Tomcat gets 200 threads. Both numbers are explicit so the arithmetic is checkable.

protected void channelRead0(ChannelHandlerContext ctx, FullHttpRequest req) {
    if (req.uri().startsWith("/slow")) {
        // The whole point: this runs ON the event loop thread.
        Thread.sleep(blockMs);
    }
    respond(ctx, "...");
}

What happens to the bystander

With a 100 ms blocking call:

Netty's /fast is unharmed at concurrency 2, because only 2 of the 4 loops were busy and the prober's connection happened to sit on a free one. At concurrency 4 its p95 jumps to 99 ms. At 64 it's 3,095 ms, and the prober completed just 3 requests in 8 seconds.

Tomcat sits between 1 and 3 ms across that whole range and doesn't move.

Now the number that explains everything. Netty's /slow throughput:

slowConc=4    37 req/s
slowConc=16   37 req/s
slowConc=64   37 req/s

Flat. Adding callers adds no throughput at all — it only adds queue.

Because the ceiling isn't set by load, it's set by arithmetic:

max throughput = workers ÷ block duration

Four loops ÷ 0.1 s = 40 req/s. Measured 37. Tomcat: 200 threads ÷ 0.1 s = 2,000 req/s; measured 1,880.

Same formula, same physics. The only variable is how many workers there are — and 4 is a very different number from 200.

Writing the endpoint correctly does not save it

The obvious objection to everything above is that /fast returns a constant. Maybe it's just too trivial to mean anything.

So here is /async. It takes 5 ms to answer, and it does so the way the reactive model intends — the latency is scheduled, not waited on, and the event loop is free for every microsecond of it:

if (uri.startsWith("/async")) {
    // Textbook reactive: 5ms of latency, zero blocking. The loop is
    // free the entire time. This endpoint does everything right.
    ctx.executor().schedule(() -> respond(ctx, "async\n"), 5, TimeUnit.MILLISECONDS);
    return;
}

There is nothing to fix in that handler. It shares no state with /slow, touches no lock, and never occupies a thread while waiting.

Then I opened 12 independent connections to it — each its own TCP connection, so each gets its own event-loop assignment — and blocked a different number of loops with /slow while measuring every connection separately:

Read the Netty rows.

With one loop blocked out of four, exactly 3 of 12 connections stall. With two blocked, exactly 6 of 12. With all four, all twelve. The damage is quantised by loop count, and the fraction is precisely the fraction of loops I occupied.

The stalled ones aren't slightly slow. They sit at 150–225 ms — a 30× regression on a 7 ms endpoint that is doing everything right.

And the healthy ones stay at 7 ms. That is the genuinely nasty part. The correctly-written endpoint is not degraded; it is fine for most callers and broken for a quarter of them, decided by nothing more than which loop happened to accept their connection. A retry may land on a healthy loop and come back in 7 ms. The p50 dashboard looks fine. A slice of callers is having a completely different experience from the rest, and nothing in the code of the endpoint they're calling has anything wrong with it.

Tomcat, at every one of those levels: all 12 connections, 8–10 ms, unmoved.

That is what "unforgiving" actually means here. It isn't that the event loop punishes bad code — bad code gets punished everywhere. It's that the event loop lets one careless handler impose its cost on code that did nothing wrong, in another route, on another connection, serving another customer. Quality is not a defence, because the resource being exhausted was never that endpoint's to protect.

What this looks like in a trace

Had I seen only what an APM tool shows, I would have gone looking in the wrong place. So I instrumented both sides.

Each server reports two timestamps: when the handler actually got the CPU, and when its scheduled continuation resumed. The client records when it sent the request and when the response landed. Both processes sit on one machine, so it is one clock, and the spans can be laid out on a shared timeline exactly as a trace viewer would draw them — one row per span, each positioned where it actually started.

The numbers behind it:

NETTY  /async  (5ms of real work, 8 blocking callers)
  req     total   accepted   continuation       work   return
  0       306ms      200ms          104ms        0ms      2ms
  1       266ms      155ms          108ms        0ms      3ms
  2       252ms      139ms          110ms        0ms      3ms
  3       259ms      146ms          109ms        0ms      4ms

TOMCAT /async  (5ms of real work, 8 blocking callers)
  0         9ms        1ms            7ms        0ms      1ms
  1         9ms        0ms            8ms        0ms      1ms
  2        11ms        2ms            8ms        0ms      1ms

Look at what the trace contains. Four server spans, every one of them 0 ms, scattered across a request that took 252 ms. Adding up every span in the whole trace gives 3 ms, all of it the network hop home.

The endpoint is not slow — it was never slow, there is nothing in it to make faster — and yet the caller waited a quarter of a second. The time isn't inside any span. It's in the empty stretches between the rows.

Two separate gaps produce that, and neither is visible from inside the server.

The first, accepted, is ~140 ms between the client writing the request and the handler getting the CPU. During that window the bytes are sitting in the kernel's socket buffer. The server cannot time this, because timing it would require running code, and running code is precisely what's unavailable. There is no span for it. It's not that the tracer chose not to instrument it — there was no thread in which an instrument could fire.

The second, continuation, is the sneakier one. The handler asked for a 5 ms delay. It got 108 ms. schedule() puts a task on the event loop's queue, and the loop is busy doing Thread.sleep for someone else, so the timer fires when the loop gets round to it. The asynchronous machinery I was relying on is late for the same reason everything else is.

This is why the tool tells the wrong story. Every span it can show is honest and fast. The waterfall has two large stretches where no bar is drawn at all, and a trace viewer renders that as blank space — the same blank space that would appear if the client had simply been slow to send. Reasonable engineers conclude it's the network, or GC, or the caller, instrument those, and find nothing, because there is nothing there.

Adding more instrumentation doesn't help either, which is the part that stings. A span cannot be placed around waiting for a thread to become available, because a span needs code to run at both ends and no code runs at either.

The Tomcat rows are what the code predicts: 1 ms to be picked up, 7 ms for a 5 ms scheduled delay, 0 ms of work. When the shape of a trace matches the shape of the code, the trace can be trusted.

The practical tell: when total time greatly exceeds the sum of the spans, suspect the scheduler, not the work. Unexplained gaps at the front of a request are queueing. On an event loop, queueing means somebody is blocking on it.

Which is why "it's only 5ms" is no defence

That formula has no notion of "small". Drop the block to 5 ms:

549 req/s on the event loop. 17,189 on Tomcat. Same call, 31× apart.

Four loops ÷ 0.005 s = 800 req/s, and we measured 549 once framing overhead is accounted for. A 5 ms call — genuinely fast, nothing I would think to optimise — caps the entire service at a few hundred requests per second. Cross that ceiling and requests queue, latency grows without bound, and /fast goes to 217 ms.

It also explains why database slowness cascades rather than degrades. The ceiling is inversely proportional to block duration. A database that slips from 5 ms to 50 ms — a bad afternoon, not an outage — drops the ceiling from 800 to 80 req/s. Traffic didn't change. Capacity fell by 10× in a single step, and every endpoint sharing those loops goes down with it.

On Tomcat the same slowdown goes from 40,000 to 4,000 req/s. Also a 10× drop, but from a height that likely stays above real traffic, and nobody gets paged.

Tomcat is not immune, it just has a bigger budget

Worth being straight about, because "use Tomcat and blocking is fine" is the wrong lesson. Push past the pool:

slowConc=128   /slow  1192 req/s   |   /fast p50   0ms  p95  1ms
slowConc=200   /slow  1859 req/s   |   /fast p50   0ms  p95 45ms
slowConc=300   /slow  1880 req/s   |   /fast p50  47ms  p95 57ms

At 300 concurrent blocking callers against 200 threads, /fast degrades too. Same failure, same mechanism — all workers occupied, everything else queues.

The difference is purely where the cliff sits. Tomcat's is at 200 concurrent blocking requests. Netty's is at 4. One of those numbers might never be reached in production. The other is reached on a quiet Tuesday.

And note Tomcat's degradation at 300 is 47 ms, against Netty's 3,095 ms at 64. Even past the cliff, a pool of 200 fails more gently than a pool of 4.

The fix is one line, and it's measurable

Netty isn't the problem. Blocking on the loop is. Move the same call to a separate executor and let the loop go back to work:

if (uri.startsWith("/offload")) {
    OFFLOAD.execute(() -> {                       // a normal thread pool
        sleep();                                  // the blocking call, off the loop
        ctx.executor().execute(() -> respond(ctx, "offload\n"));
    });
    return;
}

Same server, same 5 ms call, 128 concurrent callers:

netty, blocking on the event loop      549 req/s   /fast p50 217ms
netty, same call offloaded          18,311 req/s   /fast p50   0ms

33× throughput, and the bystander is healthy again. Offloaded Netty slightly beats Tomcat here, which is the honest summary of the whole comparison: the event loop model is not worse, it's less forgiving. It gives no slack, and in exchange it asks that nothing ever blocks.

In Spring WebFlux this is subscribeOn(Schedulers.boundedElastic()). In plain Netty it's any executor that isn't the loop. The mechanism is the same: get the waiting off the thread whose job is to never wait.

Finding it before production does

The reason this bites so often is that nothing announces it. No exception, no warning, no log line. Throughput is just lower than it should be, and the latency graph that looks wrong belongs to a different endpoint than the one at fault.

Two things I do:

BlockHound in the test suite. It instruments the JVM to throw when a blocking call happens on a non-blocking thread. It is the single highest-value tool here, because it turns a silent capacity ceiling into a failing test. It usually finds something on day one — and usually in a library rather than in code I wrote.

The ceiling, worked out in advance. Event loop count divided by worst-case call duration, compared against real peak traffic. If the two numbers land anywhere near each other, there is no performance problem yet — there is one scheduled.

The usual suspects live in code I didn't write: a JDBC driver behind a reactive facade, an SDK that does a synchronous token refresh, a logging appender writing to a slow disk, a metrics client doing a blocking flush.

Caveats

Localhost, one 8-core laptop, Thread.sleep standing in for I/O, and a load generator sharing the machine with the servers. The absolute numbers are worth nothing to anyone else, and the ceilings are optimistic — real network I/O costs more.

What transfers is the arithmetic: throughput is bounded by workers ÷ block duration, and an event loop pool is small by design. Everything else in this post follows from those two facts.

The takeaway isn't that reactive is dangerous or that servlets are outdated. It's that Tomcat's thread pool is, among other things, an accident budget — 200 chances to get it wrong before anyone notices. An event loop hands over four, and spends them on whichever connections happened to land there.