Back to all use cases

ConsultingCross-Industry

Align Tomcat's timeouts with the load balancer in front of it before tuning anything else

Engineering Organization

A typical consulting engagement for Tomcat nodes that return intermittent 502 errors and hang under load: a review of connector timeouts, thread and connection pools and session handling, with a failover drill.

Tomcat consulting

Overview

A typical engagement starts with a business application on several Tomcat nodes behind a load balancer. Users see intermittent 502 errors and occasional hangs at peak, restarting a node clears it for a day, and nobody has measured what the request threads wait on.

Challenge

Tomcat's HTTP connector ships with defaults the team has never revisited: 200 request threads, an accept queue of 100, 8192 connections on the NIO connector, and a keep-alive timeout that follows connectionTimeout unless it is set. If Tomcat closes idle connections before the load balancer does, the balancer reuses a connection that is already gone and returns a 502. Separately, when a slow database call holds threads, a database pool sized below the thread count turns every spike into a queue.

Environment

Three Tomcat 9 nodes on virtual machines behind a load balancer, a relational database with a connection limit, session state held in memory on each node, and application and access logs kept for a few days.

Approach

AceMQ engineers read server.xml and the load balancer's idle timeout side by side, take thread dumps during a hang and a heap histogram before and after a redeploy, and compare the database pool size with the connection limit. Changes are tried on one node first, with the load balancer's error rate as the measure. The session options are compared against how users actually log in, and a failover drill checks that a node restart does not log everyone out.

Solution

  1. Timeouts compared: connectionTimeout and keepAliveTimeout on Tomcat against the load balancer's idle timeout, so Tomcat does not close first
  2. Thread dumps during a hang, read for what the request threads wait on
  3. Database pool size set against the thread count and the database's connection limit
  4. Forwarded-header handling and the AJP connector reviewed
  5. Session options compared: sticky sessions, replication and an external store
  6. Failover drill with clients connected and a runbook for node replacement

Outcome

The engagement delivers a written review with ranked changes, the tested connector and pool settings, a session recommendation, the failover drill result and a runbook. The team keeps the thread dump procedure so the next hang is read in minutes. Application code changes, if the dumps point there, stay with the customer's developers.

Technologies

Apache TomcatNGINXJava

Need Apache Tomcat Architecture Guidance?

AceMQ's senior Apache Tomcat engineers have handled this exact type of engagement before. Whether you need architectural guidance, hands-on remediation, or an ongoing managed partnership, we're ready to help.