How to Load Test a Node.js App

To load test a Node.js app, measure event loop lag alongside latency, because anything synchronous (big JSON, sync crypto, a slow regex) delays every request in the process at once. Before testing, set server.keepAliveTimeout higher than your load balancer's idle timeout (Node's default is 5 seconds; an AWS ALB's is 60), run one process per core or per container, and check whether UV_THREADPOOL_SIZE (default 4) is limiting bcrypt, file or DNS work. When latency climbs, profile under load with node --cpu-prof to find the function that blocks the loop.

How Node.js handles concurrent requests

A Node process runs your JavaScript on one thread. It handles thousands of concurrent connections by never waiting: when a request awaits a database query or HTTP call, the event loop moves on to other work and comes back when the result arrives. This works very well for I/O-bound APIs. It has one hard rule. While a piece of JavaScript is running, nothing else in that process runs.

Three consequences matter for load testing:

• One process uses at most about one CPU core for JavaScript. A 4-core VM running a single node server.js leaves three cores idle.
• CPU time per request sets the ceiling. If each request takes 2 ms of JavaScript, one process tops out near 500 RPS no matter how fast the database is.
• Blocking hurts everyone. A 200 ms JSON.parse of a large payload adds 200 ms to every request queued behind it.

What usually breaks first

Event loop blocking

Typical culprits found under load: bcrypt.hashSync or crypto.pbkdf2Sync in a login route, JSON.stringify of a 5 MB response, a regex with catastrophic backtracking on user input, fs.readFileSync in a request handler, synchronous template compilation, or a large Array.sort. At low traffic they cost a few milliseconds and nobody notices. Under load, they turn into queueing, and p99 rises faster than p50.

Keep-alive mismatch with the load balancer

Node's server.keepAliveTimeout defaults to 5,000 ms. AWS ALB's idle timeout defaults to 60 seconds, and many other proxies hold idle upstream connections for longer than 5 s too. Node closes an idle connection after 5 s, the balancer reuses it a moment later, and that request fails. You see a small, steady rate of 502s that grows with traffic, plus connection reset by peer in proxy logs. It looks like a capacity problem. It is a config problem.

The libuv threadpool

Some Node APIs have no non-blocking OS interface, so libuv runs them on a threadpool: file system calls, dns.lookup (used by default for hostname resolution in http.request), crypto.pbkdf2, scrypt, randomBytes, async zlib, and many native addons such as bcrypt. The Node docs give the default size as 4. Four concurrent bcrypt hashes at cost 12 can take well over 100 ms each; the fifth login waits for a free thread. Login p95 climbs while CPU looks underused.

Pool limits on the database client

pg.Pool defaults to max: 10 connections per process. Ten processes × 10 = 100 connections, the PostgreSQL default max_connections. When the pool is busy, queries queue in JavaScript, and that queue time is invisible in database metrics. See database connection pool exhausted.

File descriptors

Each open socket is a file descriptor. A Node process holding 10,000 keep-alive connections plus upstream sockets can hit a 1,024 soft limit and log EMFILE: too many open files. See too many open files.

Runtime settings to check

HTTP server timeouts

const server = app.listen(3000);
// Must be longer than the load balancer's idle timeout (ALB default: 60 s)
server.keepAliveTimeout = 65_000;
// Keep headersTimeout above keepAliveTimeout
server.headersTimeout = 66_000;
// requestTimeout defaults to 300_000 ms (5 min); lower it for APIs
server.requestTimeout = 30_000;

In Fastify, pass keepAliveTimeout and requestTimeout in the factory options. Behind nginx with upstream keepalive, make nginx's keepalive_timeout for the upstream shorter than Node's, so nginx always closes first.

Processes: cluster, PM2 or containers

# VM with 4 cores: one process per core
pm2 start dist/server.js -i max

# Kubernetes: one process per container, ~1 CPU each, scale replicas
resources:
  requests: { cpu: "1", memory: "512Mi" }
  limits:   { memory: "512Mi" }

Do not run PM2 cluster mode with -i max inside a container limited to 1 CPU. Node sees the host's cores, starts many processes, and they compete for the same CPU quota.

Threadpool and heap

UV_THREADPOOL_SIZE=16 node --max-old-space-size=384 dist/server.js

Set UV_THREADPOOL_SIZE in the environment before the process starts; the docs note that setting it from inside the process is not guaranteed to work. Set --max-old-space-size below the container memory limit, so V8 collects garbage before the kernel kills the process.

Node.js tuning table

Measuring event loop lag and profiling under load

Node ships a histogram for event loop delay. Export it to your metrics system or log it every few seconds during the test:

import { monitorEventLoopDelay } from 'node:perf_hooks';

const h = monitorEventLoopDelay({ resolution: 10 });
h.enable();
setInterval(() => {
  console.log(JSON.stringify({
    loop_p50_ms: h.percentile(50) / 1e6,
    loop_p99_ms: h.percentile(99) / 1e6,
    loop_max_ms: h.max / 1e6,
  }));
  h.reset();
}, 5000).unref();

Read it next to the load test's latency chart. If loop p99 tracks request p99, the time is spent in JavaScript on this process. If loop lag stays low while request latency climbs, the wait is outside the loop: database pool, threadpool, or a downstream service.

To find the blocking function, run one instance with the V8 CPU profiler while the load test is at the knee:

node --cpu-prof --cpu-prof-dir=/tmp/prof dist/server.js
# run the test, then stop the process cleanly (SIGINT)
# open the .cpuprofile in Chrome DevTools (Performance tab) or speedscope

Look for wide frames of your own code or of JSON.stringify, regex execution, or template rendering. Clinic.js was a popular wrapper for this, but its README now says it is not actively maintained and may give inaccurate results on current Node versions, so the built-in profiler is the safer default. 0x and --heap-prof are also useful for flame graphs and allocation profiles.

Express vs Fastify under load

Framework overhead matters most for small, fast, CPU-bound responses. Fastify compiles JSON schemas for serialization and has a fast router, so it does less work per request than Express. If your handler takes 1 ms of JavaScript, that difference shows. If it waits 40 ms on PostgreSQL, it barely registers. Express 5 is current and fine for most APIs. Profile before you switch, and if you do switch, run the same test against both builds at the same RPS.

What to put in the test scenario

• Endpoints with large payloads (big lists, exports), since serialization is where blocking hides.
• Login and signup, to exercise password hashing and the threadpool. Then reuse tokens for the rest of the session; see authentication load testing.
• Keep-alive on, through the real load balancer, with test duration well past the idle timeout, so the keep-alive mismatch can show up.
• Server-rendered routes if you use SSR; Next.js load testing covers those specifics.
• CSRF or session cookies if your app uses csrf-csrf, express-session or similar: fetch the token and cookie per virtual user before POSTs.

Worked example: per-process ceiling

An illustrative API endpoint spends 1.5 ms of JavaScript per request (routing, validation, serialization) and 30 ms waiting on PostgreSQL. One process can therefore handle at most about 1 ÷ 0.0015 ≈ 660 RPS before the event loop is busy 100% of the time. In practice, plan for 60 to 70% of that, about 400 to 450 RPS, because loop lag and tail latency rise steeply as the loop approaches full.

The 30 ms database wait does not count against that ceiling. It counts against the pool: by Little's Law, 450 RPS × 0.03 s = 13.5 connections in use on average, so pg.Pool's default of 10 is already too small and requests would queue inside the app. Raise the pool to 20 per process, check the cluster total against the database limit, and the load test should show loop lag as the first signal to rise.

Pre-test checklist

1. NODE_ENV=production; Express and many libraries skip debug work only when it is set.
2. keepAliveTimeout and headersTimeout set above the load balancer's idle timeout.
3. One process per core on VMs, or one per container with a 1-CPU request on Kubernetes.
4. UV_THREADPOOL_SIZE reviewed if you hash passwords, compress or read files per request.
5. Event loop delay exported as a metric; DB pool size and wait time logged.
6. ulimit -n raised on hosts and load balancers.
7. --max-old-space-size below the container memory limit.
8. One instance ready to run with --cpu-prof for the run at the knee.

A good first run is an exact-RPS ramp with a p95 threshold; LoadTester's RPS mode with auto-stop does this. Watch for the point where loop lag starts rising, because that is your per-process CPU ceiling. Multiply it by process count and compare with your peak traffic.