Load Testing Glossary

A load testing glossary defines the terms used to plan, run and read performance tests. This one covers 48 terms, from Apdex to warm-up, each in one to three sentences with a link to a deeper explanation where one exists. The pairs people confuse most often (VUs and RPS, throughput and latency, SLI, SLO and SLA) are compared in a table at the end.

Jump to: A · B · C · D · E · G · H · K · L · O · P · R · S · T · U · V · W · Commonly confused terms

A

Apdex

Application Performance Index: a 0 to 1 score computed as (satisfied + tolerating / 2) / total requests, for a chosen threshold T. Requests at or under T are satisfied, up to 4T are tolerating, and slower ones (or errors) are frustrated.

Arrival rate

The rate at which new requests or sessions start, set independently of how many are still in progress. It is the input of an open workload model and of RPS-mode load tests. See Open vs closed workload models.

B

Baseline

A reference test run, on a known build and environment, that later runs are compared against. Regressions are defined as statistically meaningful differences from the baseline at the same load. See Performance regression testing.

Bottleneck

The resource that limits throughput first, such as CPU, a database, a lock, a connection pool or network bandwidth. When it saturates, throughput plateaus and latency rises everywhere upstream of it. See Why latency increases under load.

C

Cache hit ratio

The share of lookups served from a cache rather than the origin: hits / (hits + misses). Load tests that repeat the same few URLs inflate it, so vary test data to keep the ratio close to production.

Capacity

The maximum throughput a system sustains while still meeting its SLOs. It is usually stated as RPS at a latency percentile and error rate, for example 1,400 RPS at p95 under 300 ms. See Capacity testing.

Closed workload model

A workload with a fixed population of users, where each user sends its next request only after the previous one completes (plus think time). Load falls automatically when the system slows down. See Open vs closed workload models.

Cold start

Extra latency on the first requests served by a new instance, such as a serverless function, container or JVM, while it loads code, initializes runtimes and opens connections. See Serverless load testing.

Concurrency

The number of requests or sessions in progress at the same moment. By Little's Law, average concurrency equals throughput multiplied by average time in the system. See Concurrent users calculator.

Connection pool

A set of reusable open connections (to a database, upstream service or HTTP host) shared by request handlers. When all connections are busy, requests queue for one, which shows up as latency rather than errors until a pool timeout fires.

Coordinated omission

A measurement error, named by Gil Tene, where a closed-loop load generator stops sending while the server is slow, so the slowest requests are never recorded and high percentiles are under-reported. See Coordinated omission.

D

Distributed load testing

Generating load from several machines or regions at once, either because one generator can't produce enough traffic or to reproduce geographically spread users. See Distributed load testing.

E

Error budget

The amount of unreliability an SLO allows: 1 minus the target. A 99.9% monthly availability SLO leaves a 0.1% budget, about 43 minutes of full outage in 30 days, or 1 failed request in 1,000. See Error budget calculator.

Error rate

Failed requests divided by total requests. Define failure explicitly before testing: usually 5xx responses, timeouts and connection errors, plus failed content checks; 4xx may or may not count.

G

Goodput

Throughput counting only useful work: successful responses, often restricted to those that also met the latency SLO. It drops when a system starts serving fast errors, even if raw throughput rises. See Throughput vs latency.

H

Headroom

The gap between measured capacity and expected peak load, often expressed as a percentage. A service with 2,000 RPS capacity and a 1,400 RPS peak has 30% headroom relative to capacity.

K

Keep-alive

HTTP persistent connections that are reused for multiple requests instead of opening a new TCP (and TLS) connection each time. HTTP/1.1 uses them by default; idle timeouts on clients, proxies and servers decide how long they last.

Knee

The point on a throughput-latency curve where latency starts rising sharply while throughput gains flatten. Running production below the knee keeps latency predictable. See Throughput vs latency.

L

Latency

The time one request takes from the client's point of view, from send to full response. Report it as a distribution (p50, p95, p99, max) rather than an average. See p95 vs p99 latency.

Little's Law

L = λ × W: average items in a stable system equal arrival rate times average time in the system. In load testing, VUs = RPS × (response time + think time). See Little's Law in load testing.

Load test

A test that applies an expected, realistic level of traffic to a system to verify it meets its performance targets. Stress, spike and soak tests are variations with different load shapes. See What is load testing.

O

Open workload model

A workload where new requests arrive at a set rate regardless of how many are still in progress, so slowness turns into a growing queue. It matches public internet traffic better than a closed model. See Open vs closed workload models.

P

Pacing

A fixed target duration for each iteration of a virtual user's loop, with the tool waiting out the remainder after the requests finish. Pacing keeps per-VU throughput constant as long as responses are faster than the pacing interval. See Think time and pacing calculator.

Percentile (p95, p99)

The value below which a given share of measurements falls: p95 latency of 300 ms means 95% of requests took 300 ms or less. High percentiles describe the slow experiences an average hides. See Latency percentile calculator.

Performance regression

A measurable slowdown or capacity loss between versions at the same load, such as p95 rising from 180 ms to 260 ms after a release. Detected by comparing runs against a baseline. See Performance regression testing.

R

Ramp-up

The period at the start of a test during which load increases gradually to the target level. It avoids an artificial thundering herd and lets autoscaling, caches and pools react; exclude it from steady-state results.

Rate limiting

A server or gateway policy that caps how many requests a client may make per time window, typically answering excess requests with HTTP 429. Load tests from a few IPs often hit limits real traffic would not. See Rate limit testing.

RED method

A service monitoring approach from Tom Wilkie: for every service, track Rate (requests per second), Errors (failed requests) and Duration (latency distribution). It maps directly onto load test output.

Response time

The total time from sending a request to receiving the complete response, as measured by the client. It includes network time, queueing and server processing; many tools use it interchangeably with latency.

RPS (requests per second)

The number of HTTP requests sent or completed per second, summed across all virtual users. It is the most common unit for throughput targets. See Virtual users vs requests per second.

S

Saturation

The state in which a resource has more work than it can handle, so work queues. Measured as queue length or wait time (run queue, pending connections), it is the S in the USE method.

Scalability

How well throughput grows when resources are added (more instances, cores or nodes). Linear scalability is rare; contention and coordination costs make each added node worth less. See Scalability testing.

SLA (service level agreement)

A contract with customers that specifies service levels and the consequences, such as credits, if they are missed. SLAs are usually set looser than the internal SLOs that protect them.

SLI (service level indicator)

A measured quantity that describes service quality, such as the proportion of requests that succeed or the proportion served in under 300 ms.

SLO (service level objective)

A target value for an SLI over a time window, for example 99.9% of requests succeed and 95% finish under 300 ms over 30 days. Load test thresholds should be derived from SLOs. See Error budget calculator.

Soak test

A long test, typically hours, at a steady realistic load, run to find problems that only appear over time: memory leaks, connection or file descriptor leaks, log growth and slow degradation. See Soak testing.

Spike test

A test that raises load suddenly, for example from 100 to 2,000 RPS within seconds, to see how the system and its autoscaling cope with a burst and how quickly it recovers. See Spike testing.

Steady state

A period during a test when load is constant and metrics have stabilized. Summary statistics and Little's Law checks should use steady-state windows only.

Stress test

A test that pushes load beyond expected capacity to find the breaking point and observe how the system fails and recovers. See Stress testing.

T

Tail latency

Latency at high percentiles such as p99 and p99.9. Dean and Barroso's "The Tail at Scale" (2013) shows why it matters: if a request fans out to 100 servers that are each slow 1% of the time, about 63% of requests hit at least one slow server. See p95 vs p99 latency.

Think time

The pause a virtual user takes between actions to imitate a real person reading or typing. It lowers RPS per VU without changing server work per request. See Think time and pacing calculator.

Throughput

Work completed per unit of time, usually RPS or transactions per second. It rises with load until a bottleneck saturates, then plateaus. See Throughput vs latency.

TPS (transactions per second)

Completed business transactions per second, where one transaction (such as a checkout) may include several HTTP requests. Always state whether a report counts transactions or requests.

TTFB (time to first byte)

The time from sending a request until the first byte of the response arrives. It captures network round trip plus server processing, but not the time to download the rest of the body; whether it includes DNS, TCP and TLS setup depends on the tool.

U

Universal Scalability Law

Neil Gunther's model of throughput versus concurrency, C(N) = N / (1 + α(N − 1) + βN(N − 1)), where α is contention and β is coherency cost. Any β greater than zero makes throughput peak and then fall. See Throughput vs latency.

USE method

A resource analysis checklist from Brendan Gregg: for every resource (CPU, memory, disk, network, pools), check Utilization, Saturation and Errors. Use it to find the bottleneck when throughput plateaus.

V

Virtual user (VU)

A simulated client in the load generator that runs a scenario in a loop, holding its own cookies, tokens and connections. VU count is not RPS; the conversion depends on response time and think time. See Virtual users vs requests per second.

W

Warm-up

An initial period of a test, or a separate short run, that lets JIT compilation, caches, connection pools and autoscaling settle before measurement starts. Results from the warm-up window are usually excluded.

Commonly confused terms

For tools that turn these definitions into numbers, see the RPS calculator and the concurrent users calculator. If you are new to the topic, start with what load testing is and then how to read load test results.