Open vs Closed Workload Models

In a closed workload model a fixed population of users loops through the system, and each user sends a new request only after the previous one completes, so load falls when the system slows down. In an open workload model new requests arrive at a set rate regardless of how many are still in progress, so a slow system accumulates a queue. Public web and API traffic behaves much more like the open model, which is why arrival-rate (RPS) load tests are usually the right default.

The two models side by side

Why the choice changes your results

Take a service with a capacity of 1,000 RPS. In a closed test with 100 VUs and no think time, the generator can never put more than 100 requests in flight. If a GC pause or lock contention doubles response time, the VUs simply send half as many requests. The test reports higher latency and lower throughput, but nothing falls over.

Run the same service in an open test at 900 RPS. At 90% utilization, a modest slowdown pushes effective capacity below 900 RPS. Requests now arrive faster than they leave, so the queue grows every second the slowdown lasts. Latency climbs for every request behind the stall, connection pools fill up, upstream timeouts fire, and you see 504 gateway timeouts. That is what production does under a traffic spike, and it is the failure you are trying to find.

The closed test is not wrong; it is answering a different question: "how does the system behave with at most 100 concurrent clients?" The trouble starts when its results are used to answer "can we handle 900 RPS?"

What "Open Versus Closed: A Cautionary Tale" found

The standard reference is Open Versus Closed: A Cautionary Tale by Bianca Schroeder, Adam Wierman and Mor Harchol-Balter, presented at USENIX NSDI 2006. At a high level, the paper makes these points:

• Open and closed systems can behave very differently even at the same load level, so the choice of model in a workload generator materially changes the measured response times.
• At high load, open systems show much higher mean response times than closed ones, and they are far more sensitive to variability in request service times.
• Scheduling and prioritization policies (for example, favoring short requests) have a much larger effect in open systems than in closed ones, so conclusions drawn from a closed benchmark may not carry over.
• Many real workloads fit a partly-open model: users arrive from outside (open), make a session of several requests with think time (closed-like), then leave. How "open" the system behaves depends largely on how many requests a typical session contains.

The practical lesson for load testers: choose the model deliberately, based on how your real clients arrive, rather than accepting whatever your tool does by default. Read the paper itself for the analysis and the specific experiments.

Which model matches real internet traffic

Traffic to a public website or API comes from a very large population of independent clients. A visitor in Berlin doesn't wait for a visitor in Austin to finish before clicking. When your server slows down, the arrival rate barely changes; at most, a few people give up or retry (retries make it worse). That is open-model behavior.

Closed or near-closed traffic exists too:

• A fixed pool of internal workers or cron jobs pulling from a queue.
• A partner or upstream service with a fixed client connection pool (say, 20 connections and no request queue).
• Long-lived sessions where a known set of clients send requests back to back, such as a trading terminal or a call-center app.

A good compromise for websites is the partly-open approach: open arrival of sessions (e.g., 20 new visitors per second), with each session running a short closed script of 3-8 requests with think time. Gatling's open injection and k6's arrival-rate executors both work this way, because the rate they control is the rate of new iterations or new virtual users.

How k6, Gatling, JMeter and Locust implement each model

k6

export const options = {
  scenarios: {
    closed: { executor: 'constant-vus', vus: 100, duration: '5m' },
    open: {
      executor: 'constant-arrival-rate',
      rate: 500, timeUnit: '1s', duration: '5m',
      preAllocatedVUs: 100, maxVUs: 1000,
      startTime: '5m',
    },
  },
};

In the arrival-rate executor, k6 starts iterations on a schedule and borrows an idle VU for each one. If none is free and maxVUs is reached, the iteration is counted in dropped_iterations instead of being delayed. Watch that metric: a non-zero count means the test silently stopped being open.

Gatling

// Java DSL
setUp(
  browse.injectOpen(rampUsersPerSec(5).to(50).during(120), constantUsersPerSec(50).during(600)),
  workers.injectClosed(constantConcurrentUsers(20).during(720))
).protocols(httpProtocol);

Gatling doesn't let you mix open and closed injection steps within one population's injection profile. Each open "user" runs the scenario once and exits, so constantUsersPerSec(50) means 50 new sessions per second, a partly-open model when the scenario has several requests.

JMeter

A standard Thread Group is closed: N threads loop, each waiting for its sampler to finish. The Constant Throughput Timer and Precise Throughput Timer delay threads so they don't exceed a target rate, but they cannot create more concurrency than the thread count, so once response times grow the achieved rate falls below the target. For a true open model use the Arrivals Thread Group or Free-Form Arrivals Thread Group from the Custom Thread Groups plugin, and set a concurrency limit high enough for your worst expected latency.

Locust

Locust is built around users with a wait_time, which is a closed model. constant_pacing(1) makes each user start a task about once per second, which approximates a fixed rate while responses stay under one second. Once they exceed it, the user falls behind and the rate drops. To approximate an open load, run many more users than you need at that pace so most are idle, and check the achieved RPS against your target.

What this changes in your load test

• Write the requirement as a rate. "300 RPS on checkout at p95 under 400 ms" maps directly to an open test. "300 users" does not; see VUs vs RPS.
• Give the open generator room. Required concurrency is λ × R by Little's Law. Allow for R at its worst, not its average.
• Expect earlier, louder failures. Open tests reach the knee at the same throughput, but past it they produce timeouts and errors instead of a quiet RPS drop. That is the honest result.
• Trust the tail. Closed generators skip the requests they would have sent during a stall, which is the source of coordinated omission. Open generators measure them.
• Keep closed tests for closed systems. Connection-limit tests, WebSocket soak tests and worker-pool tests are naturally closed, and a closed model is correct there.

In LoadTester, VU mode is the closed model and exact RPS mode is the open model. RPS mode plots active VUs live, so you can watch concurrency rise as latency grows and set a p95 threshold with auto-stop to end the run once the queue starts building.

Checklist for choosing a model

1. Can you name a fixed number of clients that generate this traffic? If yes, closed. If no, open.
2. For websites, model sessions arriving at a rate, each with a short scripted journey.
3. Set the arrival rate from production peak (requests per second, per endpoint) plus headroom.
4. Set the VU ceiling to target RPS × worst-case response time.
5. After the run, check achieved RPS against the target and look for dropped iterations before reading percentiles. The guide to reading load test results covers what to look at next.