CI-first knowledge base Reliability Engineering

Article · Root Causes of JavaScript Test Flakiness

Handling API Timeouts Without Arbitrary Waits

Every waitForTimeout(3000) in a suite is a bet that the API will answer within three seconds on the slowest runner the pipeline will ever schedule. The bet is lost occasionally, which is what flakiness is, and the usual response — raise it to five — only moves the failure further out while adding dead time to every run. This guide sits under Network Latency & Volatility Handling and replaces the fixed sleep with conditions that describe what the test is actually waiting for.

13 sections URL: /root-causes-of-javascript-test-flakiness/network-latency-volatility-handling/handling-api-timeouts-without-arbitrary-waits/
A fixed wait against a variable response time Response times form a distribution; a fixed sleep is either wasted time on fast responses or a failure on the slow tail. response time → fixed 3 s wait wasted time on every fast run slow tail — test fails raising the number trades more dead time for a slightly thinner tail; it never removes the tail
A fixed sleep is simultaneously too long for the common case and too short for the tail — the two failure modes are the same setting.

Root cause #

Network latency is a distribution, not a number. A request that takes 120 ms at the median can take 900 ms at the 99th percentile on a loaded CI runner, and the tail is fatter in CI than locally because the runner shares a host, the container has a fraction of the CPU, and the service under test is often starting cold. A fixed wait picks one point on that distribution and asserts that reality stays to the left of it. Given enough runs, reality visits the right-hand side.

The deeper problem is that a sleep expresses the wrong thing. The test does not care about elapsed time; it cares that the request completed and the interface finished reacting to it. Those are observable conditions — a response received, a spinner gone, a row rendered — and waiting on the condition is both faster than the sleep in the common case and immune to the tail. A sleep is what you write when the condition has not been made observable, which makes most sleeps a design smell in the application as much as in the test.

There is a third mechanism that keeps sleeps alive: the condition people wait on is often the wrong one. Waiting for a network response is not the same as waiting for the UI to have re-rendered from it — there is a task boundary in between — so a test that awaits the response and asserts immediately can still be a frame early. That near-miss teaches teams that “waiting properly does not work”, and they put the sleep back. The fix is to wait on the rendered consequence, which is the condition the assertion actually depends on.

Step-by-step fix #

1. Wait for the response, then for its consequence #

Playwright can wait for the network event and for the resulting DOM state; the assertion should depend on the second.

// Trade-off: waiting for both is two conditions instead of one, and it is what
// removes the one-frame gap between "response arrived" and "UI updated".
const response = page.waitForResponse(
  (r) => r.url().includes('/api/invoices') && r.status() === 200
);
await page.getByRole('button', { name: 'Load invoices' }).click();
await response;                                        // network settled
await expect(page.getByRole('row')).toHaveCount(25);   // UI settled

Web-first assertions retry until the condition holds or the timeout expires, so the second line already contains the waiting; no sleep is required and a fast response finishes immediately.

2. Alias the request in Cypress and wait on the alias #

The Cypress equivalent is an intercept alias, which is also what makes the wait visible in the command log when it fails.

// Trade-off: aliasing every request is verbose; alias the ones a test depends
// on, since an un-aliased request is one the test cannot reason about.
cy.intercept('GET', '/api/invoices*').as('invoices');
cy.findByRole('button', { name: 'Load invoices' }).click();
cy.wait('@invoices').its('response.statusCode').should('eq', 200);
cy.findAllByRole('row').should('have.length', 25);

The aliasing patterns that make this reliable — including what to do when several requests match one pattern — are covered in Aliasing and Waiting on cy.intercept Requests.

The gap between response received and UI updated A response arrives, then state updates, then the component re-renders; asserting on the first step is a frame early. responsereceived state updatedmicrotask re-renderednext frame asserting here is early assert here a retrying assertion on the rendered result spans all three steps without naming any of them
Most "waiting does not work" experiences come from waiting on the network event and asserting before the render.

3. Make the loading state observable #

If the interface has no signal for “busy” and “settled”, tests have nothing to wait on and sleeps become inevitable. Adding an explicit state attribute is a small application change that removes a whole category of flakiness — and it improves accessibility at the same time, because assistive technology needs the same signal.

// In the component: expose the state the test (and a screen reader) needs.
// Trade-off: one more attribute to maintain, in exchange for deleting every
// sleep that exists because the UI had no observable settled state.
<section aria-busy={isLoading} data-state={isLoading ? 'loading' : 'ready'}>
  {rows.map(renderRow)}
</section>
// In the test: wait for the state, not for a duration.
await expect(page.getByTestId('invoice-list')).toHaveAttribute('data-state', 'ready');

4. Set timeouts per operation, not globally #

One global timeout has to accommodate the slowest operation in the suite, which makes every fast failure slow. Scope longer timeouts to the operations that genuinely need them.

// Trade-off: per-assertion timeouts are more configuration, and they keep a
// fast test failing fast while a known-slow report still gets its budget.
await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();     // default 5 s
await expect(page.getByTestId('quarterly-report')).toBeVisible({ timeout: 30_000 }); // known slow

5. Deflake the dependency instead of the test where you can #

A test that waits on a genuinely slow third-party call is measuring someone else’s infrastructure. If the call is not the subject of the test, mock it and get a deterministic response time; if it is the subject, keep exactly one test that exercises it for real and mock it everywhere else. The mocking patterns are in Playwright Route Mocking Strategies, and the decision of what to mock versus what to exercise is the trade-off discussed in Network & API Mocking for Reliable Tests.

6. Find the remaining sleeps and treat them as debt #

Sleeps accumulate quietly. Make them visible with a lint rule so each new one is a decision rather than a habit.

// Trade-off: a hard ban forces a few genuinely awkward cases into workarounds;
// a warning with an allowlist keeps the count visible without blocking work.
'no-restricted-syntax': ['error', {
  selector: "CallExpression[callee.property.name='waitForTimeout']",
  message: 'Wait for a condition (response, state attribute, assertion) instead of a duration.',
}],

7. Choose the condition that matches the risk #

Not every wait deserves the same rigour, and knowing which condition to reach for is most of the skill. Ordered from weakest to strongest:

Element present is the weakest useful condition. It passes as soon as the node exists, including when it exists empty, so it suits structural checks and little else. Element visible adds layout, which rules out the hidden-placeholder case but still says nothing about content. Content matches — a specific text, a count, a value — is the first condition that ties the wait to the data the assertion depends on, and it should be the default. Network settled plus content matches is the strongest, and it is worth the extra line when a test needs to distinguish “the list is empty because the response said so” from “the list is empty because the response has not arrived”.

The failure mode to avoid is waiting on something correlated with, but not caused by, the state you need. A spinner disappearing is correlated with data arriving; it is not the same event, and the two diverge exactly when a response is cached or an error path skips the spinner entirely.

Pitfalls #

  • Raising the sleep until CI is green. The tail moves with runner load, so the number is never large enough for long. Mitigation: wait on the condition and delete the sleep.
  • Waiting for the response but asserting immediately. There is a render boundary after the response. Mitigation: assert on the rendered consequence with a retrying assertion.
  • A pattern matching several requests. cy.wait resolves on the first match, which may not be the one that matters. Mitigation: narrow the matcher by method, path and query.
  • One global timeout for everything. Fast failures become slow and slow operations still fail. Mitigation: scope timeouts per operation.
  • Waiting on a spinner that never appears. A cached response can skip the loading state entirely, so waiting for it to appear then disappear hangs. Mitigation: wait for the settled state, not for the transition through busy.
  • Testing a third party’s latency. Its slow tail becomes your flakiness. Mitigation: mock it unless it is the subject of the test.
Replacing a sleep with a condition A sleep waits a fixed duration regardless of outcome; a condition returns as soon as it holds and fails with a description when it does not. waitForTimeout(3000)always 3 s fails with "expected true, got false" — no diagnosisand costs 3 s on every passing run expect(...).toHaveCount(25)retries until true returns in 120 ms typically; fails with the actual countthe failure message names the problem
The condition is faster on success and more informative on failure — the sleep is worse on both axes.

Reliability targets #

Metric Target Notes
waitForTimeout / cy.wait(ms) calls 0 Enforced by lint rule
Median wait time per interaction < 300 ms Conditions return as soon as they hold
Timeout headroom (median ÷ timeout) < 20% Above 50% means the next slow runner turns it red
Timeout-signature failures per week < 1 Distinguished from assertion failures in reporting
Third-party calls exercised for real ≤ 1 test per integration Everything else mocked
Waiting scorecard Targets for fixed sleeps, median wait, timeout headroom and timeout failures. 0fixed sleeps < 300 msmedian wait < 20%timeout headroom < 1/wktimeout failures
Removing sleeps usually makes a suite faster and steadier at once, because the common case no longer pays for the tail.

Frequently Asked Questions #

Q: Is there ever a legitimate use for a fixed wait? A: A narrow one: waiting out a debounce or throttle whose duration is a known constant of the application, where nothing observable changes until it elapses. Even then, controlling the clock is better — advance the fake timer by the debounce interval and continue immediately, as in Testing Debounced Search Inputs Deterministically.

Q: My assertion retries but still fails intermittently. What now? A: The condition is probably not the one that matters. A retrying assertion on “the row exists” passes as soon as an empty placeholder row renders, while the test really depends on the row having data. Tighten the condition to the state the assertion needs — a count, a text value, a settled attribute — rather than lengthening the timeout.

Q: How do I know what timeout to set once the sleeps are gone? A: Measure. Take the 95th percentile duration of the operation across a few hundred CI runs and set the timeout at three to four times that. That gives room for the tail without letting a genuinely hung operation consume minutes, and the ratio is a number you can watch drift over time.

Q: Does removing sleeps make the suite faster? A: Usually significantly. Every sleep costs its full duration on every passing run, so a suite with a hundred three-second sleeps spends five minutes per run waiting for nothing. Conditions cost the actual latency, which is typically an order of magnitude less.