Record & Replay HTTP Traffic

Prerequisites #

Recording is a complement to hand-written mocks, not a replacement: use it where fidelity to a real payload matters, and keep small hand-written fixtures for Playwright Route Mocking Strategies style edge cases that a real API will not readily produce.

Step-by-step implementation #

1. Capture a HAR against a real environment #

Playwright can record every request a test makes into a HAR archive and replay it afterwards with no code change beyond a flag.

// Trade-off: recording captures everything including third-party beacons,
// which bloats the archive — filter by URL to keep it reviewable.
await context.routeFromHAR('fixtures/invoices.har', {
  url: '**/api/**',      // record only our API, not analytics or fonts
  update: true,          // record mode; set false (or omit) to replay
});

Run once with update: true against staging, inspect the archive, then commit it and drop the flag. Every subsequent run replays from disk with no network access at all.

2. Scrub credentials and personal data before committing #

A raw HAR contains Authorization headers, session cookies, and whatever the API returned — which on a staging system is often real-looking data. Scrub as a pipeline step, not by hand.

// scripts/scrub-har.js
// Trade-off: aggressive scrubbing can remove a header the app actually needs
// on replay; keep an allowlist of headers that must survive.
import { readFileSync, writeFileSync } from 'node:fs';

const SENSITIVE = /^(authorization|cookie|set-cookie|x-api-key)$/i;
const har = JSON.parse(readFileSync(process.argv[2], 'utf8'));

for (const entry of har.log.entries) {
  for (const bag of [entry.request.headers, entry.response.headers]) {
    for (const h of bag) if (SENSITIVE.test(h.name)) h.value = 'REDACTED';
  }
  entry.request.cookies = [];
  entry.response.cookies = [];
}
writeFileSync(process.argv[2], JSON.stringify(har, null, 2));

Add a repository check that fails if a committed archive contains a token-shaped string, so the scrubbing cannot be skipped by someone in a hurry.

3. Record at the Node layer for API and server tests #

Browser-level HAR does not help a server-side integration test. nock records outbound HTTP from Node and can write the exchanges back out as fixtures.

// Trade-off: nock intercepts at the http module, so it covers anything using
// Node's stack and misses calls made by a native or spawned process.
import nock from 'nock';

// Record once against the real dependency:
nock.recorder.rec({ output_objects: true, dont_print: true });
await syncInvoicesFromBillingProvider();
const recorded = nock.recorder.play();       // write these to a fixture file

// Replay in CI, with no network allowed at all:
nock.disableNetConnect();
nock.define(JSON.parse(readFileSync('fixtures/billing.json', 'utf8')));

disableNetConnect() is the important half: it turns “we think everything is mocked” into a guarantee, because any unmocked call fails loudly instead of silently reaching the internet.

4. Detect drift on a schedule, not during an incident #

A recording is a snapshot of an API on a particular day. Re-record on a schedule in a job that is allowed to talk to staging, and diff the result against what is committed — the diff is your early warning that the contract moved.

# .github/workflows/rerecord.yml
# Trade-off: a nightly job against staging costs a little and finds contract
# changes days before they reach the release branch.
on:
  schedule:
    - cron: '0 3 * * *'
jobs:
  rerecord:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npx playwright test --grep @record --update-snapshots
      - run: node scripts/scrub-har.js fixtures/invoices.har
      - run: git diff --exit-code fixtures/ || echo "::warning::recorded API drift detected"

A structural diff on the response shapes is more useful than a raw text diff, which is the technique developed in Diffing OpenAPI Fixtures Against Live Schemas.

5. Keep recordings small and reviewable #

An archive that captures fonts, images, analytics and every poll is unreviewable, and an unreviewable fixture is one nobody notices going wrong. Filter at capture time, one archive per feature rather than one per suite.

6. Decide where each recording lives in the test pyramid #

Recordings are most valuable in the middle of the pyramid and least valuable at its extremes. At the unit level a recorded payload is usually overkill: the test cares about one branch of a parser, and a three-field literal expresses that better than a two-hundred-line capture. At the top, in a true end-to-end check against a real environment, a recording defeats the purpose — you wanted to know whether the real system works.

The sweet spot is integration coverage: a component or a page rendering a realistic payload, a server-side sync job processing a real provider response, a workflow spanning several endpoints. These are exactly the tests where hand-written fixtures fail most often, because the payloads are large enough that nobody reproduces them faithfully and small omissions — a nullable field, an unexpected ordering, an extra wrapper object — change how the code behaves.

A practical rule: if writing the fixture by hand would take more than a few minutes, or if you find yourself copying a response out of a browser’s network panel, that is a recording. If you can state the fixture’s purpose in a sentence — “an empty list”, “a 402 with a decline code” — write it by hand, because the intent is the valuable part and a recording would bury it.

7. Treat recordings as reviewed artefacts, not build output #

The most consequential decision in this whole area is whether a recording update is reviewed. A fixture is a claim about how a dependency behaves, and an unreviewed update means the suite silently adopts whatever the dependency now does — including a breaking change you needed to know about. That is why the re-record job in the previous step emits a warning and a diff rather than committing.

Reviewing recordings requires them to be reviewable, which puts real weight on the earlier choices: filter at capture time so the archive contains only your API, normalise volatile values so the diff shows meaning rather than noise, and keep one archive per feature so a reviewer sees a bounded change. A team that gets those three right reviews fixture diffs as a matter of course; a team that does not will rubber-stamp them, and the detection value evaporates.

// A minimal guard that keeps recordings honest in review.
// Trade-off: another pipeline step, and it is what prevents a token or a
// half-megabyte of analytics traffic from entering the repository unnoticed.
const MAX_BYTES = 300_000;
const TOKEN_LIKE = /(bearer\s+[a-z0-9._-]{20,}|sk_live_[a-z0-9]{16,})/i;

for (const file of changedFixtures) {
  const raw = readFileSync(file, 'utf8');
  if (raw.length > MAX_BYTES) fail(`${file} is too large to review — tighten the capture filter`);
  if (TOKEN_LIKE.test(raw)) fail(`${file} contains a credential-shaped string`);
}

Configuration reference #

Data-driven analysis #

• Recording age. Days since each archive was captured. Anything past a month should be treated as suspect; the re-record job keeps this near zero for actively used fixtures.
• Drift rate. How often the scheduled re-record produces a structural diff. A rising rate means the API is changing faster than the suite is tracking it, which is a signal to invest in contract validation rather than more recordings.
• Unmatched request count. Requests attempted at replay time with no recorded match. This should be zero; a non-zero count means the test is exercising a path the recording does not cover, and the results are meaningless for that path.
• Archive size per feature. Large archives indicate insufficient filtering. Past a few hundred kilobytes, nobody reviews the diff, and an unreviewed fixture is an unverified assumption.
• Live-call escape rate. With disableNetConnect() in place this is zero by construction; without it, measure it, because it is the number that explains why the suite fails when a vendor has an outage.

Matching strategies compared #

Every replay tool has to answer the same question — which recorded exchange answers this request — and the answer it gives determines how the fixture behaves under change.

Order-based matching replays the nth recorded response to the nth request. It is the simplest to implement and the most fragile in use: a test that skips a request, adds a prefetch, or runs its requests concurrently receives the wrong bodies with no error, because every request still finds a match. Avoid it for anything beyond a strictly linear script.

Method and path matching is the common default. It survives reordering and concurrency, and it collapses variants: paginated endpoints, filtered lists and anything distinguished by a query string return whichever variant was recorded first. It is adequate when each endpoint has exactly one meaningful response in the recorded scenario, and quietly wrong the moment that stops being true.

Method, path and query matching is the practical default for browser replay. It distinguishes the variants that matter for reads, at the cost of being sensitive to harmless query additions — a cache-busting parameter or an analytics tag will cause a miss, which is why capture filters and stable client URLs matter.

Full matching including a body hash is what write-heavy flows need, because two POST requests to the same path with different bodies are genuinely different exchanges. It is the strictest option and the most brittle: any change to a request body, including a new client version field, invalidates the match. Use it where the body determines the response, and prefer a body predicate over a literal where only some fields matter.

The choice is not global. A suite can reasonably match reads on path plus query and writes on a body predicate, and the archive is the same either way — what changes is how requests are looked up on the way back out.

Common pitfalls & mitigation strategies #

• Leaving record mode on in CI. Fixtures are rewritten from live traffic and the suite silently stops being deterministic. Mitigation: gate update: true behind an explicit environment flag.
• Committing unscrubbed archives. Tokens and personal data enter the repository history. Mitigation: scrub in a script and add a secret-shaped-string check to the pipeline.
• Matching on method and path only. Paginated and filtered requests collide and replay returns the wrong body. Mitigation: include the query string and, for writes, a body hash.
• Allowing unmatched requests to fall through. The suite passes locally and fails when the vendor is down. Mitigation: notFound: 'abort' and disableNetConnect().
• One giant archive per suite. Diffs are unreviewable and every test depends on every recording. Mitigation: one archive per feature, filtered by URL.
• Never re-recording. The suite ends up testing an API version that no longer exists, which is the worst outcome: green tests and a broken product. Mitigation: a scheduled re-record with a diff warning.
• Recording against production. Real customer data lands in the repository. Mitigation: record against a seeded staging environment only.

Frequently Asked Questions #

Q: Who owns a recorded fixture once it is committed? A: The team that owns the integration, not the team that happened to write the test. A recording is a statement about a dependency’s behaviour, so when the drift job flags it, the person best placed to decide whether to adopt or escalate the change is whoever owns that integration. Routing the warning accordingly is what keeps it from becoming unattended noise — the same ownership argument made in Flaky Test Triage & Ownership.

Q: Should recorded archives live in the repository or in an artifact store? A: In the repository while they stay small and reviewable, because that is what makes a contract change visible in a pull request diff. Move to an artifact store only when size makes review impossible — and accept that you lose the review signal, so compensate with a structural diff report in the pipeline.

Q: How is this different from just writing fixtures by hand? A: Fidelity and intent point in opposite directions. A recording reproduces exactly what the API sent, including fields nobody remembered to mock, which is invaluable for realistic rendering. A hand-written fixture expresses a scenario you want to test, including ones the API rarely produces. Use recordings for the realistic path and hand-written fixtures for errors, empty states and edge cases.

Q: What happens when the API changes and the recording does not? A: The suite keeps passing against the old contract, which is the failure mode replay is most prone to. That is the entire reason for the scheduled re-record and the structural diff — without them, replay converts an unreliable test into a confidently wrong one.

Q: How large is too large for a recorded archive? A: The practical limit is review, not disk. Once a diff no longer fits on a screen, nobody reads it, and an unread fixture diff is the same as no drift detection at all. A few hundred kilobytes per feature is a workable ceiling; past that, tighten the capture filter, split the archive, or ask whether the test really needs the whole payload rather than one representative record.

Q: Do recordings replace contract testing? A: No, they consume it. A recording tells you what the provider returned once; a contract test tells you what the provider promises to keep returning. Recordings go stale silently, which is exactly why the detection loop in this topic leans on schema validation — the schema is the contract, and the recording is a sample of it.

Q: Can recording capture non-deterministic values like timestamps and ids? A: It captures them literally, which is usually fine because replay returns the same bytes every time. The problem is assertions: a test asserting “created just now” against a timestamp recorded last month will fail. Normalise volatile fields during scrubbing, or freeze the clock as described in Fixing Timezone and Locale Dependent Test Failures.

Related guides #

• Recording and Replaying HAR Files in Playwright
• Detecting Stale Recorded Fixtures in CI
• Replaying HTTP Traffic in Jest with nock
• Environment Parity & Mock Data Management
• MSW in JavaScript Test Suites