CI-first knowledge base Reliability Engineering

Article · Flaky Test Detection & Quarantine Engineering

Quarantine Policies in Monorepos

A quarantine policy that works for one application becomes contentious the moment several teams share a repository. Whose budget does a shared-library flake count against? Can one team's quarantine decision unblock a pipeline everyone depends on? What stops a package from quarantining its way to a permanently green build? This guide adapts the mechanisms in Building Auto-Quarantine Workflows to a repository with many owners.

13 sections URL: /flaky-test-detection-quarantine-engineering/building-auto-quarantine-workflows/quarantine-policies-in-monorepos/
Quarantine scope in a shared repository Package-scoped quarantine affects one team's suite; a shared library's quarantine affects every consumer downstream. packages/uishared library apps/checkout apps/admin apps/storefront one quarantined library test removes coverage from all three consumers
In a monorepo the blast radius of a quarantine decision follows the dependency graph, not the directory it was made in.

Root cause #

A single-application quarantine policy makes an assumption that quietly fails in a shared repository: that the person quarantining a test and the people affected by it are the same. In a monorepo, a test in a shared package protects every consumer, so quarantining it is a coverage decision on behalf of teams who were not consulted and may not notice.

The second mismatch is the budget. One repository-wide flake rate makes every team responsible for every other team’s instability, which produces the worst of both worlds: teams with clean suites are blocked by teams with dirty ones, and the dirty ones face no specific pressure because the number is diluted. Aggregate metrics in shared repositories have to be decomposed by owner or they cease to motivate anyone.

The third is the affected-package problem. Monorepo pipelines typically run only the tests affected by a change, which is what keeps them fast. Combined with quarantine, this produces a subtle gap: a test can be quarantined by the team that owns it, then never run by anyone else’s pipeline, so its non-blocking lane is empty and the stability gate has no data. The quarantine becomes permanent by omission rather than by decision.

Step-by-step fix #

1. Scope quarantine entries by package and record the consumers #

Make the blast radius explicit in the record, so a decision about a shared package is visibly different from one about a leaf application.

// quarantine.json — one file, entries scoped by package
// Trade-off: a single file is easy to audit and becomes a merge-conflict point
// in a busy repository; per-package files trade that for a harder global view.
[
  {
    "test": "Button › fires onClick once",
    "package": "packages/ui",
    "owner": "@design-systems",
    "affects": ["apps/checkout", "apps/admin", "apps/storefront"],
    "since": "2026-07-20",
    "expires": "2026-08-17"
  },
  {
    "test": "checkout › applies discount",
    "package": "apps/checkout",
    "owner": "@payments",
    "affects": ["apps/checkout"],
    "since": "2026-07-28",
    "expires": "2026-08-25"
  }
]

2. Require consumer sign-off for shared packages #

A quarantine that removes coverage from other teams should need their acknowledgement. Use the ownership mapping to request it automatically rather than relying on anyone remembering.

// scripts/quarantine-review.js
// Trade-off: requiring sign-off slows quarantine of a shared test, which is
// appropriate — that decision spends other teams' safety margin.
const entry = proposedQuarantine;
const consumers = dependents(entry.package);            // from the workspace graph

if (consumers.length > 1) {
  await requestReview(consumers.map((c) => ownerOf(c)));
  console.log(`::notice::shared-package quarantine requires sign-off from ${consumers.length} consumers`);
}
Two policies by blast radius A leaf application test can be quarantined by its owner; a shared package test requires consumer sign-off and a shorter expiry. leaf application test owner decides alone expiry: 28 days affects one team's coverage shared package test consumer sign-off required expiry: 14 days affects everyone downstream
Shorter expiry for shared code reflects the larger coverage debt it creates, not a judgement about the team.

3. Budget per package, and roll up for visibility #

Give each package its own flake rate and threshold. The repository-level number is for reporting; the package-level number is what gates.

// scripts/package-budgets.js
// Trade-off: per-package budgets are more configuration and are the only way a
// team can be accountable for a number it can actually move.
const BUDGETS = {
  'packages/ui': 0.5,          // shared code, stricter
  'apps/checkout': 1.5,
  'apps/admin': 2.0,           // legacy suite, ratcheting down
};

for (const [pkg, budget] of Object.entries(BUDGETS)) {
  const rate = flakeRateFor(pkg);
  if (rate > budget) {
    console.error(`::error::${pkg} flake rate ${rate.toFixed(2)}% exceeds ${budget}%`);
    failures.push(pkg);
  }
}

Shared packages deserve the strictest budgets: their tests protect the most consumers, and their instability propagates into every dependent pipeline.

4. Keep quarantined tests running even when the package is unaffected #

Affected-package selection would otherwise starve the stability gate. Run the quarantine lane on a schedule across the whole repository, independent of what changed.

# .github/workflows/quarantine-lane.yml
# Trade-off: a nightly full-repository quarantine run costs runner time and is
# what keeps the gate supplied with data for packages nobody touched this week.
on:
  schedule:
    - cron: '0 2 * * *'
jobs:
  quarantined:
    steps:
      - run: pnpm -r test --project=quarantined   # every package, non-blocking
        continue-on-error: true
      - run: node scripts/update-stability.js

5. Prevent quarantine from becoming a package’s default #

A package that quarantines steadily will eventually have a green pipeline and no coverage. Cap the proportion of a package’s tests that may be quarantined at once.

// Trade-off: a hard cap can block a legitimate quarantine during a bad week,
// which is the point — at that ratio the package needs attention, not another skip.
const MAX_QUARANTINE_RATIO = 0.02;      // 2% of a package's tests

const ratio = quarantinedCount(pkg) / testCount(pkg);
if (ratio > MAX_QUARANTINE_RATIO) {
  throw new Error(`${pkg}: ${(ratio * 100).toFixed(1)}% of tests quarantined — fix before adding more`);
}

6. Report the coverage debt where consumers will see it #

Each consumer’s pipeline should surface the quarantined tests it inherits from its dependencies. A team that cannot see the coverage it has lost cannot weigh it.

Pitfalls #

  • One repository-wide flake budget. Nobody is accountable and clean teams are blocked by dirty ones. Mitigation: per-package budgets, rolled up for reporting.
  • Quarantining shared code unilaterally. Coverage is removed from teams who never agreed. Mitigation: consumer sign-off derived from the dependency graph.
  • Relying on affected-package selection for the quarantine lane. The stability gate starves. Mitigation: a scheduled full-repository quarantine run.
  • The same expiry for leaf and shared tests. Shared coverage debt lasts as long as local debt. Mitigation: shorter expiry for shared packages.
  • No cap on quarantine ratio. A package quarantines its way to a green, meaningless pipeline. Mitigation: cap the proportion and fail on breach.
  • Invisible inherited debt. Consumers do not know what coverage they have lost. Mitigation: report inherited quarantines in each consumer’s summary.
  • A single quarantine file in a busy repository. Constant merge conflicts push people to work around it. Mitigation: per-package files with a rolled-up view.
Where each control applies Per-package budgets gate, the scheduled lane feeds the gate, the ratio cap bounds accumulation and inherited debt is reported to consumers. per-package budgetgates that package scheduled lanefeeds the gate ratio capbounds accumulation inherited debtshown to consumers four controls, each addressing a failure mode that only appears with multiple owners
None of these are needed in a single-application repository, and all four become necessary once ownership is shared.

Reliability targets #

Metric Target Notes
Packages with their own budget 100% Shared packages strictest
Shared-package quarantines with consumer sign-off 100% Derived from the dependency graph
Quarantined tests per package ≤ 2% of that package’s tests Hard cap
Quarantine-lane executions per test per week ≥ 5 From the scheduled full run
Consumers shown inherited quarantines 100% In each pipeline summary
Monorepo quarantine scorecard Targets for per-package budgets, sign-off, quarantine ratio and lane executions. 100%packages budgeted 100%shared sign-off ≤ 2%quarantine ratio ≥ 5/wklane executions
The ratio cap is the control most often missing, and the one that prevents a package quietly abandoning its suite.

Frequently Asked Questions #

Q: Should a shared-library flake block every consumer’s pipeline? A: While it is unquarantined, yes — that is what shared coverage means, and it is the pressure that gets library flakiness taken seriously. Once quarantined, it should block nobody and be visible to everybody. The failure mode to avoid is a library test that blocks three teams for a week because its owners have other priorities.

Q: How do we stop one team quarantining tests to unblock a release? A: The ratio cap and the expiry do most of the work, and the review requirement does the rest. A quarantine that needs consumer sign-off and expires in a fortnight is a poor tool for avoiding work, which is precisely the intention. Watching quarantine inflow per package makes a team using it as an escape hatch visible within a sprint.

Q: Does per-package budgeting create too much configuration? A: A number per package is not much configuration, and it can be defaulted — every package starts at the repository default and only differs where a team has argued for something else. The alternative, a single number, is simpler and fails in the specific way that matters: nobody feels responsible for it.

Q: What about tests that span packages, like end-to-end suites? A: Give them their own budget and their own owner, usually a platform or QA group, since they belong to no single package. Attributing an end-to-end failure to whichever package’s code happened to be involved produces endless disputes; owning the suite explicitly and routing individual failures by classification, as in Routing Flaky Tests to Code Owners, works far better.

Cross-Package Failures Need Their Own Owner #

Some tests genuinely belong to no single package: an end-to-end journey crossing three applications, a smoke suite exercising the whole platform, a contract check spanning several services. Attributing their failures to whichever package’s code happened to be involved produces endless disputes and no fixes.

The workable arrangement is to give those suites an explicit owner — usually a platform or quality group — with their own budget and their own place in the reporting. Individual failures are still routed by classification, so a wrong asserted value goes to the feature team while a timeout goes to the suite’s owner, but the suite as an asset has someone responsible for its health.

Without that assignment, cross-cutting suites become the least maintained part of a monorepo precisely because they are the most shared.

Surfacing inherited quarantines in each consumer’s pipeline summary is the piece most implementations omit, and it is what lets a team weigh coverage they have lost to somebody else’s decision rather than discovering it during an incident.