All Posts
Testing 9 min read

Flaky test suites were slowing releases: How we tamed Playwright CI pipelines

#playwright#testing#ci/cd

End-to-end tests are the ultimate safety net before shipping code to production. But when tests pass on a developer's MacBook M3 and fail 25% of the time on GitHub Actions runners, developers stop trusting the test suite. They hit "Re-run failed jobs" until it turns green by luck. That is not testing; that is a roulette wheel.

Eliminating arbitrary timeouts forever

The number one source of test flakiness is `page.waitForTimeout(2000)`. Developers add sleep calls hoping an animation or network request finishes in time. On a resource-constrained CI runner sharing CPU with other jobs, that 2-second sleep might need 2.4 seconds, causing the assertion to fail.

Playwright has built-in auto-waiting. Rely on web assertions that retry automatically until conditions are met:

- Use `expect(locator).toBeVisible()` instead of checking DOM presence - Await network responses explicitly: `await page.waitForResponse(resp => resp.url().includes("/api/checkout") && resp.status() === 200)`

Isolated test databases and state seeding

Tests that share mutable database state are doomed to step on each other. When tests run in parallel across four CI workers, Test A might delete an item that Test B was actively validating.

We enforce complete test isolation:

- Generate dynamic test tenants with unique UUIDs for each test run - Seed prerequisite data directly via backend API calls or direct database factories rather than clicking through UI setup flows - Wipe tenant data in teardown hooks

Trace viewer: The ultimate debugging weapon

When an E2E test fails in CI, reading text error logs is frustrating. We configure Playwright to record full traces on first retry: `trace: "on-first-retry"`. The trace file captures a complete DOM snapshot, network traffic log, console output, and visual video recording for every step. Developers inspect failures locally as if they were sitting in front of the browser.

Have a similar challenge?

We build production-grade software for companies that need it done right.

Let's Talk