$ npx playwright test --project=gary-parker
- playwright-smart-reporter121,356 downloads/mo · 90 stars
- playwright-affected809 downloads/mo · 6 stars
- tracescrub545 downloads/mo
- playwright-shard-balancer319 downloads/mo
- playwright-release-notes-examples36 stars
5 passed123,029npm downloads/mo · 132 stars
Gary Parker
Quality engineering for teams shipping with AI: I build open-source Playwright tooling and write up experiments with measured results.
writing
view all →- Playwright --update-snapshots Overwrote a Good Baseline and Passed: 226 of 226 RunsOn a page that never stops moving, --update-snapshots=changed or all wrote a random frame as the baseline and passed, on Playwright 1.40 to 1.65.1. Masking the container stopped it.
- AI-Generated Playwright Tools: 1 in 4 Runs Returned Wrong Data With No Error3 AI-generated Playwright tools, 10 page changes: 16 of 66 runs reported success with wrong data, and the repair loop never saw them. Four checks against the page took that to 0.
- Playwright MCP vs CLI: Token Cost per Step Is Now the SameMCP 0.0.85 and CLI 0.1.24 returned responses within 25 tokens of each other on 5 scripted tasks. In two of them, find cut the tokens 19x and 128x against a full snapshot.
- Finding Flaky Playwright Tests That Fail Every RetryA test failed 4 of 11 runs and Playwright's report never called it flaky. Tracking results across runs marked it flaky at run 5 and quarantined it. 11 scripted runs, public repo.
- Playwright 1.65: A Filter That Matches Nothing Now Fails Every Shard1,464 runs on Playwright 1.64.1 and 1.65.1. A sharded run with zero matching tests went from all green to all red. Empty shards still pass. One flag still slips through.
experiments
- poc-playwright-test-agentsMeasured evaluation of Playwright Test Agents (planner/generator/healer) against a one-pass baseline suite.
- poc-ai-code-test-coverageAn agent's own tests reached 100% line coverage and a 90.67% mutation score.
- poc-self-healing-selectorsOver two drift releases the healer made 14 repairs on a CSS-selector suite, 2 on ARIA, 1 on test IDs.
- poc-aria-snapshot-assertionsTest-ID assertions caught 0 of 5 seeded accessibility regressions; ARIA snapshots caught 5 of 5.
newsletter
QA Ark Newsletter
Test automation, AI in testing and quality engineering practice. No spam, unsubscribe any time.