Free checklist

Reviewing agent-written tests: 12 checks + a starter AGENTS.md (free)

For QA engineers whose coding agent now writes the Playwright tests.

Before you trust a green run

1. Every test traces to a rule in the story, spec or acceptance criteria. No rule, no test.
2. Assertions check behaviour (totals, states, messages), not only "element is visible".
3. No assertion was weakened, skipped or deleted to make a run pass. Check the diff, not the result.
4. No hard waits (waitForTimeout) and no blind retries hiding a flaky flow.
5. Locators are user-facing (getByRole, getByLabel) rather than brittle CSS chains.
6. Test data is synthetic. No real customer data, tokens or secrets in the repo.
7. Negative paths exist: invalid input, empty states, permissions, errors.
8. Break the feature on purpose: the test goes red. If it stays green, it isn't testing anything.
9. Tests are independent: run one alone, run them in random order.
10. Names say what is being proven ("rejects expired card"), not "test1".
11. Nothing changed outside the test folder (config, CI, secrets) without a human OK.
12. A human makes the release call. The agent's report is input, not the decision.

Starter AGENTS.md (drop it in your repo root)

# House rules for test agents
1. Never change or remove an assertion to make a failing test pass. Report the failure.
2. Only synthetic test data. Never real customer data, tokens or secrets.
3. Every test names the rule or acceptance criterion it covers.
4. No hard waits. Use Playwright's auto-waiting and web-first assertions.
5. Prefer getByRole / getByLabel / getByTestId locators.
6. Ask before touching config, CI pipelines, secrets or anything outside /tests.
7. When a test fails, report: what failed, the likely cause, and whether it is the test or the product.

Copy AGENTS.md

Follow Your Job Buddy on LinkedIn for more on QA careers and agentic test automation.

Also free: the 10-point QA CV check