Investigate flaky tests and report root causes, without fixing them yet
Category: Testing · Updated 2026-09-22 · Tools: Both
Use this when CI or local runs occasionally fail in ways that pass on retry, and you want to understand the pattern before anyone starts changing test code (fixing flaky tests blind often just hides the real bug). Small to medium usage — mostly running tests repeatedly and reading output, not writing code.
Full prompt
Run this project's test suite repeatedly (at least 10 times, or until you've caught at least 3 distinct intermittent failures, whichever comes first) and record which tests fail inconsistently. For each flaky test found, investigate the likely root cause — timing/race conditions, shared mutable state between tests, unmocked network or clock dependencies, test-order dependency, or something else — and note your confidence level in that diagnosis. Do not modify any test or application code in this pass. Produce a report: test name, how often it failed out of how many runs, suspected root cause, confidence (high/medium/low), and a one-line suggested fix direction (not an implementation) for each one, sorted most-frequent-failure first.
Tips
- Insist on the report-only constraint — an agent asked to "look into" flaky tests will often start patching them immediately, which papers over the actual bug
- Ask it to state a confidence level per diagnosis so you know which ones need a human second look before acting on them
- If your suite is slow, tell the agent up front which subset of tests to focus the repeated runs on
Still waiting on a reset? See what to do when you hit the limit, or check Codex / Claude time since last reset.
Back to the prompt library · Subscribe: all-products feed