Test failure

Extract the failing assertion, environment, and useful diagnostic evidence.

Condense supplied test output into a diagnostic brief for the person investigating the failure. Focus on an individual test or a small related failure group, not the full build history.

Name the failing test and the exact assertion or exception. Show expected and actual values side by side when available. Retain the file and line, relevant environment, seed, changed configuration, and reproducibility evidence. Preserve the first useful application frame and any setup failure that prevents the test from exercising its target.

Remove passing-test output, progress indicators, repeated framework frames, and duplicate failures. Report a group count only if the input supports it. Keep flaky behavior, timeouts, and missing evidence distinct from deterministic product failures.

Output Test, Failure, and Evidence lines, plus a supplied reproduction command if present. Do not infer root cause from the test name, rewrite assertion values, or claim a rerun passed. Do not run tests or propose an unverified fix as fact.

Example

Cache expiry CI failure

Before 83 words

The Linux CI job on Python 3.12 ran 218 tests. There were 217 passes and one failure. The failing test was tests/test_cache.py::test_expires_at_boundary at tests/test_cache.py:88. The assertion expected get("session", now=60) to be None, but the actual value was "active". This run used CACHE_TTL_SECONDS=60 and seed 481. A local rerun with the same seed passed, so the failure is not yet reproduced locally. The output also contained the progress line for every passing test and 12 framework stack frames, none of which added another error.

After 34 words

Test: tests/test_cache.py::test_expires_at_boundary (tests/test_cache.py:88). Failure: get("session", now=60) expected None; got "active". Evidence: Linux, Python 3.12, CACHE_TTL_SECONDS=60, seed 481. CI: 1 failed, 217 passed. Local rerun with the same seed passed; failure not yet reproduced locally.

Examples illustrate the method. They do not measure model output.

Files and sources

prompt.txt Complete prompt as plain text.
SKILL.md The same prompt with a skill header.
llms.txt (JSON) Prompt, metadata, example, and sources.
LICENSE MIT license text.

An original sho.rten.it skill. Source notes.