Mutation testing¶

Coverage says a line ran; mutation testing says a test would notice if it were wrong. mutmut rewrites src/ one small change at a time (a flipped comparison, a dropped argument, a changed string) and runs the suite against each rewrite, a mutant. A mutant the suite fails on is killed; one it passes survives, and is either a missing test or a change no input can tell apart from the original (an equivalent mutant).

Running it¶

The configuration is [tool.mutmut] in pyproject.toml. Mutation runs leave out the tests of the source's own shape (the API-surface snapshot, the import boundary, the no-logging and no-environment contracts), which read the rewritten files.

uv run mutmut run                       # every mutant, about 15 minutes on 10 workers
uv run mutmut run "headgate._failures*" # one module's mutants, while working on it
uv run mutmut results                   # the survivors
uv run mutmut show <name>               # one mutant's diff

Delete mutants/ if a run gets confused after files move. --max-children sets the number of workers.

On macOS, mutmut reports some mutants as segfault: the mutant made a test take a path through the system TLS stack (an SSL context built without verification, say) in a forked worker, where Security.framework aborts the process. The same tests fail on those mutants when run in a plain process, so they are counted as killed.

Latest score¶

A full run on 2026-10-26 (mutmut 3.8.0, 10 workers, about 14 minutes) generated 6,099 mutants. It ran against the source of headgate 0.7.0 (d3663d3), with the tests written for that run's survivors:

That is 5,968 of 6,099 detected (97.9%), and every mutant that changes behavior is detected. The same run before those tests left 184 survivors; the 53 that changed behavior (a reader's empty-body and refusal options, codecs built at run time, the conformance check's messages, a degraded cache fetch with a streamed body, a hedge refused by the rate budget, among others) each now fail a test.

Equivalent mutants¶

Every survivor was checked, and each one left is equivalent: no input distinguishes them. Most fall in a few families:

• header-name case: httpx2 headers are case-insensitive, so headers.get("ETag") and headers.get("etag") read the same field;
• codec, distribution and method names: "ascii"/"ASCII", "latin-1"/"LATIN-1", "utf-8"/"UTF-8", metadata.version("HEADGATE") and MethodFilter({"get"}) (names are normalized);
• falsy stand-ins: None for False or "" where the value is only tested for truth;
• defaults spelt out: an argument equal to the callee's default (to_bytes(8, "big"), zip(..., strict=False), secrets.token_bytes(32), copy_with(userinfo=None), which clears userinfo exactly as b"" does);
• unreachable fallbacks: a default for a value the code never sees missing (a header read with "" or "XXXX" when absent, both of which parse to nothing);
• type arguments only the type checker reads: typing.cast's first argument, and the type expression msgspec's encoder is handed.