> ## Content Index
> Fetch the complete content index at: https://madewithlove.com/blog/llms.txt
> Use this file to discover other available public pages before exploring further.

# Teenage Mutant Ninja Tests
- URL: https://madewithlove.com/blog/teenage-mutant-ninja-tests/
- Published: 2026-11-04T09:48:00.000Z
- Updated: 2026-11-04T09:48:00.000Z
- Description: Mutation testing was always too slow to run on a whole codebase, so nobody did. Agents change that: they touch a handful of files, don't mind a two-minute run, and escaped mutants tell them exactly which tests check nothing. A targeted Infection setup for PHP, plus the agent instructions.
- Author: Jonas Drieghe
- Tags: Testing, AI, AI Agents, Code quality, Engineering

Mutation testing is the gym membership of software engineering. Everyone agrees it's good for you. Everyone has a story about the one time they went. Nobody goes.

I think that's about to change. We didn't get any more disciplined, but we do have tireless little interns now, and they don't mind doing the boring reps.

## Who tests the tests?

A test that calls your method and asserts *absolutely nothing* gets the same code coverage as a test that nails every edge case. Coverage only knows the code ran.

Mutation testing asks a ruder, better question: **if I sabotage this code, does anyone notice?**

The tool takes your code and quietly breaks it. One tiny change at a time. Each broken version is a *mutant*:

- `>=` becomes `>`
- `true` becomes `false`
- `+` becomes `-`
- a method call just... disappears
- `return $value` becomes `return null`

Then it runs your tests.

- Tests go red? Mutant killed. Your tests are doing their job.
- Tests stay green? Mutant escaped. You can change what your code does and your test suite will smile and wave.

Exhibit A:

```php
public function isEligibleForDiscount(Order $order): bool
{
    return $order->total() >= 100;
}
```

```php
public function test_large_orders_get_a_discount(): void
{
    $this->assertTrue($this->policy->isEligibleForDiscount(OrderFactory::withTotal(150)));
}

public function test_small_orders_do_not_get_a_discount(): void
{
    $this->assertFalse($this->policy->isEligibleForDiscount(OrderFactory::withTotal(50)));
}
```

100% coverage. Green across the board. Ship it.

Now flip `>=` to `>`. Still green. Nobody tested an order of exactly €100, which is, of course, the *one* number the business actually cares about. Somewhere, a customer with a €100 basket is about to have a very confusing support chat.

In PHP land, the ooze of choice is [Infection](https://infection.github.io/). It plugs into PHPUnit or Pest, comes with a big bag of mutators, and gives you a **Mutation Score Indicator (MSI)**: the percentage of mutants your tests killed.

If you grew up on TMNT, picture every mutant as a Foot Clan soldier and your tests as the turtles. Each soldier that strolls past them unchallenged is a gap in your defences.

## So why isn't everyone doing this?

It's slow.

Every mutant means another test run. A decent-sized codebase spawns thousands of them, and a full run sends the entire Foot Clan at your suite at once. Even with all the clever tricks, a full run can take long enough to grab a coffee, drink it, regret it, and grab another.

Nobody wants that in CI. Nobody runs it locally. So mutation testing ended up where good ideas go to retire: conference talks, one-off experiments, and a shiny badge on someone's open source README.

## What agents change

**Agents write a lot of our tests now, and they're people-pleasers.**  
Ask an agent to "add tests" and you'll get tests. They'll be green. Coverage will look great. The agent wants the tests to pass, and a test that checks nothing passes too. Happy path covered, boundary at 100 skipped, return value ignored. Eh.

Coverage won't catch that. Mutants will.

**Agents don't need the whole suite, and they don't get bored.**  
The cost was always about running it on the whole codebase while a human sat there waiting. An agent building a feature touches a handful of files. Mutating just those, against just the tests that cover them, takes seconds to a couple of minutes. And the agent won't wander off to Slack halfway through.

The loop looks like this:

1. Agent builds the thing and writes tests.
2. Agent runs a targeted mutation test on what it just touched.
3. Agent reads the escaped mutants.
4. Agent writes sharper tests until those mutants are dead.

This is Splinter running drills in the dojo: a small squad of Foot soldiers per feature instead of waiting for the full invasion. Every soldier that gets through shows which turtle needs more training.

Escaped mutants also happen to be great feedback for an agent. "Line 42, `>=` became `>`, and nobody noticed" is specific enough to act on right away.

## How to set it up

Infection already knows how to keep it small:

```bash
# Only mutate lines you changed compared to main
vendor/bin/infection --git-diff-lines --git-diff-base=origin/main --threads=max --show-mutations

# Or aim it at specific files or directories
vendor/bin/infection --filter=src/Billing/DiscountPolicy.php --threads=max --show-mutations
```

`--show-mutations` prints a diff for every escaped mutant, which is exactly what the agent needs to go hunting. You'll need a coverage driver (PCOV or Xdebug) so Infection knows which tests cover which lines.

Then tell your agent to actually do it. Drop something like this in your [CLAUDE.md / AGENTS.md](https://madewithlove.com/blog/onboard-the-ai-like-youd-onboard-a-developer/), or make it a skill:

```markdown
## Test quality: mutation testing

After adding or changing functionality and its tests, run a targeted mutation test
on the code you touched:

    vendor/bin/infection --git-diff-lines --git-diff-base=origin/main --threads=max --show-mutations

- For every escaped mutant, either add or improve a test that kills it, or explain
  why the mutant is equivalent (it doesn't change behaviour) and leave it.
- Never run Infection on the whole codebase.
- Don't change production code just to kill a mutant, unless the mutant reveals
  dead or redundant code.
```

The last rule keeps the agent from twisting production code into a pepperoni pizza just to make a tool happy.

## Caveats

- **Some mutants are innocent.** Sometimes a mutation doesn't change behaviour at all. These are *equivalent mutants*, and no test will ever kill them. Don't chase 100% MSI; that way lies madness.
- **Stay targeted.** Run it on the whole suite and you're right back in coffee-regret territory. If you want it in CI, keep the diff-based scope and set a sane `--min-covered-msi` on changed code only.
- **Dead mutants ≠ great tests.** A killed mutant means *some* test cares about that line, which says nothing about whether the test makes sense. [Code review](https://madewithlove.com/blog/ai-code-review-tools/) still has a job.

## Cowabunga!

Mutation testing has always been expensive, and the bill landed on a CI pipeline running your test suite a thousand times.

Agents don't need a thousand runs. They work in small slices, they don't mind a two-minute run, and escaped mutants give them something concrete to fix.

If agents are writing your tests, give them a way to check those tests actually test something. Better to spar with a few Foot soldiers in the dojo than to meet Shredder in production.