R-CMD-check r-universe Codecov test coverage mutator pkgdown reference

mutator

Overview

mutator is an automated mutation testing tool for the R language. It applies mutation testing principles to help developers improve test suite quality by introducing small, systematic changes (mutations) to source code and verifying if tests can detect these changes.

For instance, imagine you have the following function f in your package:

f <- function(x) {
  if (x > 0) {
    return(x + 1)
  } else {
    return(x - 1)
  }
}

mutator will generate several mutants, including:

f <- function(x) {
  if (x < 0) { # Mutated comparison operator
    return(x + 1)
  } else {
    return(x - 1)
  }
}

In this mutant, > has been replaced with <.

If your test suite does not catch this change, it indicates that your tests may not be comprehensive enough.

mutator will compute how many mutants survives, i.e. the test suite does not fail for the mutant, and how many mutants are killed, i.e. the test suite fails for the mutant. The ratio of killed mutants to total mutants is called the mutation score and is a measure of test suite effectiveness.

For a broader introduction to the tool, see the useR! 2026 talk on mutator.

Features

Installation

# Install the released version from CRAN
install.packages("mutator")

Development version

# Install the development version from GitHub; package dependencies are installed automatically
# install.packages("remotes")
remotes::install_github("PRL-PRG/mutator")

# Or install pre-built binaries of the development version from R-universe
# (https://prl-prg.r-universe.dev)
install.packages("mutator", repos = c(
  "https://prl-prg.r-universe.dev",
  "https://cloud.r-project.org"
))

# Or install from local source, in an R console
# install.packages("devtools")
setwd("path/to/mutator")
devtools::install()

Quick Start Guide

library(mutator)

# Mutate a single file
mutants <- mutate_file("path/to/your/file.R")

# Optional: cap returned mutants by random selection
mutants <- mutate_file("path/to/your/file.R", max_mutants = 20)

# Mutate an entire package and run tests
result <- mutate_package("path/to/your/package")

# Optional: cap tested mutants across the whole package
result <- mutate_package("path/to/your/package", max_mutants = 100)

# Optional: set a fixed timeout (seconds) per mutant test run
result <- mutate_package("path/to/your/package", timeout_seconds = 60)

# Optional: control where mutant files are written
result <- mutate_package("path/to/your/package", mutation_dir = tempdir())

mutator, in addition to show you the number of generated mutants, the surviving ones, and the mutation score, returns an invisible list with four components, a list of the generated mutants, a list of mutant outcomes, a list of phase durations, and a summary of the mutation testing run.

Mutant outcomes are reported as:

See the pkgdown reference for the full argument and return-value documentation.

Continuous integration (GitHub Actions)

mutator ships a reusable workflow so any R-package repository can run mutation testing in CI without copying scripts. Add a caller workflow at .github/workflows/mutation-testing.yaml:

on:
  pull_request:
  push:
    branches: [main, master]

name: mutation-testing

jobs:
  mutation:
    uses: PRL-PRG/mutator/.github/workflows/mutation-testing.yaml@v0.2.0
    with:
      target-margin: "0.10" # sample to +/-10 percentage points
      fail-under: "75" # fail CI below a 75% mutation score

Pin to a released tag such as @v0.2.0; the workflow is versioned with the mutator package, so the tag matches the package version. Set deploy-badge: true (with contents: write permission) to publish a shields.io badge. See the Continuous integration vignette for every input, threshold guidance, and badge setup. From an installed copy, open it with vignette("continuous-integration", package = "mutator").

Configuration

mutate_package() exposes a number of options to control how mutants are run, which tests are selected, and how results are refined. Each is covered in depth in the Configuration vignette (from an installed copy, open it with vignette("configuration", package = "mutator")):

Mutation Operators

mutator currently generates these mutation families:

Family Mutations
Arithmetic operators + ↔︎ -, * ↔︎ /
Comparison operators == ↔︎ !=, < ↔︎ >, <= ↔︎ >=
Logical operators & ↔︎ \|, && ↔︎ \|\|, removes !, and negates if / while conditions
Assignment and call values Replaces assignment right-hand sides and ordinary function calls with 42
Scalar constants Replaces numeric zero with 42, numeric non-zero values with 0, constants with a typed NA, and constants with NULL
Returns Replaces non-constant direct return() values with NULL, for example return(x) → return(NULL)
Deletions Deletes statements inside { ... } blocks and, as a fallback, valid source lines

Dependencies

mutator depends on:

mutator’s test suite

mutator itself uses testthat for its own R tests and testthat + Catch2 for C++ tests.

Run the full test suite with:

devtools::test()

System tests

mutator includes a set of system tests that run mutation testing on 9 packages from CRAN. It compares the mutants, surviving, killed, hanged mutant detections, and the mutation score against the expected results using snapshot testing, with a fixed seed on 10 mutants or 50 mutants per package depending on the release cycle. It also compares the results across a set of configuration options that ought not to change the results. The system tests are located in tests/system and are automatically run as par of the CI.

Mutation testing

We practice dogfooding and run mutator on mutator test suite itself. This already led to numerous improvements in mutator speed, as waiting for the mutation score to be computed on a package with a long running test suite can be frustrating!