> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vizra.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Baselines & Regressions

> Every run compared against the run you trusted — row by row, with receipts.

The core workflow: pick a run you trust, make it the **baseline**, and every future run is diffed against it. This is what turns evals from "did it pass?" into "is it getting worse?"

## Baselines

* **Automatic**: in Pest, the first run that passes its gate becomes its suite's baseline. Nothing to configure.
* **Manual**: promote any completed run from the [dashboard](/evals/dashboard), or:

```bash Terminal theme={null}
php artisan evals:baseline 01kyvrjyxp9ex0f3gtvznmafdn
```

One baseline per suite; promoting a new one demotes the old one transactionally.

## How runs are joined

Rows are matched across runs by **content hash** — a fingerprint of the row's input, prior messages, and expected data, computed when the dataset is read. Reordering your JSONL file, adding rows, or renaming files changes nothing; the same logical row lines up across months of runs. (For single-model runs the reported provider/model id is deliberately ignored in the join — providers rotate dated model ids like `gpt-5-mini-2025-08-07`, and that must not sever your history.)

## Classification

For each row present in both runs:

| Class             | Condition                                                                                                 |
| ----------------- | --------------------------------------------------------------------------------------------------------- |
| **Regressed**     | Pass rate dropped at all, **or** score mean dropped by more than `evals.compare.epsilon` (default `0.05`) |
| **Newly failing** | Was passing every sample on the baseline; isn't now (a subset of regressed)                               |
| **Improved**      | The symmetric opposite                                                                                    |
| Unchanged         | Everything else — score jitter within epsilon is noise, not signal                                        |

Rows only in the current run are **new**; rows only in the baseline are **removed**.

<Info>
  **Why epsilon?** You're sampling a nondeterministic system — small score wobble is expected. A pass-rate drop is always a regression, but a `0.93 → 0.90` score drift shouldn't page anyone. Tune `evals.compare.epsilon` to your tolerance.
</Info>

## Failing the build on regressions

```php In Pest theme={null}
->gate(maxRegressions: 0)
```

```text What a regression failure looks like theme={null}
Gate failed: 2 rows regressed against the reference run (allowed: 0).
  ↓ regressed: "What is your refund policy?" 96.7% → 51.7%
  ↓ regressed: "Can I return it?" 93.3% → 55.0%
```

Or without Pest:

```bash Terminal theme={null}
php artisan evals:run SupportQuality --compare=baseline --output=json
# exit 0 = clean · exit 1 = gate failed or regressions exceeded the allowance
```

`--compare` also accepts a run id or `latest`. The dashboard's compare view renders the same diff — regressed and improved tables with before → after scores. See [Running in CI](/evals/ci) for the full pipeline recipe.