> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vizra.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# CLI Commands

> The same engine without Pest — runs, baselines, scaffolding, and machine-readable output.

Everything the Pest expectation does is also an artisan command — same engine, same tables, same dashboard.

## `evals:run`

```bash Terminal theme={null}
php artisan evals:run SupportQuality              # one class-based suite
php artisan evals:run                              # every discovered suite (app/Evals)
php artisan evals:run --filter=Support             # subset by name
```

| Option | Effect |
| - | - |
| `--samples=5` | Override samples per row |
| `--dataset=path.jsonl` | Run a JSONL file instead of the dataset the evaluation declares — the same suite against different data |
| `--concurrency=1` | Sequential execution (debugging); default from config |
| `--dry-run` | Execute on SDK fakes — zero tokens, validates wiring ([details](/evals/testing-without-tokens)) |
| `--compare=baseline` | Diff against the baseline (also accepts a run id or `latest`) |
| `--baseline` | Promote this run to baseline if its gate passes |
| `--output=json` | Machine-readable versioned document on stdout (progress on stderr) |
| `--report` / `--no-report` | Force or suppress pushing this run to [Vizra Cloud](/cloud/reporting), overriding the ambient configuration |
| `--min-score`, `--min-pass-rate`, `--max-regressions` | Gate overrides |

<Note>
  Unlike Pest, the CLI never promotes a baseline on its own. The first run of a suite has
  nothing to compare against until you pass `--baseline` (or promote a run afterwards with
  `evals:baseline`) — see [Baselines & Regressions](/evals/baselines-and-regressions).
</Note>

**Exit codes** — the CI contract:

| Code | Meaning |
| - | - |
| `0` | Gate passed |
| `1` | Gate failed, or regressions exceeded the allowance |
| `2` | Harness failure (unknown suite, unreadable dataset, run crashed) |

The terminal output is a per-row table — pass counts, score ± spread, cost — plus totals; with `--compare`, regressed/improved tables with before → after scores.

## `evals:baseline`

```bash Terminal theme={null}
php artisan evals:baseline 01kyvrjyxp9ex0f3gtvznmafdn
```

Promotes any completed run to its suite's baseline (transactionally demoting the previous one). See [Baselines & Regressions](/evals/baselines-and-regressions).

## `evals:calibrate`

```bash Terminal theme={null}
php artisan evals:calibrate storage/evals/labelled.jsonl --criteria="..."
```

Judge-vs-human agreement measurement — see [Judge Calibration](/evals/calibration).

## `make:eval`

```bash Terminal theme={null}
php artisan make:eval SupportQuality
```

Scaffolds a [class-based evaluation](/evals/class-based-evaluations) in `app/Evals` plus a starter JSONL dataset in `evals/data/`.

## JSON output

`--output=json` emits a stable, versioned document: run metadata (git sha/branch, config), aggregate summary, every row with its samples and assertion results, the comparison diff, and the gate verdict. Pipe it to artifacts storage, parse it in CI annotations, chart it — the schema is designed to be depended on.

```bash CI one-liner theme={null}
php artisan evals:run SupportQuality --compare=baseline --output=json > eval-report.json
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.