> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vizra.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# CLI Commands

> The same engine without Pest — runs, baselines, scaffolding, and machine-readable output.

Everything the Pest expectation does is also an artisan command — same engine, same tables, same dashboard.

## `evals:run`

```bash Terminal theme={null}
php artisan evals:run SupportQuality              # one class-based suite
php artisan evals:run                              # every discovered suite (app/Evals)
php artisan evals:run --filter=Support             # subset by name
```

| Option                                                | Effect                                                                                                      |
| ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| `--samples=5`                                         | Override samples per row                                                                                    |
| `--dataset=path.jsonl`                                | Run a JSONL file instead of the dataset the evaluation declares — the same suite against different data     |
| `--concurrency=1`                                     | Sequential execution (debugging); default from config                                                       |
| `--dry-run`                                           | Execute on SDK fakes — zero tokens, validates wiring ([details](/evals/testing-without-tokens))             |
| `--compare=baseline`                                  | Diff against the baseline (also accepts a run id or `latest`)                                               |
| `--baseline`                                          | Promote this run to baseline if its gate passes                                                             |
| `--output=json`                                       | Machine-readable versioned document on stdout (progress on stderr)                                          |
| `--report` / `--no-report`                            | Force or suppress pushing this run to [Vizra Cloud](/cloud/reporting), overriding the ambient configuration |
| `--min-score`, `--min-pass-rate`, `--max-regressions` | Gate overrides                                                                                              |

<Note>
  Unlike Pest, the CLI never promotes a baseline on its own. The first run of a suite has
  nothing to compare against until you pass `--baseline` (or promote a run afterwards with
  `evals:baseline`) — see [Baselines & Regressions](/evals/baselines-and-regressions).
</Note>

**Exit codes** — the CI contract:

| Code | Meaning                                                          |
| ---- | ---------------------------------------------------------------- |
| `0`  | Gate passed                                                      |
| `1`  | Gate failed, or regressions exceeded the allowance               |
| `2`  | Harness failure (unknown suite, unreadable dataset, run crashed) |

The terminal output is a per-row table — pass counts, score ± spread, cost — plus totals; with `--compare`, regressed/improved tables with before → after scores.

## `evals:baseline`

```bash Terminal theme={null}
php artisan evals:baseline 01kyvrjyxp9ex0f3gtvznmafdn
```

Promotes any completed run to its suite's baseline (transactionally demoting the previous one). See [Baselines & Regressions](/evals/baselines-and-regressions).

## `evals:calibrate`

```bash Terminal theme={null}
php artisan evals:calibrate storage/evals/labelled.jsonl --criteria="..."
```

Judge-vs-human agreement measurement — see [Judge Calibration](/evals/calibration).

## `make:eval`

```bash Terminal theme={null}
php artisan make:eval SupportQuality
```

Scaffolds a [class-based evaluation](/evals/class-based-evaluations) in `app/Evals` plus a starter JSONL dataset in `evals/data/`.

## JSON output

`--output=json` emits a stable, versioned document: run metadata (git sha/branch, config), aggregate summary, every row with its samples and assertion results, the comparison diff, and the gate verdict. Pipe it to artifacts storage, parse it in CI annotations, chart it — the schema is designed to be depended on.

```bash CI one-liner theme={null}
php artisan evals:run SupportQuality --compare=baseline --output=json > eval-report.json
```