Quickstart

This walkthrough takes you from nothing to a recorded, regression-gated eval of a real agent.

Step 1: Install

Terminal

composer require vizra/evals --dev
php artisan migrate

The migration adds three tables (eval_runs, eval_row_results, eval_assertion_results) where every run is recorded. Requirements: PHP 8.4+, Laravel 12+, laravel/ai, and Pest 5.

Step 2: Write a Pest test

Evals are ordinary Pest tests — toPassEval() is just another expectation:

tests/Evals/SupportBotTest.php

use App\Agents\SupportBot;

it('answers support questions from documented policy', function () {
    expect(SupportBot::class)->toPassEval(fn ($eval) => $eval
        ->dataset(base_path('evals/support.jsonl'))
        ->samples(3)
        ->assert(fn ($a, $row) => $a
            ->notEmpty()->gate()
            ->contains($row->expected()))
        ->judge('Answers using only documented store policy.', min: 7)
        ->gate(minScore: 0.8, maxRegressions: 0)
    );
});

SupportBot is any Laravel AI SDK agent — a class implementing Agent with the Promptable trait. A Closure or agent instance works too.

Step 3: Give it data

One JSON object per line:

evals/support.jsonl

{"input": "What is your refund policy?", "expected": "30 days"}
{"input": "Do you ship to France?", "expected": "France"}
{"messages": [{"role": "user", "content": "Hi, I ordered a lamp"}, {"role": "assistant", "content": "How can I help?"}, {"role": "user", "content": "Can I return it?"}], "expected": "30 days"}

input is the prompt. Or provide messages — the final user turn becomes the prompt and the earlier turns are replayed as real conversation context. expected is free-form reference data; any other key lands in $row->meta().

Step 4: Run it

Terminal

./vendor/bin/pest             # skipped — evals cost real tokens, so they never run by accident
./vendor/bin/pest --evals     # runs against the real model

Each row runs 3 times. Deterministic checks run first; the judge only runs on samples whose gates passed. Everything is recorded.

In Pest, the first run that passes its gate automatically becomes the suite’s baseline. No setup — regression detection is armed from your second run onward. (From the CLI it’s explicit: pass --baseline.)

Step 5: Break something, and watch it get caught

Change your agent’s prompt for the worse and run again:

Terminal

FAILED  Tests\Evals\SupportBotTest > answers support questions from documented policy

Eval [pest: answers support questions from documented policy] — score 61.7%, pass rate 33.3% across 9 samples (run 01kyw…).
Gate failed: 2 rows regressed against the reference run (allowed: 0).
  ↓ regressed: "What is your refund policy?" 96.7% → 51.7%
  ↓ regressed: "Can I return it?" 93.3% → 55.0%

Row-level receipts: which inputs got worse, and by how much. That’s the difference between “a test failed” and knowing what to fix.

Step 6: Watch it over time

Terminal

composer require vizra/evals-ui

Visit /evals for score trends per suite, per-sample drill-downs (including the judge’s reasoning for every score), and side-by-side run comparison. See Dashboard.

Next steps

How It Works

What actually happens when an eval runs — and where the data goes.

Assertions Reference

Everything you can check, from substrings to real tool calls and cost.

Datasets

JSONL, CSV, Eloquent — and replaying production conversations.

Running in CI

Fail the build when your agent regresses.

Getting Started

Core Concepts

Writing Evals

Running & CI

Dashboard

Migration

Step 1: Install

Step 2: Write a Pest test

Step 3: Give it data

Step 4: Run it

Step 5: Break something, and watch it get caught

Step 6: Watch it over time

Next steps

How It Works

Assertions Reference

Datasets

Running in CI

​Step 1: Install

​Step 2: Write a Pest test

​Step 3: Give it data

​Step 4: Run it

​Step 5: Break something, and watch it get caught

​Step 6: Watch it over time

​Next steps

How It Works

Assertions Reference

Datasets

Running in CI

Step 1: Install

Step 2: Write a Pest test

Step 3: Give it data

Step 4: Run it

Step 5: Break something, and watch it get caught

Step 6: Watch it over time

Next steps