Class-Based Evaluations

Inline toPassEval() config covers most evals. Reach for a class when a suite deserves a name of its own, needs a model matrix, or should be runnable from the CLI and the dashboard’s Run button.

Terminal

php artisan make:eval SupportQuality

app/Evals/SupportQuality.php

use App\Agents\SupportBot;
use Laravel\Ai\Responses\AgentResponse;
use Vizra\Evals\Dataset\Dataset;
use Vizra\Evals\Dataset\Row;
use Vizra\Evals\Evaluation;
use Vizra\Evals\Run\Gate;

class SupportQuality extends Evaluation
{
    public int $samples = 3;

    public function target(): mixed
    {
        return SupportBot::class;   // class-string, instance, or Closure(Row): AgentResponse
    }

    public function dataset(): Dataset
    {
        return Dataset::fromJsonl(base_path('evals/support.jsonl'));
    }

    public function evaluate(Row $row, AgentResponse $response): void
    {
        $this->assertNotEmpty()->gate();
        $this->assertContains($row->expected());
        $this->assertCostBelow(0.02);

        $this->judge()
            ->criteria('Answers using only documented store policy.')
            ->minScore(7);
    }

    public function gatePolicy(): ?Gate
    {
        return new Gate(minScore: 0.8, maxRegressions: 0);
    }
}

Inside evaluate(), the current response and row are ambient — assertion helpers take no $response argument, and every helper from the assertions reference exists as $this->assert*().

Model matrices with `across()`

Run the whole dataset against multiple provider/model combos — each becomes its own series in results, comparisons, and the dashboard:

Compare models on your own data

public function across(): array
{
    return [
        ['provider' => 'openai', 'model' => 'gpt-5.4'],
        ['provider' => 'anthropic', 'model' => 'claude-sonnet-5'],
    ];
}

This is how you answer “can we move to the cheaper model?” with your own dataset instead of someone else’s benchmark.

Transform hooks

Rewrite rows before they reach the agent — keeping row identity intact:

Prompt wrapping

public function transform(Row $row): Row
{
    return $row->withInput("Customer message:\n{$row->input}");
}

Use $row->withInput() (and friends) rather than constructing new rows — they preserve the content hash, so baseline comparison still lines up across runs even as your transform evolves.

Running class-based suites

From Pest — recorded under the class name

expect(SupportBot::class)->toPassEval(fn ($eval) => $eval->using(SupportQuality::class));

From the CLI

php artisan evals:run SupportQuality

And from the dashboard’s Run button, which executes discovered classes via a queued job. All three surfaces share the tables, baselines, and history.

Getting Started

Core Concepts

Writing Evals

Running & CI

Dashboard

Migration

Class-Based Evaluations

Model matrices with `across()`

Transform hooks

Running class-based suites

​Model matrices with across()

​Transform hooks

​Running class-based suites

Model matrices with `across()`

Transform hooks

Running class-based suites