Vizra Evals

Your AI agent passed yesterday. Vizra Evals is how you prove it didn’t get worse today. Pest tells you whether a test passed this run. Vizra Evals records every run — sampled scores, pass rates, LLM-judge reasoning, tool calls, token cost — so you can hold a baseline, fail CI when quality regresses, and watch your agent trend over time. Built for agents on the official Laravel AI SDK.

tests/Evals/SupportBotTest.php

use App\Agents\SupportBot;

it('answers support questions from documented policy', function () {
    expect(SupportBot::class)->toPassEval(fn ($eval) => $eval
        ->dataset(base_path('evals/support.jsonl'))
        ->samples(3)
        ->assert(fn ($a, $row) => $a
            ->notEmpty()->gate()
            ->contains($row->expected())
            ->costBelow(0.02))
        ->judge('Answers using only documented store policy.', min: 7)
        ->gate(minScore: 0.8, maxRegressions: 0)
    );
});

Terminal

./vendor/bin/pest             # evals skipped — zero tokens, zero cost
./vendor/bin/pest --evals     # evals run against the real model, every result recorded

Quickstart

Your first recorded eval — from install to a baseline-gated CI check in five minutes.

Baselines & Regressions

The core idea: every run compared against the run you trusted, row by row.

The LLM Judge

Structured-output scoring with persisted reasoning — and calibration against human labels.

Assertions Reference

30+ checks against the real AgentResponse: tool calls, cost, tokens, multi-turn.

Dashboard

Score trends, per-sample drill-downs, judge reasoning, and run comparison in your browser.

Datasets

JSONL, CSV, Eloquent — or replay real production conversations.

Why recorded evals?

Agents are nondeterministic

One sample proves nothing. Every row runs N times; results are pass rates and score distributions, not single booleans.

Pass/fail is a policy, not a verdict

Scores are the primary output. Gates — minimum score, pass rate, zero regressions — are applied at the run level, where they belong.

A run that isn't persisted is worthless

Every run, sample, assertion, and judge rationale lands in your database. Six weeks from now you can answer “when did quality drop, and on which inputs?”

Cheap checks first, expensive checks last

Deterministic assertions run before LLM judges; gate failures skip the judge entirely. Broken samples never burn judge tokens.

Using pestphp/pest-plugin-evals? The two coexist happily — see Vizra Evals vs pest-plugin-evals. Coming from the Vizra ADK evaluation system? See the migration guide.

Getting Started

Core Concepts

Writing Evals

Running & CI

Dashboard

Migration

Quickstart

Baselines & Regressions

The LLM Judge

Assertions Reference

Dashboard

Datasets

Why recorded evals?

Agents are nondeterministic

Pass/fail is a policy, not a verdict

A run that isn't persisted is worthless

Cheap checks first, expensive checks last

Quickstart

Baselines & Regressions

The LLM Judge

Assertions Reference

Dashboard

Datasets

​Why recorded evals?

Agents are nondeterministic

Pass/fail is a policy, not a verdict

A run that isn't persisted is worthless

Cheap checks first, expensive checks last

Why recorded evals?