Testing Without Tokens

Evals cost tokens — but developing them shouldn’t. Everything runs offline against the Laravel AI SDK’s fakes, which is also exactly how you test eval wiring in your own suite.

Fake the agent

tests/Evals/SupportBotTest.php

use App\Agents\SupportBot;

it('catches missing policy answers', function () {
    SupportBot::fake(fn (string $prompt) => str_contains($prompt, 'refund')
        ? 'Full refunds within 30 days of delivery.'
        : 'Yes, we ship to France.');

    // now run the eval inline — no network, instant
    putenv('PEST_EVALS=1');
    // ...
});

Fakes accept canned strings (consumed in order), closures, or full TextResponse objects with real Usage, Meta, and tool calls — so even costBelow() and toolCalled() assertions are testable offline. SupportBot::assertPrompted(...) then verifies exactly what was sent.

Fake the judge

The judge is an agent too:

Faking judge scores

use Laravel\Ai\Ai;
use Vizra\Evals\Judge\JudgeAgent;

Ai::fakeAgent(JudgeAgent::class, fn () => [
    'score' => 9,
    'reasoning' => 'Sticks to the documented policy.',
]);

Multi-turn rows just work

When the target agent is faked, multi-turn rows route straight to the fake — your canned responses and assertPrompted() checks behave identically for single- and multi-turn rows, and nothing leaks to the network.

Dry runs from the CLI

For class-based suites, --dry-run fakes everything automatically:

Terminal

php artisan evals:run SupportQuality --dry-run

The entire suite executes — datasets parse, assertions run, results persist — with zero tokens. Scores are meaningless (responses are auto-generated placeholders); the point is proving the wiring before you spend. Use it whenever you add rows or touch an evaluation class.

Keep faked runs out of Cloud

A faked agent produces real-looking scores from canned strings. If you use Vizra Cloud, say so, and the run stays out of the history:

Wiring test, not measurement

expect(SupportBot::class)->toPassEval(fn ($eval) => $eval
    ->dataset(base_path('evals/support.jsonl'))
    ->report(false)
    ->assert(fn ($a) => $a->notEmpty())
);

The CLI’s --dry-run is already excluded automatically — this is only needed for Pest evals that fake the agent themselves, because nothing else distinguishes them from a real run. Local recording is unaffected either way.

The pattern to internalize: fakes for wiring, --evals for measurement. The vizra/evals package’s own test suite — 150+ tests — runs entirely on fakes, no API keys anywhere.

Getting Started

Core Concepts

Writing Evals

Running & CI

Dashboard

Migration

Testing Without Tokens

Fake the agent

Fake the judge

Multi-turn rows just work

Dry runs from the CLI

Keep faked runs out of Cloud

​Fake the agent

​Fake the judge

​Multi-turn rows just work

​Dry runs from the CLI

​Keep faked runs out of Cloud

Fake the agent

Fake the judge

Multi-turn rows just work

Dry runs from the CLI

Keep faked runs out of Cloud