Reporting Runs

With a key configured, a finished run is pushed to Cloud automatically. There is no flag to remember and no separate cloud package — reporting is built into vizra/evals.

Which surfaces report

Surface	Reports
`./vendor/bin/pest --evals`	Yes
`php artisan evals:run`	Yes
`php artisan evals:runner` (the Run button)	Yes
`php artisan evals:run --dry-run`	Never
Any eval marked `->report(false)`	Never

Dry runs are excluded deliberately: their numbers come from faked agents and would sit in the history looking exactly like real ones.

Reporting cannot change a run’s outcome

A run that passed its gate passed it whether or not the network was up. So every failure in the reporter is returned as a message to print, never thrown:

One retry, because the common failure is a dropped connection.
A 4xx is never retried — sending the same rejected document again cannot start working.
A failed push writes a line to stderr and nothing else.

A flaky uplink cannot turn a green build red, and reporting successfully cannot turn a failed gate green.

Failing runs are reported too, before the test is failed. A run that got worse is the one you most need in the dashboard — a history of only your successes would answer no useful question.

Excluding a wiring test

Evals that fake the agent produce real-looking scores from canned responses. Keep them out of the history:

expect(SupportBot::class)->toPassEval(fn ($eval) => $eval
    ->dataset(base_path('evals/support.jsonl'))
    ->report(false)          // wiring test, not measurement
    ->assert(fn ($a) => $a->notEmpty())
);

From the CLI, --no-report does the same for one run. --report forces a warning when no key is configured, which is useful in CI to catch a missing secret.

What is sent

The run’s metadata (suite, status, git sha, branch, timestamps, config snapshot), the aggregate summary, every row’s scores — and, unless you turn them off, the samples: response text, structured output, tool calls, token usage, finish reason, cost, and every assertion’s expected/actual plus the judge’s reasoning.

A run in Cloud: score, dataset-comparison notice, per-row results and Copy for AI

Sending less

.env

VIZRA_CLOUD_SAMPLES=false

You keep scores, pass rates, trends and baselines. You lose the drill-down, the judge’s reasoning, Copy for AI, and dataset editing — which derives from reported samples and has nothing to work from without them. Worth weighing honestly: the sample detail is both the bulk of the storage and the whole of the compliance surface. If your eval inputs contain customer data, this is the switch.

Environments

Runs are filed under an environment so a laptop run and a CI run are not the same number.

Anything running in CI is ci, regardless of APP_ENV — CI boxes almost always set APP_ENV=testing, and filing every CI run under “testing” would bucket them with someone’s local test run.
Otherwise the app’s environment is used.
Override with VIZRA_CLOUD_ENVIRONMENT.

GitHub Actions, GitLab CI, CircleCI and Buildkite are detected automatically from the variables they set, so each run carries its build URL, branch and pull-request number with nothing to configure.

Configuration

config/evals.php

'cloud' => [
    'endpoint' => env('VIZRA_CLOUD_ENDPOINT', 'https://vizra.ai/api/v1/runs'),
    'key' => env('VIZRA_CLOUD_KEY'),
    'samples' => env('VIZRA_CLOUD_SAMPLES', true),
    'timeout' => env('VIZRA_CLOUD_TIMEOUT', 15),
    'environment' => env('VIZRA_CLOUD_ENVIRONMENT'),
    'auto_schedule' => env('VIZRA_CLOUD_AUTO_SCHEDULE', true),
],

auto_schedule is covered in Running from the dashboard.

Getting Started

Using Cloud

Account

Reporting Runs

Which surfaces report

Reporting cannot change a run’s outcome

Excluding a wiring test

What is sent

Sending less

Environments

Configuration

​Which surfaces report

​Reporting cannot change a run’s outcome

​Excluding a wiring test

​What is sent

​Sending less

​Environments

​Configuration

Which surfaces report

Reporting cannot change a run’s outcome

Excluding a wiring test

What is sent

Sending less

Environments

Configuration