Terminal
config/evals.php:
Judge defaults
config/evals.php
Gates and comparison
config/evals.php
gatePolicy()), per Pest eval (->gate(...)), and per CLI run (--min-score etc.). Pass-rate drops always count as regressions regardless of epsilon — see Baselines & Regressions.
Execution
config/evals.php
The price table
Cost tracking multiplies each sample’s tokenUsage against a user-maintained price table:
config/evals.php
cache_read/cache_write optional. Dated model ids reported by providers (gpt-5.4-2026-05-01) automatically match their family entry (gpt-5.4). Unknown models produce a null cost and a single warning — never an error, and never a silently-wrong number. Prices change; this table is yours to keep current.
Safety wordlist
config/evals.php