> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vizra.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Assertions Reference

> Every deterministic check — from substrings to real tool calls, token cost, and multi-turn behavior.

Deterministic assertions run inside `->assert(...)`, per sample, before any judge. The closure receives an assertion proxy, the row, and the full response:

```php Anatomy theme={null}
->assert(fn ($a, $row, $response) => $a
    ->notEmpty()->gate()                 // ->gate(): failure hard-fails the sample, skips judges
    ->contains($row->expected())
    ->toolCalled('lookup_policy')->weight(2.0)   // ->weight(): share of the sample score
    ->costBelow(0.02))
```

Every assertion records its expected and actual values — failures come with receipts, in the test output and the [dashboard](/evals/dashboard). In [class-based evaluations](/evals/class-based-evaluations) the same checks are `$this->assert*()` methods (e.g. `$a->toolCalled(...)` ⇢ `$this->assertToolCalled(...)`).

## Content

| Assertion | Checks |
| - | - |
| `contains($needle, $ignoreCase = true)` | Substring present |
| `notContains($needle)` | Substring absent |
| `containsAnyOf([...])` / `containsAllOf([...])` | At least one / all substrings |
| `startsWith($prefix)` / `endsWith($suffix)` | Response boundaries |
| `matchesRegex('/pattern/')` | Regex match |
| `lengthBetween($min, $max)` / `wordCountBetween($min, $max)` | Size bounds |
| `notEmpty()` | Non-blank response (a classic gate) |
| `isBritishSpelling()` / `isAmericanSpelling()` | Brand-voice spelling conventions |

## Structure

| Assertion | Checks |
| - | - |
| `validJson()` / `jsonHasKey('a.b')` | Response parses as JSON / has a (dot-notation) key |
| `validXml()` / `xmlHasTag('tag')` | XML validity / tag presence |
| `outputHasKey('score')` | Structured-output agents: key exists |
| `outputKey('score', 5)` or `outputKey('score', fn ($v) => $v >= 1)` | Key equals a value, or passes a callback |
| `outputKeyMatches('id', '/^ORD-/')` | Key matches a regex |

## Agent behavior

These inspect the **real `AgentResponse`** — actual tool calls with their arguments, actual steps — not text parsing:

| Assertion | Checks |
| - | - |
| `toolCalled('lookup_order')` / `toolNotCalled('escalate')` | Tool was / wasn't invoked |
| `toolCalledWith('lookup_order', ['id' => 7])` | Called at least once with this **argument subset** (nested arrays supported, extra keys allowed) |
| `toolCallOrder(['lookup_order', 'create_return'])` | Tools appear in this relative order (others may interleave) |
| `stepsBelow(4)` | Agent loop finished in under N steps |
| `finishReason(FinishReason::Stop)` | How generation ended — failures flag `Length` as truncation and `ContentFilter` as refusal |
| `noPendingApprovals()` | Response isn't paused on human-in-the-loop tool approval |

## Usage & cost

| Assertion | Checks |
| - | - |
| `costBelow(0.02)` | Sample cost (USD, from the [price table](/evals/configuration)) |
| `tokensBelow(4000)` | Total tokens across prompt/completion/cache/reasoning |
| `cacheHitRateAbove(0.5)` | Prompt-cache effectiveness |
| `durationBelow(3000)` | Wall-clock milliseconds for the invocation |
| `modelUsed('gpt-5.4')` / `providerUsed('openai')` | Which model/provider actually served it |

## Safety pre-filters

Named honestly — these are wordlist and regex checks, useful as cheap gates, **not** classifiers:

| Assertion | Checks |
| - | - |
| `containsNoBlockedWords([...])` | Built-in blocklist + `evals.safety.blocked_words` + extras |
| `noObviousPII()` | Regex-grade scan: emails, US SSNs/phones, card-shaped numbers, IPv4 |

<Tip>
  Anything needing actual judgment — tone, toxicity in context, factuality — belongs to the [LLM judge](/evals/judge), not a wordlist.
</Tip>

## Scalars

`equals($expected, $actual)`, `true($condition)`, `false($condition)`, `greaterThan($threshold, $actual)`, `lessThan($threshold, $actual)` — for checks you compute yourself from `$row` and `$response`.

## Custom assertions

Implement the contract once, use it everywhere:

```php app/Evals/Assertions/MentionsOrderNumber.php theme={null}
use Laravel\Ai\Responses\AgentResponse;
use Vizra\Evals\Assertions\Assertion;
use Vizra\Evals\Assertions\AssertionResult;
use Vizra\Evals\Dataset\Row;

class MentionsOrderNumber implements Assertion
{
    public function __invoke(AgentResponse $response, Row $row): AssertionResult
    {
        $found = preg_match('/ORD-\d{6}/', $response->text) === 1;

        return AssertionResult::bool(
            'mentions_order_number',
            $found,
            'an ORD-xxxxxx reference',
            $response->text,
            'Response never cites the order number.',
        );
    }
}
```

```php Using it theme={null}
->assert(fn ($a) => $a->with(new MentionsOrderNumber))
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.