> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vizra.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Assertions Reference

> Every deterministic check — from substrings to real tool calls, token cost, and multi-turn behavior.

Deterministic assertions run inside `->assert(...)`, per sample, before any judge. The closure receives an assertion proxy, the row, and the full response:

```php Anatomy theme={null}
->assert(fn ($a, $row, $response) => $a
    ->notEmpty()->gate()                 // ->gate(): failure hard-fails the sample, skips judges
    ->contains($row->expected())
    ->toolCalled('lookup_policy')->weight(2.0)   // ->weight(): share of the sample score
    ->costBelow(0.02))
```

Every assertion records its expected and actual values — failures come with receipts, in the test output and the [dashboard](/evals/dashboard). In [class-based evaluations](/evals/class-based-evaluations) the same checks are `$this->assert*()` methods (e.g. `$a->toolCalled(...)` ⇢ `$this->assertToolCalled(...)`).

## Content

| Assertion                                                    | Checks                              |
| ------------------------------------------------------------ | ----------------------------------- |
| `contains($needle, $ignoreCase = true)`                      | Substring present                   |
| `notContains($needle)`                                       | Substring absent                    |
| `containsAnyOf([...])` / `containsAllOf([...])`              | At least one / all substrings       |
| `startsWith($prefix)` / `endsWith($suffix)`                  | Response boundaries                 |
| `matchesRegex('/pattern/')`                                  | Regex match                         |
| `lengthBetween($min, $max)` / `wordCountBetween($min, $max)` | Size bounds                         |
| `notEmpty()`                                                 | Non-blank response (a classic gate) |
| `isBritishSpelling()` / `isAmericanSpelling()`               | Brand-voice spelling conventions    |

## Structure

| Assertion                                                           | Checks                                             |
| ------------------------------------------------------------------- | -------------------------------------------------- |
| `validJson()` / `jsonHasKey('a.b')`                                 | Response parses as JSON / has a (dot-notation) key |
| `validXml()` / `xmlHasTag('tag')`                                   | XML validity / tag presence                        |
| `outputHasKey('score')`                                             | Structured-output agents: key exists               |
| `outputKey('score', 5)` or `outputKey('score', fn ($v) => $v >= 1)` | Key equals a value, or passes a callback           |
| `outputKeyMatches('id', '/^ORD-/')`                                 | Key matches a regex                                |

## Agent behavior

These inspect the **real `AgentResponse`** — actual tool calls with their arguments, actual steps — not text parsing:

| Assertion                                                  | Checks                                                                                           |
| ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------ |
| `toolCalled('lookup_order')` / `toolNotCalled('escalate')` | Tool was / wasn't invoked                                                                        |
| `toolCalledWith('lookup_order', ['id' => 7])`              | Called at least once with this **argument subset** (nested arrays supported, extra keys allowed) |
| `toolCallOrder(['lookup_order', 'create_return'])`         | Tools appear in this relative order (others may interleave)                                      |
| `stepsBelow(4)`                                            | Agent loop finished in under N steps                                                             |
| `finishReason(FinishReason::Stop)`                         | How generation ended — failures flag `Length` as truncation and `ContentFilter` as refusal       |
| `noPendingApprovals()`                                     | Response isn't paused on human-in-the-loop tool approval                                         |

## Usage & cost

| Assertion                                         | Checks                                                          |
| ------------------------------------------------- | --------------------------------------------------------------- |
| `costBelow(0.02)`                                 | Sample cost (USD, from the [price table](/evals/configuration)) |
| `tokensBelow(4000)`                               | Total tokens across prompt/completion/cache/reasoning           |
| `cacheHitRateAbove(0.5)`                          | Prompt-cache effectiveness                                      |
| `durationBelow(3000)`                             | Wall-clock milliseconds for the invocation                      |
| `modelUsed('gpt-5.4')` / `providerUsed('openai')` | Which model/provider actually served it                         |

## Safety pre-filters

Named honestly — these are wordlist and regex checks, useful as cheap gates, **not** classifiers:

| Assertion                       | Checks                                                              |
| ------------------------------- | ------------------------------------------------------------------- |
| `containsNoBlockedWords([...])` | Built-in blocklist + `evals.safety.blocked_words` + extras          |
| `noObviousPII()`                | Regex-grade scan: emails, US SSNs/phones, card-shaped numbers, IPv4 |

<Tip>
  Anything needing actual judgment — tone, toxicity in context, factuality — belongs to the [LLM judge](/evals/judge), not a wordlist.
</Tip>

## Scalars

`equals($expected, $actual)`, `true($condition)`, `false($condition)`, `greaterThan($threshold, $actual)`, `lessThan($threshold, $actual)` — for checks you compute yourself from `$row` and `$response`.

## Custom assertions

Implement the contract once, use it everywhere:

```php app/Evals/Assertions/MentionsOrderNumber.php theme={null}
use Laravel\Ai\Responses\AgentResponse;
use Vizra\Evals\Assertions\Assertion;
use Vizra\Evals\Assertions\AssertionResult;
use Vizra\Evals\Dataset\Row;

class MentionsOrderNumber implements Assertion
{
    public function __invoke(AgentResponse $response, Row $row): AssertionResult
    {
        $found = preg_match('/ORD-\d{6}/', $response->text) === 1;

        return AssertionResult::bool(
            'mentions_order_number',
            $found,
            'an ORD-xxxxxx reference',
            $response->text,
            'Response never cites the order number.',
        );
    }
}
```

```php Using it theme={null}
->assert(fn ($a) => $a->with(new MentionsOrderNumber))
```