> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vizra.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Datasets

> JSONL, CSV, arrays, Eloquent — or your actual production traffic.

A dataset yields rows; each row is one thing your agent should handle well. Pass anything below to `->dataset(...)` in Pest, or return it from a class-based evaluation's `dataset()` method.

## JSONL — the preferred format

One JSON object per line, streamed lazily (large files never load fully into memory):

```json evals/support.jsonl theme={null}
{"input": "What is your refund policy?", "expected": "30 days"}
{"input": "Do you ship to France?", "expected": "France", "category": "shipping"}
{"messages": [{"role": "user", "content": "Hi, I ordered a lamp"}, {"role": "assistant", "content": "How can I help?"}, {"role": "user", "content": "Can I return it?"}], "expected": "30 days"}
```

Recognised keys:

| Key           | Meaning                                                                                                                                                                                      |
| ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `input`       | The user prompt (single-turn rows)                                                                                                                                                           |
| `messages`    | `[{role, content}, …]` — the final entry must be a `user` turn and becomes the prompt; earlier turns are replayed as conversation context ([multi-turn](/evals/multi-turn))                  |
| `expected`    | Free-form reference data — a string, or an object read with `$row->expected('key')`                                                                                                          |
| `hash`        | Optional. Overrides the computed [content hash](#row-identity) with one you supply — for datasets that already know their row identities. Must be a sha256 hex digest or the row is rejected |
| anything else | Lands in `$row->meta('key')`                                                                                                                                                                 |

<Warning>
  `hash` is identity, not data — a column of that name is **not** available as meta, and a
  value that isn't a 64-character sha256 digest aborts the read. Rename the column if you
  meant it as your own field.
</Warning>

Malformed lines fail loudly with their line number — no silent row-dropping.

## CSV

```csv evals/support.csv theme={null}
prompt,expected,category
"What is your refund policy?","30 days",billing
"Do you ship to France?",,shipping
```

Header row required; `prompt` and `expected` columns by default (both configurable via `Dataset::fromCsv($path, inputColumn: ..., expectedColumn: ...)`); other columns become meta. CSV exists so non-developers can own datasets in a spreadsheet.

## Arrays and Eloquent

```php Inline rows theme={null}
->dataset([
    'A bare string is a prompt',
    ['input' => 'What is your refund policy?', 'expected' => '30 days'],
])
```

```php From your database theme={null}
use Vizra\Evals\Dataset\Dataset;

->dataset(Dataset::fromEloquent(
    Ticket::query()->where('resolved', true),
    fn ($ticket) => ['input' => $ticket->question, 'expected' => $ticket->resolution],
))
```

## Production conversations

The differentiated one — build the dataset from conversations your agent **actually had**, stored by the Laravel AI SDK's conversation feature:

```php Replay real traffic theme={null}
expect(SupportBot::class)->toPassEval(fn ($eval) => $eval
    ->fromConversations(take: 50)
    ->judge('At least as helpful as the reply we actually sent.')
);
```

Each stored conversation becomes a multi-turn row: the latest user turn is the prompt, prior turns replay as context, and the reply your agent actually gave is exposed as `$row->expected()` — ideal for judging new prompts against real production answers. The standalone `Dataset::fromConversations(SupportBot::class)` form adds `->latest()`, `->take(n)`, and `->where(...)` refinement.

## Row identity

Every row gets a **content hash** over input + messages + expected (not meta). It's how [baseline comparison](/evals/baselines-and-regressions) joins rows across runs — stable across file reordering, unaffected by meta changes, changed only when the row's actual content changes.