Multi-Turn Conversations

Real support and chat agents don’t answer isolated prompts — they answer turn seven, with six turns of context behind them. Vizra Evals replays that context for real.

Multi-turn rows

evals/support.jsonl

{"messages": [
  {"role": "user", "content": "Hi, I ordered a lamp last week"},
  {"role": "assistant", "content": "Thanks for reaching out! How can I help with your lamp?"},
  {"role": "user", "content": "Can I still return it?"}
], "expected": "30 days"}

The final entry must be a user turn — it becomes the prompt. Everything before it is replayed as genuine conversation history through the Laravel AI SDK’s conversation channel, so the agent responds exactly as it would mid-conversation in production: aware the customer bought a lamp, answering “can I return it” correctly.

Replay is real, not prompt-stuffing: prior turns are hydrated into SDK Message objects and delivered through the same mechanism the SDK uses for stored conversations. The agent’s instructions, tools, and middleware all still apply.

Replaying production traffic

If your agents use the SDK’s conversation persistence (RemembersConversations), your agent_conversations tables are a free eval dataset:

tests/Evals/SupportBotTest.php

expect(SupportBot::class)->toPassEval(fn ($eval) => $eval
    ->fromConversations(take: 50)
    ->judge('At least as helpful and accurate as the reply we actually sent.')
);

For each stored conversation of this agent: the latest user turn becomes the prompt, everything before it replays as context, and the reply your agent actually gave becomes $row->expected(). Two powerful patterns fall out:

Judge against the production answer

->judge('Answers the customer at least as well as the reference reply.', min: 7)
// the judge sees $row->expected() as reference information automatically

Regression-test a prompt change against real traffic

->fromConversations(take: 100)
->samples(2)
->gate(maxRegressions: 0)
// first run baselines real traffic; future prompt changes diff against it

The standalone form gives you query refinement: Dataset::fromConversations(SupportBot::class)->latest()->take(50)->where('title', 'like', '%refund%').

Eval runs never write to your conversation tables — replay is read-only by construction, so running evals can’t pollute production history.

Assertion behavior on multi-turn rows

Everything works the same: $row->input is the final user turn, $row->messages holds the prior turns, and all assertions run against the agent’s real AgentResponse. In the recorded results and dashboard, multi-turn rows display as the final question with a “(after N prior turns)” marker rather than a wall of JSON.

Getting Started

Core Concepts

Writing Evals

Running & CI

Dashboard

Migration

Multi-Turn Conversations

Multi-turn rows

Replaying production traffic

Assertion behavior on multi-turn rows

​Multi-turn rows

​Replaying production traffic

​Assertion behavior on multi-turn rows

Multi-turn rows

Replaying production traffic

Assertion behavior on multi-turn rows