Counterfactual replay

On any agent tool call in chat, choose Try alternative to re-execute with edited parameters. Original and alternative results appear side by side.

Console

In the agent tool thread panel (CounterfactualReplayPanel / CounterfactualCompare):

1. Expand a completed tool call.
2. Edit JSON parameters.
3. Replay. The worker runs dry-run for side-effect tools.
4. Compare outputs in the two-column view.

Requires agent runs stored with tool traces (default for WorkflowAgent).

Guardrails

• Side-effect tools (send message, HTTP POST, etc.) run in dry-run mode.
• Max one nested counterfactual level per run.
• Stored in agent_runs_counterfactual (D1).

API

POST /rooms/:roomId/agent-runs/:runId/counterfactual
GET  /rooms/:roomId/agent-runs/:runId/counterfactual

SDK

const alt = await client.replayCounterfactual(roomId, {
  runId: originalRunId,
  toolCallId,
  modifiedParams: { query: "alternative search terms" },
});

Related

• Agent observability
• Multi-agent debate