> ## Documentation Index
> Fetch the complete documentation index at: https://help.statisfy.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Test your digital workers with Claude

> Use the Statisfy MCP server to test a digital worker on real situations before you switch it on or change it: build a test set, run the worker, read what it would have done, and score each case. Nothing is sent.

With the Statisfy MCP server, Claude can test a digital worker for you. It builds a set of situations from your real conversations, runs the worker against them, reads back what the worker would have done, and scores each one against what it should have done. Change a setting, run again, and compare.

Use it before you switch a worker on, after you change its guidelines or knowledge, or to see how it handles the questions your customers actually send.

> [!NOTE]
>
> Testing workers is part of the [Statisfy MCP server](https://help.statisfy.com/mcp/overview.md). Connect **Statisfy** in Claude first: see [Connect to Claude](https://help.statisfy.com/mcp/connecting.md). Your workspace needs AI CSM, and you need the Admin role.

The test sets and runs Claude creates also appear on the worker's **Evaluations** tab in Statisfy, where test sets are called case sets. See [Evaluating a Digital Worker](https://help.statisfy.com/digital-workers/evaluations.md).

## Safe to run on your live workspace

A test run is a rehearsal. The worker reads your real accounts, contacts, and knowledge bases, so its answers reflect your workspace, but:

* **Nothing is sent.** No email, chat reply, ticket, or task leaves the run. Everything the worker would have done is recorded instead.
* **The worker doesn't change.** A run uses the worker's saved settings. To try a change, ask Claude to run the test with different ones: they are recorded against that run only, and the live worker keeps working as before.
* **Customers see nothing.** The situations are copies of real messages. Nobody is contacted.

## How it works

**1. Build a test set**

A test set is a list of situations, each with what the worker should do. Ask Claude to build one from your real conversations, emails, or support tickets. Claude reads them, picks a spread of questions, and writes what a good outcome looks like for each: answer it, or hand it to a person.

Each test keeps a link to the thread it came from, so you can check any result against what really happened. A set holds up to 200 tests.

**2. Run the worker**

Claude starts a run. The worker handles every situation in a full session of its own, as it would live, so a run takes several minutes or more. Claude checks on it and tells you when it's done.

**3. Read what it did**

For each test, Claude reads the worker's reply, whether it handed off to a person and why, and what it looked up to get there, such as which knowledge-base articles it searched.

**4. Score each test**

Claude judges each result against what the worker should have done: was the outcome right, was the answer accurate and backed by what the worker found, did it link the right article, did it ask a clarifying question only when it needed one. The scores are saved with the run, so you can come back to them.

**5. Change and compare**

Ask Claude to run the same set again with a change and compare the two runs case by case. A run can try different guidelines, voice, greeting, how the worker says it's an AI, or knowledge bases. Changes to its tools, skills, or channels can't be tried this way. When you're happy, change the worker itself.

## Two kinds of test

| | Replies | Outreach |
| - | - | - |
| **What it tests** | How the worker answers a message from a customer | Whether the worker reaches out on its own, and what it would write |
| **Each test holds** | The customer's message, who sent it, and optionally what your app says about them (such as their role, the same `context` your in-app token carries) | An account and how it looked on a given date |
| **What good looks like** | Answer it, or hand it to a person, and the reference answer when it should answer | Reach out or not, and why |
| **Needs** | Nothing extra | The worker must have [proactive tasks](https://help.statisfy.com/mcp/overview.md#what-claude-can-do) that apply to the accounts in the set |

## Prompts to try

Replace the names with your workers and accounts.

> [accordion] Build a test set
>
> * *"Build a test set for our Digital CSM from last month's support tickets: 50 real questions, a mix of how-to questions it should answer from the help center and problems it should hand to the team."*
> * *"Add the questions customers asked our digital worker in chat this week to a test set, with what it should have done for each."*
> * *"Write an outreach test set for our Digital CSM from ten at-risk accounts as they looked on the first of the month."*

> [accordion] Run and read the results
>
> * *"Run the help-center test set against our Digital CSM and tell me when it's done."*
> * *"Show me every test where the worker handed off a question it should have answered."*
> * *"For the tests it got wrong, show the customer's question, the worker's reply, and what it searched."*

> [accordion] Score and compare
>
> * *"Score each result: right outcome, accurate, backed by what it found, links the article. Give me the pass rate by type of question."*
> * *"Run the same set again with these guidelines instead, and compare the two runs."*
> * *"Which tests passed in the first run but failed in the second?"*

## What you're allowed to do

* **Admins only.** Building test sets, starting runs, and scoring need the Admin role, like changing a worker.
* **Read-only connections can read.** If you connected without **Make changes**, Claude can read existing test sets, runs, and scores, but can't build a set or start a run.
* **Changes are recorded.** Test sets, runs, and scores Claude creates are in **Settings → Audit logs**, under your name, with **MCP** as the source.

## Troubleshooting

> [accordion] A run stays queued
>
> Runs wait their turn with other background work in your workspace, such as indexing knowledge-base documents. Ask Claude to check again in a few minutes. A queued run is never lost.

> [accordion] A test shows an error instead of a result
>
> The worker's session for that test couldn't run, so there's nothing to judge. It's counted apart from the tests the worker got wrong. To try it again, start a new run: a finished run isn't resumed.

> [accordion] Claude can't start an outreach run
>
> An outreach test needs the worker to have proactive tasks that apply to the accounts in the set. Give the worker a task first, or test its replies instead.

> [accordion] Claude says it can't build a set or start a run
>
> * You may have connected without **Make changes**. Disconnect **Statisfy** in Claude and connect again, with the box ticked.
> * Testing workers needs the Admin role. Check with your Statisfy admin.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.