- Check a new worker before you set it to Active.
- Test a change to the greeting, persona voice, guardrails, disclosure mode, or knowledge bases before you save it.
- Track whether the worker still answers, hands off, or reaches out the way your team expects.
Case sets
A case set is a named list of cases with an expected outcome for each. Every set is one of two kinds:View case sets
On the Evaluations tab, click Case sets. The list shows each set’s Name, Kind, number of Cases, Created date, and Created by. Hover over a name to see the set’s description. Click a set to open it. The set page lists its cases:- Proactive sets show each case’s Account, As of date (the day the case is replayed as), Expected outcome, Rationale, and Source thread (when the thread it came from happened).
- Reactive sets show each case’s From, Message, Expected outcome, Reference reply (the reply the author would accept), and Source thread.
Create a set case by case
- On the Case sets page, click New set.
- Enter a Name (for example, Quiet accounts, 45+ days).
- Choose Proactive or Reactive, and add an optional Description. After you add the first case, the kind is locked.
- Fill in the case and click Add case:
- Proactive: choose an Account (required), an optional Contact the worker would write to, the Expected outcome (Reach out or Stay quiet), and an optional Rationale (for example, quiet 60 days, renewal in March).
- Reactive: enter the customer’s Message (required), an optional From name and Account, the Expected outcome (Answer or Hand off), and an optional Reference reply.
- Repeat for each case. Use the edit and remove buttons on a case card to change the list.
- Click Create set. Statisfy opens the new set.
Create a set with an assistant
An AI assistant connected to Statisfy over MCP can build a set from your workspace’s own threads and accounts. The worker is then measured on work it actually sees.You need an assistant, such as Claude, connected to the Statisfy MCP server under your own user, with Make changes in Statisfy allowed. See Connect to Claude.
- On the Case sets page, click Write with an assistant.
- Under Connect an assistant, check that your connection is listed. If no assistant is connected for you yet, copy the server URL shown, add Statisfy as a connector in your assistant, connect, and come back.
- Under Give it the brief, choose Proactive or Reactive.
- Click Copy brief and paste it into your assistant. Copy brief stays disabled until a connection is listed.
- Click Done. The new set appears in the Case sets list once the assistant has created it.
- Proactive: a mix of accounts the worker covers that warrant a check-in (quiet for 45 days or more, a renewal within 120 days, an open issue) and accounts that should be left alone.
- Reactive: real customer questions from the last 90 days, each paired with the reply your team actually sent, mixing questions that were answered with ones that were handed off.
Start a run
- On the Evaluations tab, click Run evaluation. You can also click Run this set on a set page to open the form with that set selected.
- Choose Proactive or Reactive, then pick a Case set.
- Read the readiness check below the set. Ready confirms how many accounts or cases will run against the worker. Nothing is sent. A set with no cases can’t be picked.
- Optionally enter a Label (for example, Warmer opening). Runs are listed by label, so say what you changed.
- Optionally change settings under Settings for this run (see below).
- Click Start run. Statisfy queues the run and opens it. If something blocks the run, the readiness check changes to Can’t run yet and lists why. For example, a proactive run needs the worker to have proactive tasks that apply to the set’s accounts.
Override settings for one run
Settings for this run lets you try a change and see what the worker would do with it, without saving it to the worker. You can override five settings:- Greeting
- Persona voice
- Guardrail instructions
- AI disclosure mode
- Knowledge bases
Read results
Run statuses
The runs list shows each run’s label (or Untitled run), Set, Kind, Size, Result, Mean score, Started, Took, and Started by. The Result column follows the run’s status:
Each case is a full worker session, so a run takes a few minutes per case. You can close the page; the list updates on its own.
Run summary
Open a run to see summary tiles, the list of cases, and the settings the run used (not the worker’s current settings).- Proactive runs: Would contact, Left alone, Good enough to send (passed judged drafts), and Errors.
- Reactive runs: Passed, Mean score (out of 5), Handed off (hand-offs and loop-ins), and Errors.
Judge verdicts
Once an assistant scores the run, each case shows its verdict. A run can be scored by more than one judge.- Passed or Failed, with a Score from 1 to 5 and the judge’s Reason.
- Not judged when nobody has scored the case yet. A case that errored shows a dash.
Outcomes
The outcome is what the worker did with a case, before any judgement. For Reactive runs:
For Proactive runs, the outcome is Drafted (the worker would reach out, and the draft is shown), Left alone, or Error.
Related
Configuring a Digital Worker
Set the persona, knowledge, tools, and autonomy a run tests.
Statisfy MCP Server
Connect an assistant to Statisfy to write case sets and score runs.