---
title: "Create and Run Simulator Scenarios"
slug: "create-and-run-simulator-scenarios"
updated: 2026-04-03T00:30:34Z
published: 2026-04-03T00:30:34Z
canonical: "help.gladly.com/create-and-run-simulator-scenarios"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://help.gladly.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Create and Run Simulator Scenarios

> [!NOTE]
> This feature is currently in an Early Access phase
> 
> If you're interested in learning more, contact Gladly Support.

To use the Simulator for test scenarios, you’ll need to understand how to configure each field, run tests, and review results.

## Create a scenario

The Simulator panel opens from within any Gladly Agent page. How you create a new scenario depends on whether you've created one before.

### **Create your first scenario**

When you open the Simulator panel for the first time, the **Getting Started with Scenarios** view provides an overview of key fields: **Customer Goal**, **Additional details**, and **Success Criteria**.

1. From a Gladly Agent page, click **Simulator** in the top navigation.
2. In the Simulator panel, click **Create Manually**.

![Gladly interface showing scenario creation with highlighted options for user guidance.](https://cdn.us.document360.io/7047b671-c4f2-4df0-bb0a-b9b511fd2452/Images/Documentation/create manually simulator(1).png)

### **Create additional scenarios**

Once you've created your first scenario, opening the Simulator panel will display any existing scenarios for that Gladly Agent. Click any scenario to open and edit it, or create a new one.

1. From a Gladly Agent page, click **Simulator** in the top navigation.
2. In the Simulator panel, click **New Scenario**.

![A simulator interface showing scenarios with a highlighted button for creating a new scenario.](https://cdn.us.document360.io/7047b671-c4f2-4df0-bb0a-b9b511fd2452/Images/Documentation/additional scenarios.png)

The scenario form has two sections: **Customer Setup** and **Test Configuration**.

### Customer Setup

**Scenario title (optional)**

Use this section to provide a short label to identify this scenario in the list. If left blank, the Customer goal is shown in the Scenarios list instead.

![Simulator interface showing scenario title and instructions for setting customer goals.](https://cdn.us.document360.io/7047b671-c4f2-4df0-bb0a-b9b511fd2452/Images/Documentation/scenario title.png)

**Customer goal (required)**

Use this section to describe what the simulated Customer is trying to accomplish. Write this in first person, starting with "I want to..."

The Customer goal guides the simulated Customer's behavior throughout the Conversation.

![Customer goal input section with examples for price matching and subscription cancellation.](https://cdn.us.document360.io/7047b671-c4f2-4df0-bb0a-b9b511fd2452/Images/Documentation/customer goal.png)

**Initial message from customer (required)**

Use this section to input the Customer's first message to Gladly. Write it the way a real Customer would engage–short and natural.

![Customer service chat interface showing initial message and examples for customer inquiries.](https://cdn.us.document360.io/7047b671-c4f2-4df0-bb0a-b9b511fd2452/Images/Documentation/initial message.png)

**Additional details from the customer (optional)**

Use this section to include any facts the simulated Customer knows and can share if Gladly asks. Think of this as details the Customer might have available to them, but wouldn’t volunteer without being prompted.

![Customer support information with order details and escalation tips for chatbot interactions.](https://cdn.us.document360.io/7047b671-c4f2-4df0-bb0a-b9b511fd2452/Images/Documentation/additional details from customer.png)

The simulated Customer shares this information only when Gladly asks for it, unless you enable **Allow proactive pushback**.

**Allow proactive pushback (optional)**

When selected, the simulated Customer can volunteer one relevant fact from the **Additional details** section, if Gladly declines a request. This is useful for testing scenarios where a Customer might push back on a policy.

![Instructions for allowing proactive pushback in customer service interactions with bots.](https://cdn.us.document360.io/7047b671-c4f2-4df0-bb0a-b9b511fd2452/Images/Documentation/allow proactive pushback.png)

### Test Configuration

**Customer data available to Gladly (optional)**

Use this section to include any other facts about the Customer or your organization that are true in this scenario. For example, information about order status, account history, or product availability. This differs from **Additional details**, which are things the*Customer*knows and can share.

![Customer data overview with order details and account status for service inquiries.](https://cdn.us.document360.io/7047b671-c4f2-4df0-bb0a-b9b511fd2452/Images/Documentation/customer data avail to gladly.png)

**Success criteria (required)**

Use this section to add a list of yes/no questions that define what a successful interaction looks like. Every criterion must pass for the test to pass overall.

Write each criterion as an objective observer watching the Conversation. Use third person and focus on what's observable in the transcript.

![Guidelines for evaluating AI performance with clear yes/no criteria examples.](https://cdn.us.document360.io/7047b671-c4f2-4df0-bb0a-b9b511fd2452/Images/Documentation/success criteria.png)

Click **Add Criterion** to add multiple criteria.

**Expect handoff (optional)**

By default, the Simulator treats any handoff to a human Agent as a test failure. Check this box when specifically testing to ensure that Gladly AI correctly escalates to a team member for a given scenario.

When selected, the test keeps running until the handoff occurs (or the Conversation reaches its [turn limit](/product-docs/docs/create-and-run-simulator-scenarios#what-happens-during-a-run)). The test passes only when both the success criteria are met and the handoff happens.

![Instruction on enabling handoff for customer service agents in a user interface.](https://cdn.us.document360.io/7047b671-c4f2-4df0-bb0a-b9b511fd2452/Images/Documentation/expect handoff.png)

## Save a scenario

Click **Save** to save the scenario. A "Scenario saved" confirmation appears when the save is successful.

![User interface showing test configuration options and highlighted save button in Gladly simulator.](https://cdn.us.document360.io/7047b671-c4f2-4df0-bb0a-b9b511fd2452/Images/Documentation/click save for sim (2).png)

## Run tests

### Run a single scenario

Click any scenario in the Scenarios list to open it, then click **Run** to run it individually.

![Gladly simulator interface showing test configuration and highlighted run button.](https://cdn.us.document360.io/7047b671-c4f2-4df0-bb0a-b9b511fd2452/Images/Documentation/click run for sim.png)

### Run all scenarios

Click **Run All** from the Scenarios list to test every scenario for a Gladly Agent at once.

![Gladly interface showing scenarios with options to create and run simulations.](https://cdn.us.document360.io/7047b671-c4f2-4df0-bb0a-b9b511fd2452/Images/Documentation/run all.png)

## What happens during a run

When you start a test, the Simulator:

1. Creates a simulated Customer session.
2. Sends the initial message to Gladly.
3. Waits for Gladly to respond.
4. Evaluates whether the success criteria are met.
5. If criteria aren't met, generates a natural follow-up Customer message and continues the Conversation.
6. Repeats until all criteria pass, a handoff occurs, or the Conversation reaches 20 turns.

> [!NOTE]
> The 20-turn limit prevents tests from running indefinitely
> 
> If the Gladly Agent can't satisfy the success criteria within 20 exchanges, the test fails.

> [!WARNING]
> Cancel a run
> 
> You can cancel a running test at any time. The cancellation takes effect after the current step completes.

## Review results

Each completed test shows one of the following outcomes:

| Result | Description |
| --- | --- |
| **Pass** | All success criteria were met (and handoff expectations were satisfied, if applicable) |
| **Fail** | Success criteria were not met after 20 turns, or an unexpected handoff occurred |
| **Error** | Something went wrong during the test (i.e. a technical issue with mock data generation) |
| **Timeout** | Gladly didn't respond within the expected time window |
| **Canceled** | You canceled the test before it completed |

> [!NOTE]
> Pass and Fail are normal test outcomes
> 
> Error, Timeout, and Canceled indicate something interrupted the test before a verdict was reached.

### Reading the results

For each completed run, you can view:

- **Pass/fail status** — The overall test result
- **Conversation transcript** — The full back-and-forth between the simulated customer and Gladly
- **Per-criterion evaluation** — Each success criterion shown individually with its pass/fail status and an explanation

The per-criterion breakdown is especially useful for debugging failed tests. It tells you exactly which criteria weren't met and why.

### View conversation

Click **View conversation** on any test result to open the full Conversation within Gladly Team, where you can review how AI responded turn by turn. Click **Resume in tester** to reopen the Conversation in the Simulator test view.

### Run history

Each scenario stores a history of past runs. This makes it easy to track whether Guide changes improved or regressed test results over time.

*Conversation*

or

*Conversations*

(with a capital "C") provides a holistic view of Customer communications and interactions contained in the Conversation Timeline.