Ongoing Sidekick Answers Management

Prev Next

Monitor and maintain Sidekick Answers performance

It’s helpful to think of training your generative AI assistant as you would train an Agent. Like ramping a new Agent, we suggest reviewing Sidekick Answers daily for 1-2 weeks after implementation. Like an Agent, once Sidekick Answers is fully launched and situated within your setup, monitor the AI assistant weekly to identify areas for improvement.

Train AI like you would a human Agent to to get better

AI acts as your digital Agent, and you should treat it like your human Agents. Teach AI new information (e.g., ensure you have the correct information documented as Public Answers). You can teach it how to behave or engage with Customers by configuring its voice and tone, words, and terminologies to avoid, and more.

Performance metrics

You’ll want to make sure you are monitoring your Sidekick Answers metrics regularly by using the Sidekick Dashboard.

  1. Open the Sidekick Dashboard.

  2. Take a look at resolutions and engagement by Thread, in particular, “Chat – Answer Thread” and “SMS – Answer Thread.”

    • Do you want Sidekick Answers to engage in more? If so, think about:

      • What other parts of your Chat experience could benefit from Sidekick Answers?

      • Are you transferring too many Conversations?

      • Are there top topics (see the Topics dashboard) for which you still need to write Public Answers?

Maintenance

The metrics above provide you with a helpful gut check as to how Sidekick Answers is performing, while evaluating individual Conversations within the Conversations page offers a more in-depth look at specific Sidekick Answers engagements. For routine maintenance, we suggest analyzing the Conversations page.

  1. Log into Gladly and then click on Guides.

  2. Go to Conversations.

  3. Here, you will see a list of Conversations where Sidekick has engaged with your Customers.

  4. Review a handful of Conversations where Sidekick has engaged. Use the Conversations Page Overview to evaluate responses.

Daily Sidekick Answers Management

Review key metrics and Sidekick Answers engagements on a daily basis following initial implementation. These steps help ensure your assistant is delivering expected results and aligning with your company’s standards in the first 1-2 weeks post-launch:

  1. Review the Sidekick Dashboard by going to Reports > Sidekick Dashboard.

  2. In addition to a general review of the dashboard, pay attention to the following two metrics:

    • Number of Assists in comparison to SMS and/or Chat volume

    • Number of Resolutions

With up-to-date Answers, customers can see ~10-15% of Assisted Conversations resolved following initial Sidekick Answers implementation

Customers may see as high as 25% of Assisted Conversations resolved with consistent augmentation of Public Answers. The best way to increased Number of Assists and Number of Resolutions is to optimize Public Answers.

Once you have a sense of how Sidekick Answers is performing, you can explore individual engagements more deeply on the Sidekick Conversations page.

Review Suggested Answers daily

In instances where Sidekick does not have enough information to fully resolve the Customer’s inquiry and hands off to an Agent, Sidekick will generate a batch of new Answers based on the Agent’s response to the Customer. Carving out time each day to review these recommendations will strengthen your existing Answer repository and improve Sidekick’s ability to respond accurately and completely to Customers.

Manage Sidekick Conversations Page

Before you start

Search Sidekick Answers interactions

As a Gladly Team Manager or Administrator, you can review every Customer interaction Sidekick Answers engaged in through the Conversations page.

This is a great place to understand how Sidekick Answers interacts with Customers, how it responds, and why it hands off certain interactions to agents. This is information you need to understand how Sidekick Answers is performing and where you could make improvements.

On this page, you can search for interactions by entering a customer’s phone number, email address, or name. Icons indicate if Sidekick Answers has interacted with the customer through SMS or Chat.

Quality Check review

If any of the Quality Checks — Knowledge, Hallucination, or Unsupported Actions — fail, Sidekick will automatically handoff the Customer to an Agent.

Quality check results showing failed hallucination check and original bot message details.

Avoid Hallucinations

Hallucinations occur when generative AI attempts to generate a response but Sidekick believes that the response is not accurate because it’s unable to find a suitable Public Answer.

Hallucinations can be avoided by:

  • Ensuring you have Public Answers for as many answers to questions that Customers may ask.

  • Advice is fine-tuned to align with the expected brand tone and overall behaviors you’d like the generative AI assistant to adopt. Review tips for managing advice to ensure it’s configured for success.

  • The list of handoff scenarios includes instances where you want Sidekick to handoff to an Agent instead of attempting to generate a possible inaccurate reply.