We have been offering conversation analytics as a means to systematise quality assessment for some time now. As each company has a standard call flow, the basics of the calls are similar and query-based solutions can be built to find calls which either do or do not adhere to the call flow or its components. Now we’re taking it one step further using Generative AI solutions – introducing our new Automatic Quality Score!
What is the Automatic Quality Score solution that Feelingstream offers and how does it work?
The Automatic Quality Score is a Large Language Model (LLM) based systematic automated assessment of call conversations. It evaluates customer service interactions in a simple and straightforward manner, focusing on key aspects of the conversation.
The goal of the Automatic Quality Score is to ensure high-quality customer service with a focus on communication fundamentals.
The main users for the Automatic Quality Score are Team Leaders and Quality Coaches working on systematic assessment of the performance of Agents and Teams in phone calls. The automatic and objective assessment across all calls streamlines the work of training, onboarding, and quality assessment.
Once the calls are loaded into our environment, we process them. We transcribe and take statistical measurements, apply various classifications and evaluations as needed, and attach metadata. Then, we anonymise the conversations using a combination method of Named Entity Recognition (NER) models and black/whitelisting. Only then do we move on to feeding the data to the LLM with the prompt to assess the call on the required key aspects and give it a final score. We work with anonymised conversations, meaning no Personally Identifiable Information (PII) is processed.
What are the key aspects?
With the Automatic Quality Score solution, we focus on five main aspects relevant to customer service conversations: Greeting, Mapping the customer needs, Explaining the next steps, Solving the issue, and Closing.
Note that the requirements are not simply to say “Hi” or “Bye” to the customer – we have set expectatios for each part of the conversation. We expect the Agent to create a welcoming environment, use open questions, and assist the customer in a way that they understand what is being done and why. The Agent should work to solve the customer issue (or escalate and explain why) and in the closing phase reflect on the conversation with a recap of what has been discussed and done, ensuring that the customer leaves the conversation without any confusion or unclarity.
If the issue does not get solved during the call for any reason, we additionally categorise the reasons for the case not being solved. This information can be used as an additional method for analytics for various purposes.
How to use this knowledge?
We score each call per aspect and give the call a combined final score. This enables the Team Leaders and Quality coaches to:
- compare the Agent’s average score for each aspect to the Team or customer service average
- easily locate the calls that offer improvement opportunities (where the Agent lost points, approaching each element separately)
- decide which part of the conversation should be focused on as either a strong point or weakness
- select the calls to evaluate and discuss with the Agents, and give pointers on specific cases
- filter out specific topics or issues that have not been solved – find training needs
- monitor the change in the selected area over time
How does a Team Lead or Quality Coach see this data?
When using the Automatic Quality Score service, the company is given access to specific views to help explain and understand the data. This includes templates for the Team Leads and Quality Coaches to use to filter out their teams and Agents for monitoring and analytics.
These views are designed to explain the aspects and give easy access to information about average scores, events per each score, and monitoring methods.
Who else can benefit from the Automatic Quality Score?
Analysing NPS scores and customer conversations side-by-side, we have seen that the satisfaction of the customer is most importantly impacted by whether their issue was solved during the conversation. Therefore, issue resolution is a crucial point of the conversation and a focal point of the Automatic Quality Score.
As mentioned above, in the case that the customer issue was not resolved during the conversation, we give a categorical classification label to the conversation explaining the reason why the issue was not solved. Sometimes the process requires multiple contacts or follow-ups. Other times it is miscommunication that happens during the calls and in occasions, it indicates need for Agent training.
Combining this information with the reason for the call from our Automatic Summary solution, the assessment creates insights not only for quality assessment purposes, but can be used very effectively by Process Owners, Product Managers, and others.
Note that these evaluations can also be combined with existing metadata, classifiers, and textual searches from the conversations. Currently, the Automatic Quality Score solution can be applied to phone calls, but this can also be implemented for chats if the need arises.
If you want to see how the Automatic Quality Score can help you streamline coaching, improve customer service quality, or simply grant visibility to the service quality of 100% of phone calls, reach out to us and request a demo call here.
