Tips on How to Train a Chatbot for Businesses
If your chatbot cannot understand what people are asking, then it requires chatbot training. Learn the basics and strategies to successfully train your chatbot.
Chatbot training is an essential practice if your chatbot is powered by conversational AI or you plan to make your button-based chatbot become an AI chatbot. Our guide explores the basic steps in chatbot training before actual development and the best practices with conversational AI after the chatbot launch.
This guide will be helpful for businesses who plan to start with conversational AI or a whitelabel chatbot and those who want to improve their AI chatbot’s answer accuracy. You will know why sometimes chatbots fail to understand people’s requests and how to train your bot to understand humans better.
How can chatbots simulate conversations like people?
As it is illustrated below, there are some chatbots that can react when people behave not as we predicted in our scenario.
These chatbots are powered by conversational AI and have not only predefined flows but also the ability to understand free language. Apart from that, these chatbots can remember the context of the conversation and the user’s preferences. They can jump from one point of conversation scenario to another when needed and address random user requests at any moment.
What makes conversational AI so conversational?
These conversational AI chatbots use Natural Language Processing (NLP) to understand people. When you type “Hello,” it is the NLP that lets the chatbot know that you sent a Welcome message, and in this case, the chatbot will likely respond with a return greeting.
Without NLP, a chatbot can’t meaningfully differentiate between the responses, for example, “Hello” or “Where is my order.” NLP helps provide context and meaning to text-based user inputs so that the chatbot can come up with the best response.
Thanks to NLP, a chatbot can handle our day-to-day texting problems like spelling, grammatical errors, and poor language use in general. A chatbot with NLP capabilities can identify spelling and grammatical errors and interpret the intended message despite the mistakes.
Chatbot platforms usually integrate with NLP platforms that allow NLP implementation into a chatbot to power it with conversational AI. NLP helps a chatbot analyze multiple input types from your customers, including text or audio inputs (like from a phone or voice recording). It can also respond to your customers in a couple of ways, either through text or synthetic speech.
The major tech companies like Google, Microsoft, and Amazon offer NLP platforms as a free service, and usually, chatbot platforms are integrated with them.
The basics of NLP for chatbot training
Imagine a user typed in the chatbot:
In order to trigger the correct answer, a chatbot first needs to understand the intent behind the question.
An intent is, essentially, what the user wants to accomplish when they type their request.
In our case, this would be shipping details or order status. If the current user phrase is unclear on what a user wants to accomplish via a chatbot, then a clarifying question will occur.
When a user types a complex question, it has to have some meaningful chunks of information so a chatbot can provide a specific answer. These meaningful chunks are entities.
An entity is a keyword that makes the user's intent more clear. If a user writes "show me my order shipping information" or "show me my order status," the entities are "shipping information" and "order status." When creating an entity, you make what is called an entity type. Entity type defines the type of information you want to extract from user input. Entities can be a location, date, color, size, flavor, etc.
Alright, now the chatbot knows the user’s intent and the entity, but for the final answer, it has to know the context to trigger the correct answer.
The concept of a context is very similar to the natural language context as we know it. It refers to the data around the current user phrase, filling in the missing links. Defining a context lets the bot 'remember' what the user was referring to for longer than just one request. By defining a context or triggers, you specify what must be true for your intent to be triggered.
In our case, there could be two contexts that will trigger different answers:
- Did a user already provide his login, email, phone number, or any information connected with an account?
- Did they already provide the order number?
In the image below, you will see how depending on a different context, a chatbot will respond:
Now, you probably are curious: where does the chatbot training start? Or what does the chatbot training mean? When chatbot experts talk about training and improving NLP, they mean making the AI chatbot understand what users say.
User phrase/user say/user expression etc.
A user say are all the different ways the users may ask the same question and/or get the same answer.
Just look at how many phrases are just for canceling an order:
However, we have to emphasize two crucial things:
- Training chatbots differ depending on the chatbot platform you’re using.
On the BostCrew Platform the training looks like this:
2) Training a chatbot also depends on the language the chatbot speaks.
Suppose you plan to create a multilingual AI chatbot that speaks languages with different linguistic structures like German and Chinese. In that case, we suggest having a consultation with translators on a language structure and how many synonyms exist for each user phrase.
How to choose the questions for your chatbot
The first step in chatbot development is deciding on the chatbot's goal. Choosing the questions for the chatbot is the first part of this step. At this step, you should talk to your team whose workflows you plan to automate with a chatbot.
Customer Support case tips
1) Ensure that your customer support team is on board so they know why you are automating, what you are trying to achieve, and what their role is. Because when your team is aligned, you will be amazed by how many great and creative ideas they'll bring to the table in terms of automation. You will receive much better results than if you brainstorm on your own, thinking that as a manager, you know everything. When speaking about workflows inside the customer support team, find different ways to ask your team what tasks they do every day that don't require much personal attention — things like moving one ticket from here to there.
2) When the team is aligned, you should analyze the data you have gathered. Like the most popular questions, who is your client, what would they prefer as a communication channel, whom they would prefer to communicate with, and what are the current workflows, etc. You can review call logs, scripts, and email chains and analyze FAQ pages. Check your @support or @info Inbox for repetitive requests. Think about the most repeating questions and issues your clients stumble upon. Your goal is to find out the most common gaps and the most repetitive blockers for both clients and the team.
3) Start the automation with far-reaching questions. Some get surprised, but in the e-commerce space, the number one question that can take 30% of all chats is the order status. So by automating just this one question, you will get rid of 30% of requests.
The tips can be applied for other use cases like engagement, marketing, and lead generation, as the general rule is to have the same sort of understanding of what your end-user wants to know or receive.
Chatbot training steps: Make a chatbot understand humans
1) Categorize main customer requests
Create categories that will contain different customer requests on the same topic. For example, you have pulled the information about popular requests from customer service and noticed that most interactions are about a delivery date.
You can group requests like “when my parcel will be delivered?”, “What is the delivery date?”, “when I will receive my order,” etc., into one category – “Delivery info.” These categories will be our intents. Make sure to name them clearly and consistently. You may want to group them into topics; in that case, you can put the topic in the name, for example: ‘Order - status,’ ‘Order - the cancellation,’ etc.
2) Prepare training phrases dataset
When you have created categories with the main requests, you'll need to fill these groups with "user says." If you have a set of sample conversations, you may also extract the same request's various phrasings.
The absolute minimum number of training phrases for effective chatbot training is 5. The optimal number is around 20-25 for the English language. Try to balance the number of user phrases per intent. Keep it more or less the same. If in one intent, the number of phrases is too big while another intent is undertrained, with less than ten phrases – this will cause the chatbot to fail.
3) Decide on a chatbot training approach
This step is better to discuss with a conversational designer as the training approach is based on the chatbot goal, information complexity, industry, and other factors.
To train your agent, you can take several approaches:
- Intent-based approach
- Entity-based approach
- Knowledge base approach
Intent-based approach
When to apply
An intent-based approach is best when user sayings for each request are very different and clear.
Example:
Change my password
Meanwhile, you have only one thing when this type of request is needed. For example, change a password for a website account. However, if there are two or more similar requests like “cancel my order” and “cancel my subscription,” this approach won’t work for you and eventually will lead a chatbot to fail.
❗️ The most common reason for chatbot failures is choosing a bad strategy for chatbot training.
In this case, it’s better to use an entity-based approach.
Entity-based approach
When to apply
The entity-based approach is good for industry-specific bots where requests may be very similar and use mostly the same words while the users want to achieve vastly different results. This approach is also good for slot-filling, like searches with multi-layer filtering or ordering.
What to do
The entity-based approach will require you to define a lot of entities. As has been mentioned before, entities are meaningful chunks of information, depending on which the bot’s answers will differ. You have to create a dataset of all possible similar actions that lead to different outcomes.
For example:
- change the items on my subscription
- change my password
- change my profile information
- change the address to which my orders ship
Use case of entity-based approach
The Bean
Customer: The City of Chicago
Industry: Government, Travel
Use case: Engagement
Platform: Website
The Bean provides information about museums, galleries, events in the city, etc. During the Discovery phase, we identified what questions a chatbot could possibly answer. To find out, we looked at what people were searching on the Choose Chicago website. It was both helpful for understanding the most popular topics and getting some insight into how people searched, their spelling mistakes, what isn’t on a website but people were searching for, etc. Any of this type of information can help enrich the strategy of your virtual assistant.
Knowledge base approach
When to apply
This approach is useful if you want to train a chatbot with tons of information.
What to do
This approach differs the most from the others. In this case, the chatbot uses a knowledge base: a document or a set of documents containing information. That info usually consists of some articles, wikis, etc. For this approach, you just need to define questions, and the software will automatically search and extract the answer.
Use case of knowledge base approach
AILIRA
Customer: Adrian Cartland (Cartland Law)
Industry: Startup, Legal
Use case: Robotic Process Automation (RPA)
Platform: Website
AILIRA spends 30 seconds finding a legal question and 5 minutes generating a legal document (Generating a Will or Creating a Business). This is possible due to the NLP Knowledge base approach because the chatbot software is filled with a lot of legal information. It rapidly scans enormous volumes of data to find an accurate and valuable answer.
4) Define contexts
Defining contexts is crucial if you need your chatbot to maintain that context in certain parts of the dialogue and keep track of the info your users have already input. Firstly, you have to know this information and all possible scenarios yourself.
Prepare a dataset of all of these moments when a chatbot needs to ask additional questions in order to provide answers. Then you or a conversational designer can map all the experiences in the diagram tree. This way, you will have a more structured approach to your chatbot training.
For example, if a user asks your chatbot to find a restaurant nearby a user's location, you have to:
- Prepare how a chatbot will react if a user hasn't provided any location information;
- Prepare a chatbot answer to ask a user to allow location sharing;
- Prepare how a chatbot will respond if a user has provided their location information;
- Prepare how a chatbot will react if there is no restaurant nearby a user's location.
What to do if a chatbot doesn’t understand users
Different platforms provide different analytics. But the number one thing that indicates poor chatbot’s performance at understanding what people are asking is the high frequency of contacting Live Support to solve the exact problem users can solve via chatbot.
In our practice, we monitor a chatbot’s accuracy by looking at how many automated requests were handled by a live agent.
If we see that on a specific part at the conversation flow, that is not “Talk to a live agent,” users choose the agent option, then we review conversations and see what can potentially cause the problem as there could be many reasons.
How to solve problems with an existing AI chatbot
You can use QBox if your chatbot has trouble identifying what users texted. QBox helps you find potential weak spots in your chatbot training, like intents with similar training phrases which may confuse the agent or intents where training phrases are not specific enough.
However, we highly recommend booking a consultation with a conversational designer to better understand your issue. Sometimes, deleting or adding user sayings won't help much. Maybe there are some issues with your chatbot training approach, and there is a need for a total NLP reset.
Additionally, when you are only starting with an AI chatbot, we recommend having a person who will monitor the chatbot's performance during the initial launch period and gather insights on how people ask questions and what the questions are.
Also, this dedicated chatbot person/chatbot expert can take over if the communication process goes wrong. This would help you not lose the lead or potential client and update a knowledge base if users asked a question you haven't covered before.
Useful materials
