Four Questions to Ask When Getting a Chatbot

Ahh, the much-maligned chatbot. When chatbots emerged on the scene a few years ago, the promise often didn't match the results. In a world of social media and screenshots, no one wants to become a chatbot meme, and our collective lack of experience with artificial intelligence (AI) and robotic process automation (RPA) coupled with the immaturity of tools meant that, in many cases, chatbots were either too unpredictable or too rigid.

It's time to take another look. The next-generation chatbot delivers rapid benefit by guiding customers through self-service, freeing agents for more complex cases, and even helps customer service diagnose service or product weaknesses or areas where existing self-service resources fall short. They are relatively rapid to deploy, easy to track and understand, and take limited skill and time to manage and train.

But how do you know if you're getting a next-generation chatbot, or at least one that meets the AI/RPA promise? Here are four key questions to ask:

1. How long will it take to go live?

Given the maturity of chatbot technology today, the time from project kickoff to a live operational chatbot delivering measurable results should be fewer than six months. If your project plan is longer than this, you're doing it wrong. Next-gen chatbots have enough intelligence that they can be trained on existing knowledgebase content and agent interaction histories in weeks, not months. If your vendor is telling you it will take longer than that, you might be paying for an old-school bot with limited intelligence and lots of custom coding under the hood, which will limit your flexibility and increase support and maintenance costs. If your product or service offering is really broad and complex, breaking up your chatbot project into phases will enable you to gain experience with the technology while ensuring you're not on a long path to mediocre results.

2. Does the chatbot have prebuilt libraries?

Prebuilt libraries, provided they can be configured, can rapidly accelerate training and deployment time and reduce the ongoing maintenance burden of chatbots. Next-gen chatbots have prebuilt libraries for common inquiries (such as lost orders or password resets), and some have industry-specific content.

A consumer goods company we analyzed recently deployed a chatbot to support self-service inquiries, embedding a branded chatbot interface accessible from every page on its Web site. Using prebuilt workflows enabled it to deploy in weeks, and configuration efforts were focused on product-specific complex questions, not the basics.

3. What training or skills are needed to manage it?

If you have a team of data scientists, great! They should be spending their time and skills on more complex issues than managing and updating your chatbot. Customers of next-gen chatbots are learning to manage and coach the application with limited (fewer than three hours) training, and spending a few hours a week managing it.

Beyond chatbot management, look for chatbots that also have analytics and diagnostic tools that help you understand where your current self-service resources fall short. A financial services firm, for example, leverages its chatbot's analytics to quickly see the key themes or content areas that the chatbot conversations don't resolve and use that knowledge to add new FAQ resources for customers and agents.

4. What happens if I add a product or make changes?

Next-gen chatbots can be pointed at new FAQs or content and rapidly configure new conversation flows or be configured to crawl knowledgebase content regularly and update their configuration based on new information they find. In an ideal world, you want a chatbot that does both.

A large media company uses its chatbot's coaching feature to introduce it to new content, like when it has a new pay-per-view event, while timing it to crawl its knowledgebase once a month so it can ensure new content is reflected in the chatbot's vocabulary.

Next-generation chatbots are delivering on the AI/RPA promise, increasing self-service resolution rates by up to 50 percent and driving greater customer satisfaction by giving service managers more intelligence about their self-service successes and failures. While what's under the covers is important, what the vendor has wrapped around the intelligence is more important. Prebuilt libraries, drag-and-drop interfaces, embedded analytics, and intuitive interfaces ensure that you get rapid results and an agile chatbot that can continue to deliver benefits with limited effort.

Rebecca Wettemann is CEO and principal at Valoir (, a technology industry analyst firm focused on the connection between people and technology in a modern digital workplace.