6 Principles of a Successful Analytics Chatbot

At GoodData, we’ve been exploring the potential of AI in analytics since the early days of AI — e.g., template-based Natural Language Queries (NLQ). More on that in the previous article. While a chatbot isn’t the only representation of AI, its potential for analytics is immense, especially when tailored to the right users and use cases.

Our nearly 20 years of experience with analytics and recent experiments with AI have led us to formulate the following principles for successfully using chatbots (not only) in analytics.

This article is part of our AI series, in which we share our journey and progress with AI in analytics.

1. Know Your Audience

Understanding your audience is crucial when building an analytics chatbot. It is tempting to route all the questions directly to a large language model but imagine the variability of questions users will ask. As an example:

• Analytics consumers seek clear answers and insights.
• Business analysts strive for better productivity when seeking these insights.

Covering all the possible questions might become too overwhelming, so first, focus on your core audience and their most prominent tasks. Then, build upon that and provide additional functionality. This approach isn’t new; product managers and even chatbot builders have been taking it for years.

2. Provide Concise Answers

Avoid overwhelming users with too much detail. One of our research assumptions was that analytics consumers desire clear answers to questions, while business analysts want to lead a more technical conversation with intermediate results. This assumption proved to be wrong, and we saw a preference for concise answers that are clear, complete, and yet concise for all types of users.

3. Explain When Requested

While answers should be concise, the chatbot should offer explanations when asked. Providing a UI shortcut for these explanations enhances convenience and feature discoverability.

Gartner calls this concept reasoning, and while it is pretty well-known by the LLM creators, the broad public isn’t used to it yet. The discoverability of such a functionality is the king. I remember a quote from one of our research sessions where the participant, for the first time, noticed the option to reason about the answer. He immediately told us:

“I didn’t ask my chatbot about the explanations, but I will now.”

4. Offer Recommendations for Next Steps

A conversation about data rarely ends with the first question. A helpful chatbot should recommend the following steps to explore additional ideas. Including a UI shortcut for these recommendations improves the overall usability of the conversation.

5. Be Honest About Knowledge Limitations

Avoid hallucinations — situations in which a chatbot responds with a made-up response. An analytics chatbot should be honest about its limitations and suggest alternatives when it doesn’t know the answer. Users appreciate transparency and guidance on alternative solutions.

6. Avoid Making the Chatbot the Primary Interface

Chatbots excel in conversations about data, but they shouldn’t be the primary interface for complex tasks like building dashboards. Users can be thrown into endless conversations better suited to a point-and-click interface. Use the AI to generate initial dashboards or suggest changes, but avoid using chatbots for detailed dashboard construction. Let me show you an example:

Example: Adding a chart on a dashboard

Now, compare it to a simple point-and-click experience.

Other examples of such complex tasks could be a change of dashboard layout or look & feel customizations.

Conclusion

When designing chatbots (not only) for analytics, adhere to these six principles:

1. Know your audience.
2. Provide concise answers.
3. Explain when requested.
4. Offer recommendations for the next steps.
5. Be honest about knowledge limitations.
6. Avoid making the Chatbot the primary interface.

These principles are just part of our insights on chatbots. For further reading, check out the article by my fellow UX designer Vaclav Kocian, Pitfalls of Chat User Interfaces (not only) for Data Analytics.

In the previous article, I was Mapping the history of Chatbots for Analytics.