Three Reasons Why Your Analytics Are Failing

It’s an unfortunate reality: Some data analytics projects do fail. But when they fail, they almost never fail in the way you expect. At the start of an analytics effort, most project leaders lose sleep imagining the technical problems that might occur, but between project planning and support from your analytics vendor, it's almost never the technical issues that result in failure.

Instead, what typically happens is that analytics projects fail to thrive. They don't attract the new users that were expected, they don't generate the revenues promised, or they don't deliver the kind of insights that were expected. These issues are far more daunting than technical ones, and there’s usually no clear-cut solution.

But there is good news: With the right strategy, these problems are solvable. In my experience, I’ve seen that those struggling with analytic efforts tend to share some common root causes, and I’ve seen which tactics work best for remedying them.

1. Building an application for you instead of the customer

The biggest mistake you can make in implementing your analytics is to design them for yourself, not the actual user. It doesn't matter if your user is a paying customer outside your company or one of your company's employees that will be using the analytics themselves. Your perspectives and needs are often very different from your users’ needs. And don’t forget that each of those users has different needs as well based on their function.

Early in my career, I joined a business that was looking to turn its huge volume of data into paid insights for its customers. The product lead had strong opinions about the analytics we should be creating for our users and I took those opinions as fact and used them as the basis for the data product. However, that meant that we built the analytics based on the opinions of our own team of savvy engineers and technologists.

Our customers, however, didn't share the same perspective. When they used our product, it was confusing to them, and adoption remained low until we wised up and built functionality based on their pain points and not our technology goals.

How can I solve this problem?

The best way to solve the problem of analytics designed for you and not the users is to avoid the situation in the first place. Start your project with these steps:

1. Identify the 2-3 key personas to be served when you launch
2. Interview potential users who represent those personas and find out what they do in their jobs and how they solve problems today without analytics
3. Create a diagram linking the persona, what they do, and the problems they face
4. List the insights that the analytics can provide to solve the problems you identified

If you’ve gotten too far into development or already launched, recovery is still possible. In this situation, keep the "designed for us dashboards" up and running, but immediately start following the four steps outlined above. Once you've linked personas to workflows to problems, try to arrange your existing analytics by persona and problem. Often you'll find that some portion of the analytics can be re-ordered and re-used. If not, at least you've defined the roadmap for the next iteration.

2. Focusing on features instead of solutions

Analytics platforms these days are feature-rich. In fact, they have so many features that analytics teams often start their search for a vendor by creating a checklist of features. These checklists can be useful as a basis for comparison, but they can also lead to the second big problem that causes analytics projects to fail: a focus on features, not solutions.

As an example, think of early iPhones and early Android phones. Android commercials focused on features, like a 2.4GHz Snapdragon processor or SD card slot. On the other hand, Apple commercials focused on how the product would solve problems and enhance the buyer’s life, not on listing out specifications—and we all know how well that has worked out.

Analytics that were designed in an attempt to check boxes on a features list tend to have tons of options, filters, and methods of analysis but they don’t guide the user along the path from problem to solution. Many users struggle to navigate this own their own and will simply give up on the analytics altogether.

How can I solve this problem?

An easy way to determine if you've focused on features instead of problem-solving is to try and find patterns in your dashboards that guide the user to an understanding of where issues may be occurring. Do you have lots of insights but no clear path for the user to follow? You might have a problem.

If you've built dashboards with many features but no emphasis on solutions and actions to take, re-organization is the answer. For each persona you're serving, list the problems they're trying to solve and the analytics that may help them do so. Re-arrange your analytics into a workflow based on each persona’s problems. This will result in multiple dashboards (perhaps one per persona) each with various tabs or sections (one per problem to be addressed).

In future development, focus your efforts on the roadblocks preventing the user from finding solutions to the problems rather than adding new functionality that isn't directly tied to a user pain point. When you structure your analytics into this type of workflow, you’ll reduce the mental load on the user as you guide them to a solution and reinforce your product's position as the trusted analytics advisor. The end result: increased user engagement with your analytics.

3. Launching the application and then forgetting about it

Analytics must change over time to keep pace with the evolving needs of users. Even if you were 100% correct about which analytics your customers needed on launch day, this likely won't be true a year later.

Your customer's needs will evolve as their business and processes change and as they become more proficient at using your analytics. Unfortunately, many analytics teams forget this. Analytics are frequently a side project performed in addition to the "core product" work and launch day is often viewed as the finish line.

Imagine that you decide to run a marathon, so you spend months training, buying the right shoes, and learning good technique. On race day, you pin your number to your shirt, line up and wait for the starting gun to fire. Upon hearing the pistol, you take your first three steps and then head to the sidelines. Good race!

As ridiculous as this sounds, it’s essentially how many people think of their analytics. They spend a lot of time and energy getting to launch day, then relax after the analytics are released. As a result, analytics tend to spike in early user engagement, and then slowly decline in use as users find flaws and become frustrated.

How can I solve this problem?

Fortunately, this is a straightforward problem to solve: Treat your analytics like a business-essential product line and give them the care and attention they deserve. Actually changing the way you work to accomplish this is the hard part.

After launch, dedicate resources specifically to the analytics "product." Monitor user behavior, read every analytics support ticket or feature request, and start developing themes for future improvement. Even better, go talk to your users. Sit and watch how they use your analytic application to solve their problems or improve business processes. You'll likely witness gaps and ideas for future development firsthand using this technique.

Building great analytics isn't hard, but it does require discipline and attention to the non-technical side of the project. If your analytic application is failing to thrive and isn't delivering the results you expected, take a close look at the three areas described above. By taking note of the problems and then taking the right actions, you can turn a failing project into a success.