3 common mistakes in data-driven decision making

By Yongxing Deng, co-founder and CTO of Upa real estate technology startup based in Seattle, WA.

As a business leader, you’re often expected to use data to make an informed decision, whether or not your job title includes the word “data.” Everything from the budget to allocate to a marketing campaign, to the number of employees to approve, to the sales projection. However, making data-driven decisions isn’t just a slogan, it’s a tool that has best practices to follow. Here are three common mistakes business leaders make when using data to make decisions.

Skip Data Validation

When presented with a tight schedule (as we often are) and a dataset, it’s tempting to start analyzing the dataset right away. However, your results can only be as useful and informative as the quality of the underlying data, so it is crucial that you devote sufficient time and energy to validating the accuracy of your data set.

When it comes to data validation, start with a skeptical look at the data. Put on your detective hat and try to find the flaws in the data. Use your existing business knowledge to complete the following sentence: If the data is correct, then ______. Then use SQL or Excel to validate those assumptions before doing the actual analysis.

Underestimating the impact of low probability events

Events less likely to occur can sometimes have an outsized impact on the goals you are trying to achieve. For example, while pandemics happen rarely, few businesses around the world have not been significantly impacted by Covid-19 in recent years. As a business leader, it’s impossible for you to foresee every low-probability event that might occur, yet you often have to make a decision anyway. What do you do?

One approach is to ask yourself explicitly: given the length of time the data is available, what might the data not have “seen?” For example, if you have two years of sales data, you can assume that all rare events that occur once a year were likely included in your data. As such, the events do not require special attention to be considered in your analysis. On the other hand, if you only have six months of sales data, you need to work with your team to think about situations that might only happen once a year (seasonality comes to mind ) and use your business judgment to override your data results. Presenting a list of low-probability, high-impact events alongside your analysis can often help your stakeholders make much better decisions.

Neglecting the Power-User Effect in Your Analysis

Let’s say you’re a gym owner and you’re trying to estimate on average how often your members train at your gym. A “simple” way to do it: stand at the front desk, ask the next 20 members who pass by how many times they’ve been to the gym in the past month, and take an average of those 20 responses. Please note that the average you have obtained in this way will not represent your entire population of members. Why? Because a frequent gym participant is much more likely to be interviewed by you than a member who only visits the gym once a month.

When performing product usage analysis, you need to carefully consider whether the methodology you are using leads to biased results in favor of your power users. This doesn’t mean you should ignore the results you get this way, but it does mean you should proceed with caution.

It’s no exaggeration to say that many of our professional lives now revolve around data. As business decision makers, we must treat data analytics as a powerful tool that also has pitfalls and mistakes and serious ways to cause harm. By combining data with our own intuition and constantly challenging our own methodologies, we can maximize the usefulness of data analysis.

Leave a Comment