Data Analysis and Obsession

Posted on September 20, 2010

Fact finding and data analysis are essential tools, but you don’t want your team to do double the research needed to “be sure”. That’s obsession. You also don’t want them to slack off in critical areas. That’s foolish. Here are some suggestions on where to draw the line.

I hate making mistakes. They waste money, make me look foolish and I feel embarrassed. When I wore a younger man’s clothes, I would tirelessly dig for data, and analyze away. I remember with pride a massive spreadsheet I built in 1983 that held that detailed historical data and projections for three related entities. Recalculating took ten minutes, yet I waited with bated breath to see my new financial future each time.

Some organizations get obsessed with fact finding and analysis, and they keep it up until it’s too late. Too late might mean the decision was imposed upon them. Perhaps the market window closed. Perhaps the product failed horribly before the recall decision was made, and that meant it went legal and political both. These bad outcomes are in addition to the time, cost and effort of analyzing far in excess of what reality requires.

Data and analysis is only the means to an end. The end is a business result. Any data that isn’t needed to make a decision which affects results is a distraction. Any analysis performed that does not guide an action is a distraction. This is about decision support.

Four situations have typically driven me or my clients into deep analysis of data:

1. Unknown cause and effect relationships. When it is unclear why something is happening in the business, and we aren’t sure what levers to use, it’s time to dig into the data. Such digging, however, is done to some level of depth until we find causation, or we give up. If we believe we find causation, we’ll want to leave a standing measure in place to confirm that the lever we found really works.
2. Unknown material facts. Through normal reasoning, we sometimes determine that we just don’t have all the facts. If, with reasonable effort, we can sift the data and come up with solid facts that will inform our decision, there is every reason to start researching. It can, however, be useful to ask ourselves first if we will do anything differently once we have the fact(s) in hand.
3. Lack of clarity around objective (knowable) factors vs. subjective factors. Too many people go off on a quixotic search for answers that cannot exist, or for data that does not exist. Image a continuum of certainty, where on the left are factors that are absolute facts, in the middle factors that are hypothesis, and on the right factors that are merely a guess. The data and analysis process is where we, thorough research and analysis, try to transform guesses into hypothesis, then when possible, hypothesis into facts. Better to identify all the objective factors we can shift toward facts and those we cannot. Those we cannot shift are subjective factors, and once identified, trying to do quantitative research to turn them into supporting facts is a waste of time.
4. Insecurity around a judgment call. Insecurity isn’t always bad. When I’m making a decision without the proper support, I feel insecure. That’s a good sign I’ve not done enough research. But some people are just terrified about making a mistake, and search in vain for the “guarantee”. After your homework is done, often times making the judgment call takes a little courage.
   It takes good judgment and experience to be able to determine when good business practice crosses over into data and analysis obsession.

Strategies for Mitigation

First and foremost, explore and layout the consequences for making a bad judgment, and the benefits of making a superb judgment. This must be shared with the entire team involved in the research. I’m not suggesting any drama here. Be completely real. We don’t want the team to do double the research needed to “be sure”. That’s obsession. We also don’t want them to slack off in critical areas. That’s foolish. Write down your thinking, and break it down into the different areas or decision points that must be considered. Use a rating scale of importance if you like. Discuss and write down what the company’s future will look like in each scenario. What if the new produce to too feature rich and priced at the top of the range? What will happen if pricing is low, but the failure rate is 6%? If that did happen, what corrective action could be taken, and at what cost with what result? Try to imagine the future and live in it, with all the consequences. If the consequences seem small and livable, you’ll stop this process sooner and know that the data and analysis should be simple. If the consequences are large, you’ll keep your imagination running and the data and analysis should be more exhaustive.

Research work is not an all or nothing proposition. As the data is mined, we uncover new discovery opportunities (threads). Sometimes the thread is as easy as a few clicks on a filter, or a new database query, and if so, click away. But other times, the thread will take substantial resources and time to pursue, and that’s the time to stop the research and debate all the “what ifs” related to the situation. Time to become certain that this one thread is worth all the effort, and that the facts and circumstances that you discover will lead to specific actions that will make a difference.

Last minute frenetic research efforts are almost always wasteful, both by not allowing the time to discover what is really needed, and by not allowing the time to think through the research to avoid time and resource sink holes. Start early by planning out what needs to be researched. Then, long before the “project” begins, set up robust systems that collect and order the data. That probably means investing in some computer systems and forcing the company to have disciplined data collection in those areas that will really count when the big judgment calls need to be made. Begin the work well enough ahead of time that you have “slack” built into the timeline, allowing for pauses as necessary, or to chase some unexpected threads should they appear.

Boundaries and budgets surrounding the data collection and analysis are critical as well. The project should be done at a specific time and with a limit on resources. The pressure to come to the right conclusions by a deadline will help focus the efforts of the team to the most important aspects. Given that they know the consequences of bad judgment, they will become more innovative in their methods and will, in most cases, deliver the decision support that was being requested.

I still hate making mistakes, but having matured a bit since 1983 and gained some experience, I value time and agility in business as well, so I moderate the urge for analysis. Figuring out the sweet spot between too little and too much is not easy, but essential.