Paralysis by Analysis

Posted by Josh Woodard

Wednesday, May 05, 2010


There is still a dispute about who said, "If you can't measure it you can't manage it," but no matter who it was, it is a great insight. However, measuring can get in the way of doing. You should be making and selling first, measuring second.

When you are measuring, disecting, analyzing, regressing (think numbers geek), slicing, and dicing you can do too much. Every metric you can obtain from your website/business has value. Some metrics are more valuable than others; more importantly some metrics are more valuable than the time/money/effort it takes you to measure and analyze them. When the metric is less valuable than the acquisition cost you are wasting resources.

Here are two real world scenarios:

  • Let's say you have a complicated product or service where every engagement with a customer is unique. It usually takes a few phone calls, a visit, and some negotiation to attract good clients. You have determined that your offering cannot reasonably be productized and sold directly via the Internet, but you recognize that the web is still valuable in your sales process. So, you get an incredible website with great calls-to-action; it's very compelling. You want to know, "Was this website a good investment?"

    You are motivated, so you do all of the right things. You install google analytics, set up inquiry forms, use a call tracking service to identify web leads, and the data starts rolling in. For a few months it is fun to scan through the charts and graphs and spreadsheets, but all of the data doesn't seem to lead to any particular conclusion. Conversion rates, click-thru rates, call-thru rates, visits, visitors, cost-per-action, peak times, etc.... All of that information, and you still don't know if it was a good investment.

    As it turns out, you may be asking the wrong question. You should ask, "How can I use my great website to make more sales?" Frankly, whether or not the site was a good investment is irrelevant now. The price has been paid, now do something with it. Some of the data you are collecting is highly valuable for making good use of your new sales tool -- a lot of it isn't. In this case, you've got to cut the fat. What matters is, discovering which activities (or combinations of activities) lead to more/better inquiries.

    This sounds theoretical, but it isn't. The metrics that are worth my attention should focus around desirable outcomes per input -- quality leads per newspaper ad, for example. If you have some historical data you might convert it to ROI, I spent $500 on AdWords, 5 people called me, I typically close 1 out of 5 deals, and the lifetime value of a customer is $3,500. Do a little math and the money was well spent. It's not rocket science, it's not precise, but it gets you looking at the right things quickly. AND when you measure something you are more likely to manage it

  • My father-in-law owns a specialty construction company. He hounds the company controller to keep him posted on "where we are at." Controllers are numbers geeks, like web analytics consultants. The controller immediately began providing income statements and balance sheets, and then set out on the quest to give complicated, perfect status reports. The data was overwhelming, and it took hours of preparation every month to put it all together. The data entry process was altered; they even altered how the employees kept track of their time (rounding isn't accurate, ya know?) in order to get more detail.

    In the end, my father-in-law was still frustrated and had no clear idea of how things were really going. The controller was over-worked (and under-appreciated, of course); he was providing timely, accurate, and valuable data. But the value of the metrics were less than the return to the business.

    After some time my father-in-law determined there were only a few things he really needed to know. What the project "went for" and how many "mandays" did it take to complete it -- the golden metric was revenue/manday. Because he was on the production side of the business he thought in terms of production. What did we accomplish? And, how long did it take?

    Besides telling him what he wants to know this metric is golden because it has a much lower acquisition cost. Revenue and mandays are readily available because the controller is already tracking both metrics (invoices and payroll). A few minor tweaks, and a simple system was in place to report on "where we are at."

    This is not to say that the myriad of other metrics, like cash position, receivables, expenses, debt to equity, bid loss rates, etc. are not important. They are important, in the right hands at the right time -- where their value exceeds their cost.

Don't try to measure too much. And, keep in mind that the deeper you go the more it will cost -- make sure the accuracy/detail is worth the effort.