Why Big Data Can Fail You and How to Use It to Your Benefit
The following excerpt is from Benjamin Gilad and Mark Chussil’s book The New Employee Manual: A No-Holds-Barred Look at Corporate Life. Buy it now from Amazon | Barnes & Noble | Apple Books | IndieBound
Google the verb “Big Data,” and you get 3.68 billion results (as of June 8, 2018). Look at the top entries (first 10 plus paid ads), and see who’s on the first page: Oracle, IBM, SAS (a partner of IBM), McKinsey, Informatica. Big Data is Big Business. Is that expense justified?
Definitely, and not at all, depending on what you expect of it.
Definitely because of common sense. Big Data, along with the analytics to process it, means better, faster, more-accurate forecasts and, therefore better, faster, more-accurate decisions. The business press (and Oracle, IBM, et al.) can provide Big Stories, Big Anecdotes, and Big Case Studies to show how Big Data allowed companies to more precisely target advertising and promotion, segment the customer base, plan production and inventory, and a horde of other applications across all internal operations from HR to marketing to R&D. Just think about it: Which management will be more effective at competing, one using guesses and intuition or one using millions of numbers to back its decisions?
The answer should be simple, but it’s not. Ironically, there’s little data on the value of Big Data. We don’t know whether the emperor is wearing new clothes or sharing a little too much data in the open office. In a 2012 Harvard Business Review article by Andrew McAfee of MIT, titled “Big Data: The Management Revolution,” the author admits the scarcity of good measurements about the revolution of better management. He reports the results of a survey done by his team and McKinsey that shows those companies who reported using Big Data performed better. Yet he also revealed that too many executives are “pretending to be more data-driven than they actually are.” This supports Phil Rosenzweig’s conclusion in his book, The Halo Effect, about similar studies suffering from the effect of the dependent variable (performance) on the independent variable (use of Big Data), making the results quite useless. As Rosenzweig demonstrates, several very well-known studies of corporate “success formulas” resulted in spurious conclusions based on this bias.
So even though there’s no data on the value of Big Data, we can declare that using more data and better statistical techniques can only improve management. And that is actually the problem as much as the solution. The use of Big Data and analytics has been almost completely restricted to internal data or social media’s tactical information. Both belong to what Michael Porter called “operational effectiveness,” but they have little or no effect on competing.
Big data doesn't help strategy
Managing better should be, and is, on every executive’s mind. But this is a race with no winners and no sustainable competitive advantage. As one Fortune 500 company uses Big Data and analytics, another will rush to do the same. At that point, other companies will fear falling behind (consultants will spread the rumor that everyone else is already using Big Data), and these platform providers/consultants will perpetuate the expenditure until the next productivity improvement comes along. The race will continue as every one of the Big Data vendors becomes very rich, just as happened with investments in information technology. It’s exactly how capitalism is supposed to work, but it does nothing for strategic thinking... or competing.
Related: How to Identify Corporate OOPs
The difference between using Big Data and analytics for improving the current execution of existing strategy vs. improving the strategy itself (i.e., competing better) can be made crystal-clear using an example given (ironically) by McAfee to demonstrate the value of Big Data. The article tells about how a large company improved its marketing promotion for its various brands by using Big Data. The speed of the promotion cycle went down from eight weeks to one! The company? Sears.
Need we say more?
In fairness, the author at McAfee acknowledges that Big Data is not a substitute for the right vision and strategy (i.e., competing), but his caveats are naturally kind of an afterthought. They shouldn’t be. Replacing good tactical decisions with excellent tactical decisions doesn’t change the core of companies’ problems one iota. Speeding up the promotion cycle is nice, but it won’t stop the ship from sinking. Identifying strategic risks and strategic opportunities early, and acting on them before performance suffers, is what matters. That’s what it means to practice the skill of competing.
To address this core problem, Big Data (and the analytics to convert it to intelligence) need to migrate to the competitive environment with its multitude of high-impact players, thus helping the leader skilled in competing to outsmart other players. Razor-sharp superhuman segmenting of one’s existing hapless customer base to micro-target ads won’t help one bit in understanding the effect of third parties on your performance and future prospects.
Case in point: Procter & Gamble. A 2016 article in Harvard Business Review breathlessly praised the P&G innovation machine. A report by McKinsey hailed its digital revolution. Yet P&G, the King of Consumer Understanding, has become a shadow of its former self. There is more to competing than throwing millions at Big Data. Competing means taking in the big picture.
The issue of applying Big Data to competition and others in the market devolves into two simple questions:
- What are the important questions that affect how you compete well in your market?
- Do the exabytes, zettabytes, or haveanotherabytes help you answer those questions?
Numbers do not speak for themselves. Data is not wisdom; it is not even analysis. It is data. You, the maverick, the independent, critical thinker, must find the appropriate numbers, apply the appropriate analysis, and reach the appropriate conclusions.