The Rise of Big Data
By John Patrick Pullen
A massive abstraction with a cute little name, Big Data is the vast (and ever-growing) stockpile of information too large for companies to store in-house--let alone analyze. All that info offers potential to cutting-edge entrepreneurs willing to step up and do the crunching.
According to IBM, 2.5 quintillion bytes of data are born every day (enough to fill more than 531 million DVDs), and 90 percent of the world's digital information was produced over the last two years. "For so long, we've focused on human-powered businesses, and now we're transforming into data-driven organizations that are bringing a level of customer centricity that we've never seen before," says Graeme Noseworthy, marketing director for IBM's Big Data Solutions Software Group.
And this is just the Cro-Magnon stage of the info boom. By 2020, the annual data-generation rate will balloon 4,300 percent to 35 zettabytes of intelligence, according to Falls Church, Va.-based Computer Sciences Corporation. (That's 7.35 trillion DVDs, in case you're keeping track).
From yesterday's Dow Jones industrial average to a GPS tag on today's Instagram image to tomorrow's pollen count, every bit of information will be collected, cataloged, distributed, stored and analyzed by a rapidly evolving segment of companies. According to Reuters, venture capital firms invested $2.47 billion in fields around big data in 2011, nearly 10 percent of all money distributed. That's up from $1.53 billion in 2010 and $1.1 billion in 2009.
"Big Data represents a transformation of the entire IT industry and a $300 billion to $500 billion wealth-creation opportunity for entrepreneurs," says Matt Ocko, co-managing partner at San Francisco-based investment fund Data Collective. "It's as sound a bet for us today as investing in PC-related technologies in 1981 or in internet-enabling technologies in 1994."
Entire funds, like Data Collective, have popped up to support the category, using technology to inform investments in startups like Palo Alto, Calif.-based Continuuity, which helps developers make data-based cloud applications, and Portland, Ore.-based Cloudability, a cloud-computing dashboard that enables companies to monitor their online computing expenditures. And these investments appear to be solid bets. According to a report by the McKinsey Global Institute, the potential value of big data, if used creatively and effectively by the U.S. healthcare industry, would be worth more than $300 billion in that sector every year. Currently healthcare providers throw 90 percent of that information away.
And that's just one of many market segments Big Data will revolutionize. In June Ford Motor Company opened a Silicon Valley lab to process data from more than 4 million cars currently spitting out information. Twitter recently hooked 12 partners up to its massive fire hose of 340 million tweets per day, allowing fledgling firms like New York-based Dataminr to sell global event predictions and Boulder, Colo.-based Gnip to parse and resell post-related data.
No matter how you look at it--as a consumer, marketer, investor, administrator or inventor--the numbers don't lie. Big Data isn't just changing business; it's changing everything.