OK, IT’S 2012. SO WHAT’S THE NEXT BIG THING? BIG DATA

One of the most famous lines in movie history is when Dustin Hoffman in The Graduate is told the secret to his future success, “plastics”, which was a hot growth area in 1967. In my career I’ve been fortunate enough to witness a few ‘next big things’ and be in a place and time when I could act on them.

In 1971 I was asked to teach a course on data base management. At that time it was a very new subject with a worldwide installation base of no more than a few hundred sites, mostly IBM’s IMS. I thought it would become big and the topic of data base management systems ultimately led to the formation of Digital Consulting, Inc. and Data Base World.

When the IBM PC came out in 1981, I had a similar thought that this was going to really change the focus of computing away from the then popular mainframe and into distributed computing. At DCI we started a trade show called Downsizing Expo, which was very popular for a few years.

And in 1994, as a beta tester of Windows 95, I thought that this product would become explosive for Microsoft and lead to a kind of world domination. I wrote about this in my Client Server Today Magazine column, and here are a few excerpts from that article:

In its 1994 wrap-up, “The Year in Review,” PC Week listed Windows 95 as one of the six “biggest letdowns of the year.” I don’t agree…Once it arrives, Windows 95 will sweep the industry like a tornado, unlike anything the industry has seen before…

Win 95… is going to create the loudest sucking sound you’ve ever heard for Microsoft Office. At a yield to Microsoft of perhaps $100 per copy and installing on, say, two thirds of Win 95 sites, Office should bring another $1.5 billion per year or so into Microsoft’s coffers.

Microsoft’s stock price should appreciate from the strong cash flow… That will mean the company’s capitalization could escalate to the point where it will be in a position to… challenge IBM for…the highest market capitalization.

Anyone who purchased MSFT at the time this article appeared would have seen a 400% appreciation as the stock split four times from 1995 to 2000.

I have no doubt that we are now seeing the beginning of tremendous growth for the field known as “big data”. Big data has become a useful technology as result of progress made in many fields:

• Cloud computing with enormous drops in the cost of computing and data storage.
• Widely available real time computing with cloud farms from companies like Amazon, Microsoft, Rackspace, GoDaddy, IBM, etc.
• Enhanced capabilities in natural language processing and artificial intelligence (AI).
• The combination of DBMS and data mining capabilities with faster, cheaper processing as discussed above.
• Growth of data being created and made accessible over the Internet.

The basic idea behind  big data is to mine the world wide web for data that is occurring in real time and that, based on past experience, has been shown to correlate positively with variables that are valuable to predict.  The benefit to most users comes from the closely related field which is usually referred to as ‘analytics’. This is the science of how to interpret and use the Internet’s enormous sources of data to help one make those decisions which will prove profitable. And if you can use super cheap computing to do this, you don’t need to improve very much over informal ‘guessing’ to offer tremendous advantages to decision makers.

We are going to witness many new software companies building and marketing platforms that are to be offered to users as a low cost way to jump into the field. Examples include: WiseWindow, Topsy Labs, Relevant Data, Recorded Future, Crimson Hexagon, and SENSENews. And I have no doubt that existing software players such as SAS, SAP, Oracle, IBM, etc. will also be players.

Big Data is even to the point that it has its own conference and trade show industry with specialized events such as:

• Big Data Conference
• GigaOM’s Structure:Data
• O’Reilly Strata Conference and Hadoop World
• World Conference 2012: Big Data Tipping Point
• Big Data Innovation

Big Data and analytics. The new plastics!