Big Data – Who will find the patterns of value?

May 8, 2012

Big Data has great promise – no doubt that we are on the brink of discovering innovative ways to make a lot of money due to the ability to consume and analyze a greater volume, velocity, and variety of data than ever before.

The question, however, is who will make these innovative discoveries?

Can the innovation be made by a kid in a garage, a college kid in his dorm room, or a talented IT professional dreaming of a start-up in his free time?

On the one hand, the answer is NO – the college kid in the dorm will NOT be changing the world via big data the way Mark Zuckerberg did with social networking. Why? The reason quite simply is because “Big Data” requires DATA and a lot of it and in potentially many different forms.

Data is not free

In fact, there are billion dollar businesses that are essentially data companies – they sell raw data (and analytics on it). I’ve even worked at one in my time. OK – Some of that raw data is free but much of the data is not. The sheer volume of data is prohibitively expensive to acquire for a college kid to start to experiment with. Additionally, some big data use cases revolve around social data that sites like Facebook can use to make money (data is why Facebook has such a high valuation). Did you think Facebook gives away their data for free? If the college kid can’t even get his hands on the data then the game is over before it even starts.

On the bright side, however, cloud computing infrastructure as a service and pay-as-you-go pricing provides affordable infrastructure for the college kid. Powerful infrastructure is also a pre-requisite for Big Data – but it’s not a barrier for the college kid.

It goes back – again – to access to data.

The majority of the big data discoveries will come from big established companies with the resources to acquire it.

Agree or disagree? Actually, proving me wrong would make me happy.