Home > Big Data > Back into the swing of things, …now with a Big Data focus

Back into the swing of things, …now with a Big Data focus

It has been a busy summer so far, just getting back home from San Diego the other day for last week’s TDWI world conference.  As usual it was a great event, but one thing that was quite apparent at the event was the advent of ‘Big Data’. I don’t mean just the addition of Hadoop based companies to the mix (ie Hortonworks and the like) but most every vendor had some sort of Big Data story to tell. Even SAS had their booth just plastered with Big Data messages, and does not offer any specific ‘Big Data’ product.

For full disclosure, I’m partial to this growth of Big Data promotion in the marketplace, as I recently migrated my professional focus at IBM to this specific area. There is a level of excitement that surrounds Big Data that I have not seen since the early days of Linux adoption.  Folks are clamoring to get in on this technology and surrounding buzz. From developers through consultants, many of my warehouse discussions had some sort of ‘Big Data’ piece to it.

So what is ‘Big Data’? That is a question that I hear left and right – not just at the conference, but in general day to day business. In my limited layman type approach to the definition, I refer to ‘Big Data’ as the challenge organizations and professionals have to start using ALL of the data available to them to make better decisions.  Are you using all of it – even the stuff you normally throw away.

Does Hadoop help this? – sure – it is a key enabler in the movement.

Does streaming technology ?– absolutely – decisions in motion – awesome.

Data Warehousing? uh of course – what do you do with all of this Hadoop based data once you sort through it and deem it ‘relevant’? 

Look the list goes on and on here, but the fact is that in my opinion ‘Big Data’ is a part of a larger information ecosystem. It is the challenge to leverage all of your data available to you – not just the items that are placed in front of you, but also the ‘digital/data exhaust’ (great term that was created that I openly support) that to this point in time you have not had time to analyze.

I would suggest that part of the issue of understanding is that every company that has a dog in this fight has crafted the message to suit their own needs. Many vendors only have a subset of the products that make up an exhaustive Big Data platform and skew the definition to support this.

At the end of the day – regardless of definition, the question that you have to ask yourself is ‘Would you be better off leveraging ALL of the information that is available to you, to make better decisions?’ If you desire to be an analytical competitor, your answer is definitely yes.  This is why Big Data is big …and why the hype is warranted.

  1. August 10, 2012 at 8:43 pm

    I am counting on you Larry to bring some maturity to this discussion. There has been far too much hype by everyone over the past year+. All seasoned Information Management Professionals understand the value proposition of “leveraging all of your data sources” (especially unstructured). Until just recently however, we have been limited in large part by technology and complexity issues. Both of these have been solved to a great extent by Hadoop/Map Reduce, etc., but this is more of an incremental outcome rather than a disruptive one. While I appreciate all of the attention and focus that the “Big Data Craze” has brought to Information Management and Analytics, I would not want to world to believe that somehow we have arrived at the end game. We have a long ways to go (most of it on the cultural adoption front). Best of luck in your new assignment. Hope to see you at IOD this year.

    • August 10, 2012 at 8:58 pm

      🙂 Yes, it is a journey as well – Part of the goal is to start mapping defined examples, use cases for that matter with tangible goals (and results). Yet I appreciate the point of ‘too much’ hype in the past. For example, hadoop is not the endgame for most, but instead an awesome approach to mining digital exhaust …we still need the infrastructure, the warehouse, etc. we are dependent on today.

      We should grab some time to chat – …and yes, will see you at IOD.

  2. August 14, 2012 at 6:27 pm

    Sounds good. I am onsite with clients T-Th and in office M & Fr. if we don’t catch up before IOD we can set up something just beforehand. I look forward to your news series of discussion

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: