Home > Big Data, business intelligence, data warehousing, Hadoop > Sorting Through the Clutter

Sorting Through the Clutter

Entering into this world of Big Data headfirst, I am overwhelmed with the amount of buzz and hype surrounding the topic. The other day I read the article ‘How Big Data Became so Big’ by Steve Lohr via the NY Times website and it really set the stage for the world’s challenge of Big Data. You know something has hit the big time when Dilbert references it in passing.

Per my previous post, I do not view Big Data as a product (or as a group of products), but instead as a challenge that organizations face in their journey to analyze ALL of the data made available to them to make better decisions. Hadoop is one tool to get there – yet not the only one. Over the years we have gone from machine readable punch cards to petabytes of data stored on an array of different disk types – commodity through high performance solid state.

Great – lots of storage for data – more clutter – just like my email account. Could end up being an episode of hoarders for techos.

It’s not just the analysis of the data that is important (think a superfast data warehouse appliance cranking through queries – ala Netezza) but also the determination if the data is actually worth being stored. It is like one big garage sale. There is so much to dig through, so many items old and new – You sure as heck are not going to take it all home with you – as most of the items are garbage and not needed – they would just sit around in your house  (warehouse that is) and waste premium storage – and perhaps trip you up on the way to the car (or to your analytical appliance) that you have revved and raring to go

This is where hadoop comes in handy. Hadoop sorts through your ‘digital exhaust’ (as well as any other massive load of data) and culls insight or information from it. This result can then be sent to the data warehouse for analysis – It does not have to be sent there, but in most cases I’m assuming that many folks would like to include the new insights into their analytics.

Think customer churn models, if hadoop was able to determine 1 or 2 other hidden or unknown traits of a customer segment from lets say, web click through routines  (the exhaust) – The analysis would be much more accurate and theoretically save (or make) the organization money.

There are many ways that hadoop technologies can be a part of your enterprise data warehouse or big data platform – this was just one simple example that I like to use to get my head around the technology.

At the end of the day, hadoop enables analysis of Big Data problems – It might not answer them all on its own – but it is a key player (if not ‘the’ key player) in Big Data Analytics.

  1. June 25, 2013 at 6:35 am

    I read this piece of writing fully about the resemblance of hottest and previous technologies,
    it’s remarkable article.

  2. March 7, 2014 at 12:19 pm

    I’ve been exploring for a little for any high-quality articles
    or weblog posts on this sort of space . Exploring in Yahoo
    I eventually stumbled upon this website. Reading this info So i’m satisfied to exhibit that I have an
    incredibly good uncanny feeling I discovered exactly what I needed.
    I most unquestionably will make certain to don?t forget this site and give
    it a look regularly.

  3. May 9, 2014 at 8:30 pm

    excellent publish, very informative. I wonder why
    the other specialists of this sector do not realize this.
    You must proceed your writing. I’m sure, you’ve a huge readers’ base

  4. May 10, 2014 at 4:42 pm

    Fine way of describing, and fastidious article to
    obtain facts about my presentation subject, which i am going to
    convey in institution of higher education.

  1. May 20, 2014 at 10:35 am
  2. July 6, 2014 at 3:31 pm
  3. July 23, 2014 at 4:51 pm
  4. July 23, 2014 at 11:04 pm
  5. July 24, 2014 at 11:10 pm
  6. July 25, 2014 at 12:11 pm
  7. July 25, 2014 at 9:13 pm
  8. August 4, 2014 at 11:56 am
  9. August 5, 2014 at 12:35 am
  10. August 5, 2014 at 7:00 am
  11. August 8, 2014 at 4:25 am
  12. August 18, 2014 at 1:41 pm
  13. August 18, 2014 at 3:45 pm
  14. August 19, 2014 at 2:08 am
  15. August 19, 2014 at 2:20 pm
  16. August 19, 2014 at 11:31 pm
  17. August 22, 2014 at 7:37 pm
    financial planning analysis
  18. August 22, 2014 at 8:28 pm
  19. August 25, 2014 at 8:28 pm
  20. September 3, 2014 at 4:08 am
  21. September 4, 2014 at 11:36 pm
  22. September 5, 2014 at 12:26 am
  23. September 19, 2014 at 12:26 pm
  24. September 19, 2014 at 8:08 pm
  25. September 19, 2014 at 8:36 pm
  26. September 20, 2014 at 5:29 pm
  27. September 22, 2014 at 8:42 pm
  28. September 23, 2014 at 1:21 pm
  29. September 23, 2014 at 3:13 pm
  30. September 24, 2014 at 6:00 pm
  31. September 26, 2014 at 1:16 am
  32. September 29, 2014 at 2:54 pm
  33. October 1, 2014 at 12:24 am

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: