Archive

Posts Tagged ‘ibm’

The Evolving Perception of Big Data – My Personal Experience

March 8, 2013 1 comment

Sometimes you need to throw yourself out of your comfort zone to see what is really happening.

Over the past few months, I’ve been on the road presenting at events and meeting with clients (both current and prospective) across the globe. Now I’m no stranger to travel (having trekked similar international routes back in my data warehousing heyday) but this big data thing that is going on – is a wee bit different.

Over the past year, I’ve seen a major shift in the perception and adoption of a big data approach. Coming in to the job last year – I was constantly bombarded with complaints of ‘This is just hype…” “…big data is a horrible name…” “What is the 4th V?… it should be this, not that…” “ Hadoop is big data…  Hadoop will take over the warehouse…”  “NoSQL means NO SQL and thus NO Database…”

<sigh>

For the most part – All rubbish.

WarsawA year later (and still getting my fair share of naysayer voice) I ended my tour in Warsaw, Poland delivering a keynote at the Computerworld Big Data event. Now, any time you have to present in English in a non-native English speaking country, you have to take into consideration the speed of your speaking, the usage of slang and the overall ability of the audience to comprehend your words. The week before I was in Japan using a translator after each sentence. This being my first time to Warsaw to present to an audience of over 300 (where the majority of sessions were in Polish …mine had a little asterisk next to it ‘in English’ on the conference brochure which did not promote confidence), I was assuming that conveying the overall message would be a crapshoot. Boy was I wrong.

Step back a year ago while presenting at a TDWI (The Data Warehousing Institute) event…. I asked the audience  “How many of you have heard about ‘Hadoop’?” Like one or two hands would go up…  I asked the same question here… and two thirds of the room raised their hands. Granted, this was a ‘Big Data’ event… but so was the theme of that TDWI event… Times had changed over the past year in a big way.

Computerworld Warsaw

After my presentation I spoke with numerous conference attendees and the questions and topics were night and day different to what I was hearing a year ago. No more was it ‘What is Hadoop’…or ‘What is big data (and some crazy deep dive on the Vs and which ones were missing…)?’ It was more around…”So as you illustrated the use case for Data Warehouse Augmentation… Where did the actual loading of data occur?” or even better… “We have already starting using this Hadoop technology, yet we want to get a more real time perspective on our users… How can we do this?” Here I was in Poland at the end of winter (yup – cold and snowing), thinking that my talk might miss the mark – and I am engulfed by questions and briefings on actual usages of big data. This was awesome.

We all might not agree on what big data is, or what the big data challenge is at any one organization, but we can all agree that technology has been (and still is being) developed to answer questions and tap into data that was one deemed impossible. This is the big data challenge in my eyes.

mini_B4F0688But while specific technologies (like Hadoop) are fantastic… they are simply just one facet of a very complex answer. Business intelligence and reporting is irrelevant without trusted data underneath… A data warehouse appliance can be a foundation for new insights, but on its own – it is a just a dumb rack with disk. But together – all of these technologies combined – can offer the next frontier of insight – answers to problems that once were left unsolved.

It brings me back to that discussion in Warsaw that I had with a customer on how to augment their big data solution. They were already neck deep in using Hadoop to sort through clickstream data on their customers – yet could not leverage it fast enough to make a difference to their customer base. They were looking not just for a 360 degree view of their customer – but also for a way to reach out to their clients (with targeted offers) in real time.

This was the fantastic story that we were telling years ago when we built in continuous ingest into the data warehouse and brought InfoSphere Streams to market… yet here (last week) in Poland, a customer was coming to me with the same exact story. Awesome

Sushi Breakfast in TokyoAs I sit in my office today and finish up on my trip reports (really missing that sushi breakfast at the Tsukiji fish market in Tokyo the other week) I’m increasing excited for 2013 and the advances that we will make this year. Organizations around the globe are making data a priority and leveraging the latest in technologies to ensure that they are making better decisions. What I have seen in meetings and briefings so far has been nothing short of astonishing. Clients are coming to the table with some absolutely innovative ideas on how they want to leverage big data technologies – ideas that could (and will) make front page on Wired magazine one day as they are developed and implemented.

It’s going to be a fun year in big data.

Saint John Bound – The T4G Big Data Congress

January 22, 2013 1 comment

Headed out this evening up to Saint John, New Brunswick for the T4G Big Data Congress. I’ll be up there speaking on a panel with associates from a number of  Hadoop vendors including Cloudera and Hortonworks on our respective big data platforms.

It will be interesting to see how the audience is currently interpreting big data and the challenges that they face, and I will share my perspectives from the event in the coming days.

Screen Shot 2013-01-22 at 3.16.00 PM

I’m personally looking forward to hearing from Tom Davenport, who will be speaking with us as well.  Competing on Analytics came out as I was kicking off my 1st tour of duty for IBM in the warehousing space, productizing the old ‘balanced configuration unit (BCU)’ into our first data warehouse ‘appliance’, the Balanced Warehouse. Yeah seems like a decade ago. It will be very interesting to hear his current view on big data in his session and the associated application of analytics.

I’ve never actually been up to Saint John, but have already been warned by the local office that the temperatures will be a bit on the cold side. From 0 to -15 to be exact. Ouch. Reminds me of the MONY tower back in the day in Syracuse – Would always be interesting to make a run to Marshal Street at that negative temperature mark flashing in the sky.

Advancing big data technologies for the end users

November 19, 2012 Leave a comment

Last week we formally announced and GA’d a slew of new core big data offerings in our big data platform (There was a reason that I have been quiet/offline) and I thought it would be a great time to share them with you all. We started the discussion of our new technologies at the Information on Demand conference at the end of October – but they are now all fully baked in the marketplace.

The 3 new offerings are:

I’ll plan on digging deeper into each one of these offerings over the next few posts – but in summary, we are building out a platform portfolio that is unmatched in the world of big data. We are making it easier for organizations of all sizes to leverage and exploit ‘big data’ to make better decisions.

On the hadoop front, BigInsights not only updates and includes the latest support/versions for the Apache hadoop initiatives but also starts implementing technologies from across the big data platform. In addition to making hadoop more enterprise (rather than a standalone, open source project) BigInsights 2.0 offers a slew of advanced visualizations and tools for users across the organization.

With InfoSphere Streams 3.0, making decisions in real time has just become easier. While Streams has always incorporated a rich programming language (spl) not every user has had the time and effort to master it on the fly. With version 3.0, InfoSphere Streams now incorporates a visual GUI ‘drag-and-drop’ interface to program your own streams… and yes, that interface also generates the proper code as well so that you can alter and enhance granularly as well.

Last but not least, the Vivisimo acquisition in late Spring has already been integrated into the portfolio with the new InfoSphere Data Explorer 8.2 (formally known as Vivisimo Velocity). This offers fast ROI and minimizes risk by helping customers understand their big data assets and unlocking the value – including federated search – leaving data where it is BEFORE you determine if you are going to use it/analyze it….

Yeah – I’m a product guy, and well – new things are cool – so I get a little nerded out when we release offerings like this. In my next blog installments, I’ll drill into each one of these offerings and show you why we added the features and capabilities that we die (Yes, we did listen) – and most importantly how it helps you with your big data initiatives.

Back into the swing of things, …now with a Big Data focus

August 10, 2012 3 comments

It has been a busy summer so far, just getting back home from San Diego the other day for last week’s TDWI world conference.  As usual it was a great event, but one thing that was quite apparent at the event was the advent of ‘Big Data’. I don’t mean just the addition of Hadoop based companies to the mix (ie Hortonworks and the like) but most every vendor had some sort of Big Data story to tell. Even SAS had their booth just plastered with Big Data messages, and does not offer any specific ‘Big Data’ product.

For full disclosure, I’m partial to this growth of Big Data promotion in the marketplace, as I recently migrated my professional focus at IBM to this specific area. There is a level of excitement that surrounds Big Data that I have not seen since the early days of Linux adoption.  Folks are clamoring to get in on this technology and surrounding buzz. From developers through consultants, many of my warehouse discussions had some sort of ‘Big Data’ piece to it.

So what is ‘Big Data’? That is a question that I hear left and right – not just at the conference, but in general day to day business. In my limited layman type approach to the definition, I refer to ‘Big Data’ as the challenge organizations and professionals have to start using ALL of the data available to them to make better decisions.  Are you using all of it – even the stuff you normally throw away.

Does Hadoop help this? – sure – it is a key enabler in the movement.

Does streaming technology ?– absolutely – decisions in motion – awesome.

Data Warehousing? uh of course – what do you do with all of this Hadoop based data once you sort through it and deem it ‘relevant’? 

Look the list goes on and on here, but the fact is that in my opinion ‘Big Data’ is a part of a larger information ecosystem. It is the challenge to leverage all of your data available to you – not just the items that are placed in front of you, but also the ‘digital/data exhaust’ (great term that was created that I openly support) that to this point in time you have not had time to analyze.

I would suggest that part of the issue of understanding is that every company that has a dog in this fight has crafted the message to suit their own needs. Many vendors only have a subset of the products that make up an exhaustive Big Data platform and skew the definition to support this.

At the end of the day – regardless of definition, the question that you have to ask yourself is ‘Would you be better off leveraging ALL of the information that is available to you, to make better decisions?’ If you desire to be an analytical competitor, your answer is definitely yes.  This is why Big Data is big …and why the hype is warranted.

Video: IBM Smart Analytics System 5710 Installed

About a month or so ago, we installed one of our Smart Analytics System 5710s in our executive briefing center here in Raleigh, North Carolina. Having obtained my video camera the day before for my current trip to Nanning, China I decided to see if I could give it a test run with the installation.

It was remarkable that the installation took under 30 minutes. We crafted a 2 minute (or so) video that walks through highlights of the installation and some of the actual screenshots of the dashboards and reports that you get right out of the box.

Take a look and let me know what you think: