Project Management and Big Data – as a project

As part of this month’s Ideaca blogging network challenge, we were tasked with discussing our thoughts on Big Data.

This is going to be a 2 part post:

  • The first part will cover how you, as a project manager, should approach a project that carries the mantle of “Big Data.”
  • The second part will cover how you, as someone in a Project/Program Management Office, can use Big Data without getting snookered by the hype.

Part 1 – So you’ve been asked to “implement Big Data”… what now?

Defining Your Terms

I am going to assume that you – like me – tend to be baffled by the marketing speak until you can speak with someone intelligently about a topic.  In the case of Big Data, I have heard a few definitions.  The one that seems to stick the most for me is the one from Wikipedia:

  • Data sets that are too big for traditional database management systems to handle
  • Data sets that comprise information from multiple sources to try to infer correlation

Sounds easy enough, right?

Where it starts to get complicated (thanks Wade!) is when you try to integrate “unstructured and semi-structured data with our “traditional” structured data”.

You will never “implement Big Data”

When it comes to Big Data, you do not implement it.  You may be implementing a technology to support the analysis, but you will never actually implement this “thing”.  A project of this sort relies on understanding the user requirements, selecting the right technology, and taking an exploratory approach when developing reporting capabilities.

Understanding the User Requirements

In the case of a new process and technology, such as this, your user requirements may be fairly light.  “We want to correlate information from disparate sources to identify predictive trends” or “I don’t know – but I really want some cool looking reports” may be common lines that you hear.

Like all projects, the user requirements are your definition of success.  Because “Big Data” is still a technology in the exploratory stage, though, expecting detailed requirements may be the wrong sorts of requirements.  The ones that you should be really focused on are the data sources and ensuring that the information being presented is right.

To wit, if I were to ask you to present the information on the average CEO compensation for the top 50 companies in North America, how would you start?  How would you define the Top 50?  By Market Capitalization?  By Environmental Performance?  By Stock Price?  By Revenue?  What about getting access to private company information?  All of the sudden, a fairly simple question about the average CEO compensation gets a little more complex.

The same will be true of your Big Data project.  Start by understanding that to present the information your users want, you will either have to ask a whole lot of detailed questions, or provide a platform to enable them to answer their own questions.

Understanding the available technology

As Project Managers, we know that when we are asked to Implement something, it’s never that simple.  Understanding what the technology can and cannot do is critical to ensuring that your project can meet the user’s definition of success.

One might want to satisfy the guiding principles of a company’s Enterprise Architecture.  A quick scan of the landscape will reveal that tools like SAP HANA, Oracle’s Exadata, and Amazon’s AWS can all fulfill the technology requirements quite nicely and potentially support a company’s Enterprise Architecture.  However, as this is a new application of technology, fulfillment of requirements needs to trump Enterprise Architecture.

Take an Exploratory and Iterative Approach to reporting

Some organizations will judge success of your project by its ability to deliver a load of reports.  If this sounds like your organization, be realistic as to what can be delivered.  Deliver a robust and reliable dataset, some transactional reports, and one report that really helps demonstrate the art of the possible.

Smarter organization will judge success of your project by its ability to deliver analytic capabilities to the user base.  The robust and reliable dataset is still mandatory, but the ability for users to generate their own reports will satisfy all of the “what about …?” requirements that would blow your project budget and schedule out of the water.

In the end… it’s the people that matter…

If we believe all of the marketing hype, Big Data will help us explore all the myriad of ways our world is constructed.  But from the perspective of a Big Data as a project, an empowered user base will produce much more value than some canned reports.

Have you been asked to “implement big data”?

If so, what did your project look like?  Let me know in the comments down below.  Stay tuned for a month-end post on making the most of Big Data in a PMO.

Special thanks to Wade Walker and Chris Sorensen for keeping me honest with this post.


About Jason H Zalmanowitz
I am a nerdy Management Consultant / Project Manager with a MBA, have spent the majority of my career working for consulting firms in Calgary, and I race in triathlons because of (and thanks to) my wife. As a Project Manager, I have managed field implementations, strategy development projects, software development projects, and hardware implementation projects. As a consultant, I have helped companies articulate how and why they are going to implement and interact with sustaining technology to support their business.

One Response to Project Management and Big Data – as a project

  1. Pingback: The Practical Application of Big Data for the PMO | Unnatural Leadership

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: