How to Really Eat an Elephant

Craig Jordan Craig Jordan
February 15, 2019 Big Data, Cloud & DevOps

experfy-blogNeed training for Hadoop? Browse courses developed by industry thought leaders and Experfy in Harvard Innovation Lab.

Many Hadoop installations have been focused on individual teams and their particular data analysis projects. But that’s changing as Scott Carey points out in “11 Hadoop case studies in the enterprise,” business from big banks to airlines and retailers are deploying Hadoop at the enterprise scale. Further he asserts, that “Forrester now says enterprise adoption of Hadoop is ‘mandatory,’ so any business that wants to derive value from its data should, at the very least, be looking at the technology.”   Installing Hadoop and enabling a single team to use it has become a simple process, particularly if a cloud-based offering of the platform is acceptable. Whether on Azure, AWS, the Google Cloud Platform or another cloud provider’s infrastructure, provisioning a multi-node Hadoop instance can be done quickly…point and click.

Does that mean, however, that the newly-minted Hadoop cluster is ready for the enterprise?  Is it ready to be used concurrently by teams of data scientists and analysts, and applications across your company? Is your data lake ready to be stocked with information of every shape, size, and specie? Probably not. Being ready for that takes more than having processes running on servers ready to respond to REST service calls.  To enable your enterprise to share a common Hadoop cluster and the information it contains really means “eating the elephant.”

A quick Google search will give you plenty of hits related to breaking dauntingly-large tasks down to size in order to “eat the elephant” of an audacious goal. Mike Martel, however, takes a different tack — he suggests “You hack it up and have a party.” While still being focused on achieving an overwhelming goal or project, his emphasis is on collaboration and parallel activity.

So what is the path for taking Hadoop from the confines of being a pet technology of a data science team here or there to it being a platform on which business strategy with data can be built?  It takes collaboration and parallel activity.

And that takes investment.  And, like any significant investment, it deserves a vision to give it a purpose and a roadmap to give it direction.  Here are three key benefits using a strategic roadmap for adopting Hadoop for your enterprise can offer.

“If you don’t know where you’re going,
any road will take you there” — Lewis Carroll

First, a roadmap identifies a destination and the direction to take to get there.  Unlike a traditional map that illustrates many locations along with the highways, neighborhood streets, dirt paths or hiking trails that connect them without strongly preferring any particular destination, a strategic roadmap for adopting Hadoop for your enterprise lays out the particular destination:  A reusable, well-organized data lake on the Hadoop platform with myriad data subjects, types and formats useful to a varied audience of scientists, analysts, applications and others.  By clarifying the destination, the roadmap also clarifies the paths that can be followed, and how far each goes toward helping you reaching your overall destination.  Preventing you from getting stuck along the way, a roadmap like this helps you maintain momentum on the journey.

“All roads lead to Rome” — proverb

Second, a roadmap can illustrate how there are numerous paths that can be taken to arrive at a destination. If Hadoop is to be useful for a variety of teams and different kinds of data analysis, prediction and reporting capabilities, it is unlikely that all the teams that contribute to an enterprise grade Hadoop environment will follow the same path.  Give them all the same roadmap, however, and you can help them to arrive at the same destination from different directions.

“Are we there yet?” — your kids
“Turn around when possible.” — your GPS

Finally, a roadmap points out milestones by which you measure progress and provides warning signs that you may be off course.  Your kids like to know “how much further?”  Your spouse likes to know “are you sure this is the right way?”  Similarly, there will be people in your company who wonder the same things during your Hadoop journey.  “It’s installed!  Are we done?”  “There’s lots of data in it, how much more work do we have to do?”  “Are you sure this is the right way to use the capability of the platform we are building?”  A strategic roadmap gives guidance regarding how to answer these questions.

Building this kind of roadmap takes time, but it’s worth the effort.  Give some thought to the destination your company is seeking to achieve through the capacity and power of the Hadoop platform.  If you want help in putting the pieces of a strategic roadmap together to get you there, check out “Adopting Hadoop for the Enterprise“.

  • Experfy Insights

    Top articles, research, podcasts, webinars and more delivered to you monthly.

  • Craig Jordan

    Tags
    Big Data & Technology
    Leave a Comment
    Next Post
    Ripple: Why the Anti-Bitcoin is Loved by Banks and Hated by The Internet

    Ripple: Why the Anti-Bitcoin is Loved by Banks and Hated by The Internet

    Leave a Reply Cancel reply

    Your email address will not be published. Required fields are marked *

    More in Big Data, Cloud & DevOps
    Big Data, Cloud & DevOps
    Cognitive Load Of Being On Call: 6 Tips To Address It

    If you’ve ever been on call, you’ve probably experienced the pain of being woken up at 4 a.m., unactionable alerts, alerts going to the wrong team, and other unfortunate events. But, there’s an aspect of being on call that is less talked about, but even more ubiquitous – the cognitive load. “Cognitive load” has perhaps

    5 MINUTES READ Continue Reading »
    Big Data, Cloud & DevOps
    How To Refine 360 Customer View With Next Generation Data Matching

    Knowing your customer in the digital age Want to know more about your customers? About their demographics, personal choices, and preferable buying journey? Who do you think is the best source for such insights? You’re right. The customer. But, in a fast-paced world, it is almost impossible to extract all relevant information about a customer

    4 MINUTES READ Continue Reading »
    Big Data, Cloud & DevOps
    3 Ways Businesses Can Use Cloud Computing To The Fullest

    Cloud computing is the anytime, anywhere delivery of IT services like compute, storage, networking, and application software over the internet to end-users. The underlying physical resources, as well as processes, are masked to the end-user, who accesses only the files and apps they want. Companies (usually) pay for only the cloud computing services they use,

    7 MINUTES READ Continue Reading »

    About Us

    Incubated in Harvard Innovation Lab, Experfy specializes in pipelining and deploying the world's best AI and engineering talent at breakneck speed, with exceptional focus on quality and compliance. Enterprises and governments also leverage our award-winning SaaS platform to build their own customized future of work solutions such as talent clouds.

    Join Us At

    Contact Us

    1700 West Park Drive, Suite 190
    Westborough, MA 01581

    Email: support@experfy.com

    Toll Free: (844) EXPERFY or
    (844) 397-3739

    © 2023, Experfy Inc. All rights reserved.