Big Data Weekly: Hadoop, Cloudera, Leverage Big Data

This weekly roundup includes Hadoop, Intel’s Cloudera, the Leverage Big Data summit, and more.

Big Data Summit

Scheduled from May 20 to May 22 at the Park Hyatt Aviara Resort in San Diego for the Leverage Big Data (LBD) convention. Although an entry to this summit is restricted to invitation only, if you happen to be one of the fortunate invitees, you will witness presentations on hardware and software infrastructure challenges facing big data system architects and organizational teams.

This summit will feature speakers from GE Research, the National Cancer Institute, and other leading companies and research institutes—debating the business issues surrounding IT transformation.  The range of featured presentation topics will include Database Systems & Management, Big Data Infrastructure, Hadoop, Analytics, Data modeling, Storage, Cloud Analytics, & Case Studies.

Hadoop or Data Warehouse?

Following the news of Cloudera announcing Hadoop as the preferred platform for data storage and analytics, Scott Gnau, president of Teradata Labs, seemed quite exasperated by all the attention given to big data at the marketplace.

Cloudera has recently managed to raise a huge sum of $900 million from Intel and some other venture capital firms to promote their technological platform. In an interview preceding a Teradata conference in Prague, Mr. Gnau objected about the circulating prevailing misinformation about big data taking over Teradata’s data warehousing technology.

It’s not one size fits all, he said. 

Hadoop’s emergence in the IT market may not wipe out data warehousing leaders like Teradata, but their businesses can certainly face price wars technology challenges from Hadoop— as more and more, both customers and IT talent embrace big data!

Gradually, as the software applications designed for Hadoop storm the market with their faster, more reliable, and more data-handling capabilities, will Teradata, IBM, or Oracle—meet with similar fortune? Mr. Richard Winter, an independent technology consultant said in an interview,

The new technology is a real challenge. Hadoop isn’t threatening to replace something like Teradata’s data warehouse software, but it is part of the competition for new data workloads…Hadoop will be this huge thing that grows alongside. But it’s a very different kind of model and capability than that of a data warehouse.

According to Mr. Winter, Teradata is the market leader for high-end data warehousing.

Data warehousing will always retain its position because the most valuable business data like customer records, operations data—are generally preserved in data warehouses as they have to be carefully handled and maintained over a long period of time. For such long time usage, data warehousing is the best practice.

Hadoop, on the other hand, is most suited for rapid and inexpensively storing of vast amounts of data of widely disparate types. Hadoop may be used for smart analytics on semi- or un- structured data for gaining valuable business insights. For day-to-day operations, Hadoop will be the star player.

Agricultural products manufacturer utilizes big data analytics

You will find a hardware setup with analytics capabilities at tractor company John Deere’s headquarters, which is packed with computer wizardry that harvests huge volumes of valuable data as it gathers crops.

For a commodities trader or an investment bank, this type of information could prove to be invaluable. The modern-day farmers use data machines to increase operational efficiency and productivity on their farms. Many large agricultural companies have invested in data storage and analytics tools for extracting micro detail on crops and weather conditions to increase their yield.

Right now, the American Farm Bureau Federation (AFBF) is trying to set guidelines for data use with Deere and other industry heavyweights like Monsanto and DuPont Pioneer who control three-quarters of the U.S. corn seed market.

In February this year, farmer Brian Marshall of the AFBF told the U.S. House Committee on Small Business that

Virtually every company says it will never share, sell or use the data in a market-distorting way but we would rather verify than trust,

For trading purposes, the farm data could easily be misused although there are no documented instances on data being misused to date. Privacy and security concerns surrounding data gathering are not confined to agriculture. While only around 14 percent of farmers use this kind of precision agriculture technology at the moment, its popularity is expected to soar over the coming years.

Now is the time to step in and make sure that some of the concerns we have get answered,

said Mary Kay Thatcher, a senior director at the AFBF.

Intel’s high-profile investment in Cloudera 

Intel’s $740 million investment in big data software company Cloudera at a $4.1 billion valuation is claimed to be one of the largest single venture capital investment of all times. Intel bought 18% stake in Cloudera, which required some other venture capital investors to sell back their shares to accommodate Intel.

 Intel has a huge ecosystem [of hardware and software companies] and Cloudera is a big partner around big data applications,

said Ping Li,General Partner of Accel Partners—another Cloudera investor.

IDG Conducts 2014 enterprise big data research

IDG conducted its enterprise-wide, big data research project for the year 2014—primarily through online interface with the audience of its internal sub-brands, namely Computerworld, ITworld, InfoWorld CIO, CSO, and Network World. The 751 odd research participants submitted their responses through forums, pop-up windows, and emails.

The key findings of this research include:

1. 49% (almost half) of the respondents reported that big data strategy implementation has already happened in their organizations.
2. 50% of respondents reported there was an absence of leadership in their big data initiative.
3. CEOs have made big data a part of their value chain by partnering with IT executives to implement big data strategies.
4. The amount of data that organizations handle will increase by 76% in the next year to year and a half.
5. Organizations are intensifying their infrastructural investments in preparation for big data initiatives.
6. In the next year to year and a half, organizations will step up their hiring process for the data scientist community.

1. On an average, organizations are facing challenges and skilled manpower shortage in their big data initiatives.

If you require more information on this research report, then please contact Sue Yanovitch.

Avendus and Zodius to invest in SMAC

Avendus and Zodius form a partnership to build a multi-stage technology fund, which is worth around $500 million.  The primary goal of this partnership is to invest approximately $400- $500 million in emerging businesses in either Digital or SMAC- (Social, Mobile, Analytics and Cloud) based space. Their target market is expected to include online and mobile markets. The funds may be used to restructure and re-channelize companies towards high-growth markets.

Both the partners have impeccable track records—with the appropriate business network to develop companies and achieve a planned exit either through an IPO or a strategic sale. Together, they are well positioned to exploit the growing Indian SMAC market.

Mr. Ranu Vohra, Co-founder, MD & CEO of Avendus Capital added,

We have joined hands at a time when digital and SMAC businesses across the world are eliciting exceptional and rightly deserved investment interest. The IPO surge for digital India centric businesses is just beginning and we expect it to create a very high growth environment in this sector.