Welcome!

Containers Expo Blog Authors: Elizabeth White, Liz McMillan, Scott Sobhani, Pat Romanski, John Esposito

News Feed Item

EMC Introduces World's Most Powerful Hadoop Distribution: Pivotal HD

Unlocks Hadoop as Key to Big Data's Transformational Potential for Data-Driven Enterprises; Delivers Over 100X Performance Improvement with HAWQ

SAN FRANCISCO, Feb. 25, 2013 /PRNewswire/ --

News Summary:

  • EMC's new Pivotal HD™ is an Apache Hadoop distribution that natively integrates the industry-leading EMC Greenplum massively parallel processing (MPP) database technology with the Apache Hadoop framework.
  • The engineering innovation and technology called HAWQ™, brings Greenplum's decade of research and development on the industry's best MPP analytical database platform to Apache Hadoop, the open-source Big Data platform.
  • New HAWQ technology brings more than 100X performance improvement (in some cases up to 600X) to a wide range of query types and workloads, making Pivotal HD the single most powerful Hadoop distribution in the industry.
  • Pivotal HD with HAWQ represents the world's first true SQL processing for Hadoop, expanding the platform's reach to SQL programmers and developers, as well as enabling compatibility with traditional BI tools.

Full Story:

EMC® Corporation (NYSE: EMC) today announced a new distribution of Apache Hadoop: Pivotal HD. Pivotal HD features native integration of EMC's industry leading Greenplum® massively parallel processing (MPP) database with Apache Hadoop—the most cost-effective and flexible open source Big Data platform ever developed.  The new EMC Greenplum-developed HAWQ™ technology brings 10 years of large scale data management research and development to Hadoop and delivers more than 100X performance improvements when compared to existing SQL-like services on top of Hadoop, making Pivotal HD the single most powerful Hadoop distribution in the industry.

Hadoop has rapidly emerged as the preferred solution for Big Data analytics applications that grapple with vast repositories of unstructured data. It is flexible, scalable, inexpensive, fault-tolerant, and enjoys rapid adoption rates and a rich ecosystem surrounded by massive investment. However, customers face high hurdles to broadly adopting Hadoop as their singular data repository due to a lack of useful interfaces and high-level tooling for Business Intelligence and datamining—components that are critical to data analytics and building a data-driven enterprise. As the world's first true SQL processing for Hadoop, Pivotal HD addresses these challenges.

By offering the full spectrum of the SQL interface, and by extension, the entire ecosystem of products that support SQL, customers no longer need an army of developers to build a dashboard or run a report. Unlike competitive Hadoop distributions, Pivotal HD does this without moving data between systems or using connectors that require users to store the data twice. Pivotal HD cuts out the complexity of using Hadoop, thus expanding the platform's potential and productivity, and allowing customers to enjoy the benefits of the most cost-effective and flexible data processing platform ever developed.

About HAWQ

HAWQ (pronounced hawk) represents the EMC Greenplum engineering effort that brings 10 years of large-scale data management research and development to the Apache Hadoop framework. Leveraging the feature richness and maturity of the industry leading Greenplum MPP analytical database, this innovation has resulted in the world's first true SQL parallel database on top of the Hadoop Distributed File System (HDFS). HAWQ is the key differentiating technology in making Pivotal HD the world's most powerful Hadoop distribution. Capabilities of note include Dynamic Pipelining, a world-class query optimizer, horizontal scaling, SQL compliant, interactive query, deep analytics, and support for common Hadoop formats.

Pivotal HD and HAWQ Deliver:

  • True SQL Query Capabilities – With Pivotal HD's advanced database services (HAWQ) enterprises can now unlock the potential of Hadoop's scalable, fault-tolerant storage capabilities by bringing to bear the vast pool of "data worker" tools and languages into the Hadoop ecosystems.  With Pivotal HD's support for true, SQL-standards compliant query interfaces data mining tools, SQL-trained data analysts, and standard BI tools can now easily connect to, query, and analyze data sets stored in the Hadoop file system (HDFS).
  • Unprecedented Query Performance – Bringing over 10 years of parallel database processing technology to Hadoop, Pivotal HD delivers query response time improvements that are up to 600x faster than current SQL-like interfaces for Hadoop.
  • Robust Operational Support – Command Center enables administrators and developers to easily install and manage large clusters from interactive web user interfaces.  Command Center also exposes Command Line Interface for scripting and programmer friendly web services API for complex automation tasks. Using Command Center administrators can deploy large cluster, configure services/roles, manage services and monitor HDFS jobs and tasks.

Hadoop -- The Foundation for Change

EMC believes that Hadoop has the potential to reach beyond Big Data to catalyze new levels of business productivity and transformation. As the foundation for change in business, Hadoop represents an unprecedented opportunity to improve how organizations can get the most value from large amounts of data. Businesses that rely on Hadoop as the core of their infrastructure can not only do analytics on top of vast amounts of data, but can also go beyond analytics and the foundation for that data layer to build applications that are meaningful, and that have a very tightly coupled relationship with the data. Consumer Internet companies have reaped the benefits of this approach, and EMC believes more traditional enterprises will adopt the same model as they evolve and transform their businesses.

Pivotal HD is expected to be available at the end of the first quarter of this year as a software-only or appliance-based solution, backed by EMC's global 24x7 support infrastructure.

Executive Quotes:

Scott Yara, senior vice president of products, Greenplum, a division of EMC

"Hadoop is a big deal and poised, for many enterprises, as the key to unlock Big Data's transformational potential. Marrying the extraordinary capabilities of the Greenplum technology—essentially the Greenplum crown jewels—with this amazing open-source phenomenon has been no small feat. But we're 'all in' — investing in a manner that no other company is — to help catapult Hadoop into wide-scale adoption."

Steven Hirsch, chief data officer, SVP Global Data Services, NYSE Euronext

"With Pivotal HD, we can check off many of the items on our Hadoop wish-list—things like plug-in support for the ecosystem of tools, improved data management and greater elasticity in terms of the storage and compute layer. But above all, it provides true SQL query interfaces for data workers and tools—not a superficial implementation of the kind that's so common today, but a native implementation that delivers the capability of real and true SQL processing and optimization. Having a single Hadoop infrastructure for Big Data investigation and analysis changes everything. Now add to all of this functionality the fact that the SQL performance is up to 100x faster than other offerings and you have an environment that we at NYSE Euronext are extremely excited about."

Jeff Kelly, Big Data analyst, The Wikibon Project

"By integrating the Greenplum database natively within Hadoop, Pivotal HD opens up Big Data analytics to a new class of practitioner – the savvy business user –providing both real-time query responsiveness via a familiar SQL interface and eliminating the need to move data back and forth between Hadoop and separate analytic databases. Most importantly, by providing both real-time and batch analytics capabilities in a single environment, Pivotal HD moves Hadoop several steps closer to realizing its full potential as the de facto, comprehensive Big Data platform of the 21st century."

John Myers, senior analyst, Enterprise Management Associates 

"The Pivotal HD release reflects the results of recent Enterprise Management Associates Big Data Research and our vision for the Hybrid Data Ecosystem.  With the sharing of a common data store via HDFS and processing information based on business requirements via either a SQL or NoSQL access, Pivotal recognizes that different analytical challenges require different processing/access layers."  

Sam Grocott, vice president of marketing and product management, EMC Isilon

"The introduction of Pivotal HD, combined with EMC Isilon's native integration of the Hadoop Distributed File System (HDFS) protocol, continues the evolution of the industry's first and only enterprise-proven Hadoop solution on a scale-out NAS architecture. This powerful combination succeeds in reducing the complexities traditionally associated with Hadoop deployments and allows enterprises to easily extract business value from unstructured data."

Partners Supporting the Pivotal HD announcement:

Find out what our partners are saying about Pivotal HD by clicking here.

Additional Resources

About Greenplum, a division of EMC

Greenplum, a division of EMC, is driving the future of Big Data analytics with breakthrough products that harness the skills of data science teams to help global organizations realize the full promise of business agility and become data-driven, predictive enterprises. The division's products include Greenplum® Unified Analytics Platform, Greenplum® Data Computing Appliance, Greenplum® Database, Greenplum® Analytics Lab, Greenplum® HD and Greenplum® Chorus™. They embody the power of open systems, cloud computing, virtualization and social collaboration, enabling global organizations to gain greater insight and value from their data than ever before possible. Learn more at www.greenplum.com

About EMC

EMC Corporation is a global leader in enabling businesses and service providers to transform their operations and deliver IT as a service. Fundamental to this transformation is cloud computing. Through innovative products and services, EMC accelerates the journey to cloud computing, helping IT departments to store, manage, protect and analyze their most valuable asset — information — in a more agile, trusted and cost-efficient way. Additional information about EMC can be found at www.EMC.com.

EMC, Greenplum, HAWQ, Pivotal HD, and Chorus are trademarks or registered trademarks of EMC Corporation in the United States and/or other countries. All other trademarks used are the property of their respective owners.

This release contains "forward-looking statements" as defined under the Federal Securities Laws.  Actual results could differ materially from those projected in the forward-looking statements as a result of certain risk factors, including but not limited to: (i) adverse changes in general economic or market conditions; (ii) delays or reductions in information technology spending; (iii) the relative and varying rates of product price and component cost declines and the volume and mixture of product and services revenues; (iv) competitive factors, including but not limited to pricing pressures and new product introductions; (v) component and product quality and availability; (vi) fluctuations in VMware, Inc.'s operating results and risks associated with trading of VMware stock; (vii) the transition to new products, the uncertainty of customer acceptance of new product offerings and rapid technological and market change; (viii) risks associated with managing the growth of our business, including risks associated with acquisitions and investments and the challenges and costs of integration, restructuring and achieving anticipated synergies; (ix) the ability to attract and retain highly qualified employees; (x) insufficient, excess or obsolete inventory; (xi) fluctuating currency exchange rates; (xii) threats and other disruptions to our secure data centers or networks; (xiii) our ability to protect our proprietary technology; (xiv) war or acts of terrorism; and (xv) other one-time events and other important factors disclosed previously and from time to time in EMC's filings with the U.S. Securities and Exchange Commission.  EMC disclaims any obligation to update any such forward-looking statements after the date of this release.

SOURCE EMC Corporation

More Stories By PR Newswire

Copyright © 2007 PR Newswire. All rights reserved. Republication or redistribution of PRNewswire content is expressly prohibited without the prior written consent of PRNewswire. PRNewswire shall not be liable for any errors or delays in the content, or for any actions taken in reliance thereon.

@ThingsExpo Stories
"delaPlex is a software development company. We do team-based outsourcing development," explained Mark Rivers, COO and Co-founder of delaPlex Software, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, wh...
The IoT is changing the way enterprises conduct business. In his session at @ThingsExpo, Eric Hoffman, Vice President at EastBanc Technologies, discussed how businesses can gain an edge over competitors by empowering consumers to take control through IoT. He cited examples such as a Washington, D.C.-based sports club that leveraged IoT and the cloud to develop a comprehensive booking system. He also highlighted how IoT can revitalize and restore outdated business models, making them profitable ...
Machine Learning helps make complex systems more efficient. By applying advanced Machine Learning techniques such as Cognitive Fingerprinting, wind project operators can utilize these tools to learn from collected data, detect regular patterns, and optimize their own operations. In his session at 18th Cloud Expo, Stuart Gillen, Director of Business Development at SparkCognition, discussed how research has demonstrated the value of Machine Learning in delivering next generation analytics to imp...
The cloud market growth today is largely in public clouds. While there is a lot of spend in IT departments in virtualization, these aren’t yet translating into a true “cloud” experience within the enterprise. What is stopping the growth of the “private cloud” market? In his general session at 18th Cloud Expo, Nara Rajagopalan, CEO of Accelerite, explored the challenges in deploying, managing, and getting adoption for a private cloud within an enterprise. What are the key differences between wh...
Connected devices and the industrial internet are growing exponentially every year with Cisco expecting 50 billion devices to be in operation by 2020. In this period of growth, location-based insights are becoming invaluable to many businesses as they adopt new connected technologies. Knowing when and where these devices connect from is critical for a number of scenarios in supply chain management, disaster management, emergency response, M2M, location marketing and more. In his session at @Th...
Basho Technologies has announced the latest release of Basho Riak TS, version 1.3. Riak TS is an enterprise-grade NoSQL database optimized for Internet of Things (IoT). The open source version enables developers to download the software for free and use it in production as well as make contributions to the code and develop applications around Riak TS. Enhancements to Riak TS make it quick, easy and cost-effective to spin up an instance to test new ideas and build IoT applications. In addition to...
When people aren’t talking about VMs and containers, they’re talking about serverless architecture. Serverless is about no maintenance. It means you are not worried about low-level infrastructural and operational details. An event-driven serverless platform is a great use case for IoT. In his session at @ThingsExpo, Animesh Singh, an STSM and Lead for IBM Cloud Platform and Infrastructure, will detail how to build a distributed serverless, polyglot, microservices framework using open source tec...
Apixio Inc. has raised $19.3 million in Series D venture capital funding led by SSM Partners with participation from First Analysis, Bain Capital Ventures and Apixio’s largest angel investor. Apixio will dedicate the proceeds toward advancing and scaling products powered by its cognitive computing platform, further enabling insights for optimal patient care. The Series D funding comes as Apixio experiences strong momentum and increasing demand for its HCC Profiler solution, which mines unstruc...
IoT offers a value of almost $4 trillion to the manufacturing industry through platforms that can improve margins, optimize operations & drive high performance work teams. By using IoT technologies as a foundation, manufacturing customers are integrating worker safety with manufacturing systems, driving deep collaboration and utilizing analytics to exponentially increased per-unit margins. However, as Benoit Lheureux, the VP for Research at Gartner points out, “IoT project implementers often ...
It is one thing to build single industrial IoT applications, but what will it take to build the Smart Cities and truly society changing applications of the future? The technology won’t be the problem, it will be the number of parties that need to work together and be aligned in their motivation to succeed. In his Day 2 Keynote at @ThingsExpo, Henrik Kenani Dahlgren, Portfolio Marketing Manager at Ericsson, discussed how to plan to cooperate, partner, and form lasting all-star teams to change t...
In his general session at 18th Cloud Expo, Lee Atchison, Principal Cloud Architect and Advocate at New Relic, discussed cloud as a ‘better data center’ and how it adds new capacity (faster) and improves application availability (redundancy). The cloud is a ‘Dynamic Tool for Dynamic Apps’ and resource allocation is an integral part of your application architecture, so use only the resources you need and allocate /de-allocate resources on the fly.
In his keynote at 18th Cloud Expo, Andrew Keys, Co-Founder of ConsenSys Enterprise, provided an overview of the evolution of the Internet and the Database and the future of their combination – the Blockchain. Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life sett...
Presidio has received the 2015 EMC Partner Services Quality Award from EMC Corporation for achieving outstanding service excellence and customer satisfaction as measured by the EMC Partner Services Quality (PSQ) program. Presidio was also honored as the 2015 EMC Americas Marketing Excellence Partner of the Year and 2015 Mid-Market East Partner of the Year. The EMC PSQ program is a project-specific survey program designed for partners with Service Partner designations to solicit customer feedbac...
There are several IoTs: the Industrial Internet, Consumer Wearables, Wearables and Healthcare, Supply Chains, and the movement toward Smart Grids, Cities, Regions, and Nations. There are competing communications standards every step of the way, a bewildering array of sensors and devices, and an entire world of competing data analytics platforms. To some this appears to be chaos. In this power panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, Bradley Holt, Developer Advocate a...
Cloud computing is being adopted in one form or another by 94% of enterprises today. Tens of billions of new devices are being connected to The Internet of Things. And Big Data is driving this bus. An exponential increase is expected in the amount of information being processed, managed, analyzed, and acted upon by enterprise IT. This amazing is not part of some distant future - it is happening today. One report shows a 650% increase in enterprise data by 2020. Other estimates are even higher....
SYS-CON Events announced today that Bsquare has been named “Silver Sponsor” of SYS-CON's @ThingsExpo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. For more than two decades, Bsquare has helped its customers extract business value from a broad array of physical assets by making them intelligent, connecting them, and using the data they generate to optimize business processes.
19th Cloud Expo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterpri...
Internet of @ThingsExpo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 19th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devices - comp...
The 19th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Digital Transformation, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportuni...