A version of this article appeared on TechTarget SearchSOA as EMC builds for cloud and next-generation applications
EMC World is the company’s traditional venue for major announcements, but this year the company’s premier event for customers, partners, analysts and journalists included more than just a laundry list of product updates. Sure, there was plenty of new hardware, like a major upgrade to the XtremIO all flash array, a new VMAX3 tiered storage system and updates to the Data Domain data protection software, but in retrospect EMC World 2015 will be seen as a milestone in the firm’s transformation from a purveyor of bulletproof, big iron storage systems to an integrated supplier of complete software and infrastructure stacks for building next-generation enterprise applications. Of course, EMC has long had its hands in many facets of IT infrastructure via its Federation partners VMware, Pivotal and now VCE (after buying out Cisco’s share of the joint venture), but there seemed more emphasis this year on how all the parts fit together to build clouds and next-generation applications.
EMC’s transformation mirrors that occurring within enterprise software writ large, as each migrates from the world of what EMC calls Platform 2, client-server applications and infrastructure designed for thousands of PC users to Platform 3 systems built with virtual servers, cloud services, big data and social features designed for millions of primarily mobile clients. Indeed, the event’s theme, Redefine Next, captured its essence: EMC redefining itself and product portfolio to better serve changed enterprise IT strategies and priorities, what VMware’s chief strategist calls the New IT Agenda.
EMC Embraces Openness
The first shocker came not from what EMC is selling, but what it’s giving away. The company announced that the ViPR software defined storage controller will be going into the wild aas an open source project. Sticking with the snake metaphor, Project CoprHD is a Mozilla Public Licensed version of ViPR, including all existing storage automation and control features with code available on GitHub in early June. The extent of corporate and developer adoption for ViPR won’t be clear for months, but the project has already been endorsed by executives at Intel and Canonical. This is the first step towards creating an open ecosystem around EMC’s SDS technology and is likely a precursor to ViPR’s eventual incorporation into OpenStack. When asked about the prospect, EMC executives were predictably cautious, stating they didn’t want to get ahead of themselves and were more focused on nurturing the open source community, but didn’t rule out the possibility. Although EMC will continue to sell a supported version of ViPR, it will be based on the public code base and all new feature development will occur within the open source project.
EMC’s next concession to openness comes via ScaleIO, although in this case the company isn’t giving away the store, just free samples. ScaleIO, software that turns direct attached storage across multiple servers into shared block devices on a virtual SAN, is now free to download for test and development: users must still license the product for production use. Although freemium distribution is nothing new, EMC’s move is more significant than it may appear since ScaleIO has been almost impossible for developers to get their hands short of paying for a full license, a problem EMC execs admit was greatly limiting adoption. Acknowledging new norms among developers weaned in the era of open source and GitHub, Jeremy Burton, EMC’s President of Products and Marketing said, “if developers want to steal software, we want them stealing our software.” He believes that getting ScaleIO in the hands of developers will serve to illustrate the product’s performance advantages versus public cloud services (e.g. AWS EBS, Google Persistent Disks) and open source alternatives like Ceph.
EMC’s OpenStack Erector Set
Outside the realm of storage systems and services, EMC has been busy building software and services to support next-generation Platform 3 applications. The company’s Pivotal division, a well-known leader in PaaS with a commercial Cloud Foundry release, announced several updates to its big data suite of Hadoop, Greenplum, Spring and other software for data aggregation and analysis.
The bigger news came out of EMC’s Federation Solutions group, which serves as a systems integrator for products spanning the company’s major product divisions (EMC II, VMware, VCE, Pivotal and RSA). The company unveiled and demoed Project Caspian, an enterprise OpenStack-in-a-box hardware, software bundle. Chad Sakac, EMC’s President of Global Systems Engineering, describes Caspian as an “industrialized software stack” plus converged infrastructure, designed for cloud native, Platform 3 applications using open source components. The core of Caspian is a customized OpenStack using technology from EMC’s CloudScaling acquisition, however Sakac stresses its goal is to “stay as close to the “vanilla” OpenStack core as possible.” Next, Caspian adds a persistent data layer of object storage (like Swift), HDFS (Hadoop) and block, using ScaleIO. Finally, Caspian includes what EMC calls a Cluster Manager, an orchestration layer like Kubernetes and Mesos, to automate deployments and monitor workload and system condition.
Caspian runs on VCE’s newly announced VXRACK hyper-converged systems (think rack-scale EVO RAIL), although Caspian’s primary hardware dependence appears to be in the orchestration layer meaning customers prepared for some DIY reconfiguration could likely repurpose other converged hardware into a Caspian cloud. Indeed, Sakac’s blog mentions a hardware abstraction layer, meaning there’s some attempt to insulate the software control plane from underlying hardware details. Caspian’s persistence layer currently uses ScaleIO on DAS, but as an impressive demonstration of the new DSSD rack-scale all-flash array made clear, future versions will include an all flash tier.
Hardening the Cloud
In total, EMC World demonstrates a traditional IT vendor working to make the cloud easy, reliable and safe enough for even the most conservative enterprise. A final example to illustrate the point is Cloudlink, a small Canadian company EMC acquired earlier this year. EMC talked more about Cloudlink during a media event, but as Sakac puts it, “Cloudlink does something simple – and does it well. They deliver a product called SecureVM that encrypts (and attests to, and otherwise controls and manages) compute instances in clouds, whether the VM images be Windows or Linux or reside on AWS, Azure, Google Cloud or vCloud Air). Cloudlink integrates on-site key management with in-cloud key repositories allowing companies to maintain full control over encryption credentials and policies while providing end-to-end VM, boot volume and application encryption.
EMC World didn’t feature a blockbuster announcement, but had many important surprises that were significant to both EMC and its customers. Overall, the event shows a company modernizing, expanding and normalizing its products into a full-stack enterprise cloud suite that can be deployed on premise or consumed as shared services. In retrospect, EMC World 2015 was ‘Pivotal’ to the company’s future.