Monthly Archives: December 2014

Open Sesame: IBM Hoping OpenPOWER and Members Google, Rackspace Can Foster Intel Alternatives

By | December 22, 2014

Intel remains the brains behind most cloud and enterprise servers and having vanquished AMD is without materially significant challengers. That doesn’t mean Chipzilla can be complacent, since new competitors are trying to use the disruptive transition to hyper scale clouds to take a slice of its lucrative data center business. As I recently detailed, Intel’s performance and market lead, while real, can’t be taken for granted. ARM and its partners finally have a 64-bit, VM-ready architecture that promises to compete on scale out, power-sensitive workloads. While early products like the Applied Micro X-Gene system I mentioned are still early in their lifecycles and far from optimized, performance will only get much better. Indeed as Applied Micro’s VP of engineering Gaurav Singh subsequently pointed out to me, initial systems are designed for software porting not performance benchmarking. But ARM isn’t the only contender for cloud server sockets and it’s easy to forget that two other server platforms hope to infiltrate the next generation of hyper scale cloud data centers and that one, IBM POWER platform, is already widely used for mission critical enterprise applications. As I detail in this column, MIPS, the other anti-Intel is still mostly consigned to low-end embedded applications, however IBM is hoping to reenergize the POWER platform with help from others. If you can’t beat them, go open source and that’s what Big Blue has done via OpenPOWER.

The OpenPOWER consortium has been quietly toiling out of public scrutiny since its founding announcement last winter, which given the tech industry’s collective ADD isn’t a good thing. In hopes of getting back on IT’s radar and preempting its creeping anonymity, the foundation recently made some noise by announcing new members and 2015 plans. No, OpenPOWER isn’t just a debating society and has ambitious goals for the coming year, with 6 work groups developing technical specifications across the technology stack from chip design to cloud software and “dozens of products introduced and under development” by member organizations. Yet the big news isn’t what the group plans to do, but who is now part of the team: former (and hopefully future) cloud heavyweight Rackspace. Rackspace is an important addition to a group that largely consists of component suppliers and ODMs since it’s both an important cloud service provider and potential bridge to two other significant cloud technology working groups: the Open Compute Project and OpenStack Foundation.

Outwardly Rackspace seems to be hedging its bets between competing open infrastructure projects, however as the column points out, the key Rackspace liaison to both OpenPOWER and Open Compute, Aaron Sullivan, sees them as complementary. Unlike Compute Compute, which focuses more on system packaging (rack and chassis design, electrical distribution, system module interconnection), Sullivan says OpenPOWER allows developers to design and modify system boards, firmware, even new processors. He adds that the raw technology IBM has opened up is significant, “IBM has contributed an enormous amount of IP in the chip and firmware space.”

OpenPOWER Members Suggest Tantalizing Possibilities

Aside from Big Blue, OpenPOWER has attracted some important technology companies as top, Platinum-level sponsors, but none more significant than Google. Indeed, a Googler chairs the board. Unfortunately, but not surprisingly, Google’s Senior Director of hardware platforms, Gordon MacKean wasn’t willing to discuss the firm’s use of or plans for POWER platforms, although he posted about its customer POWER8 motherboard on Google+. Google uses the board as a development platform to port its software to the POWER architecture and as MacKean told me, hyper scale cloud workloads and applications are incredibly diverse, implying that some are better suited to the POWER architecture than x86. “Architectural choice is good,” says MacKean adding “the Foundation creates another viable option to choose from.”


MacKean showing off Google’s custom POWER8 motherboard.

IBM has seen what happens to proprietary processor architectures developed and used by a single vendor and in OpenPOWER has taken an important, strategic step to avoid becoming the next Alpha, SPARC or Itanium. Indeed, OpenPOWER is the last, best hope for maintaining POWER processor viability and growing it into a realistic Intel alternative for cloud servers, but on that metric it’s impossible to judge success or failure at this point. Google or Rackspace using IBM-designed and build (through foundries) POWER8 chips won’t move the needle on Intel’s server market share unless it’s followed by a groundswell of support from other hyperscalers like Amazon, Facebook and Microsoft. Don’t hold your breath unless OpenPOWER leads to disruptive hardware innovation. 

Tyan OpenPOWER reference motherboard.

Tyan OpenPOWER reference motherboard.

The real potential of OpenPOWER will only be realized if and when people design custom SoCs using POWER cores and that exploit  CAPI (Coherent Accelerator Processor Interface), the architecture’s elegant support for hardware coprocessors, to offload and accelerate various functions. Although OpenPOWER now makes this possible, it’s far from an easy task and neither MacKean or Sullivan would comment on when or if we’ll see custom POWER-based silicon of the type that has made ARM the leader in mobile devices. 


IBM POWER multi-chip module used in System Z mainframes. Source: author

It’s unclear whether OpenPOWER and its growing community will jump-start the POWER architecture into a realistic Intel competitor for hyperscale cloud data centers, but by opening the kimono and revealing circuit, interface and firmware details, OpenPOWER has the opportunity to catalyze meaningful, disruptive innovation.



The SAP Money Pit: The Quintessential Business Application Is Immune to SaaSification

By | December 16, 2014

There’s a giant sucking sound emanating from most large enterprise IT departments. It’s called SAP and as I write in this column, the devilishly complex ERP product cum application platform has a life of its own that resists all attempts at efficiency, simplification and IT budget cutting. SAP is the enterprise equivalent of an out-of-control defense contract with C-level business managers channelling Congress and the Pentagon by endlessly shoveling money at overdue and over-budget projects. This harsh characterization may seem unfair to those that haven’t read the a new book by Vinnie Mirchandani. SAP Nation catalogs the real and hidden costs of using SAP through painstaking research into dozens of SAP customers and a detailed cost model exposing the many, often hidden costs of deploying and operating the platform. Mirchandani is perhaps uniquely qualified to assess the state of enterprise application software, is a long-time observer of SAP, its ecosystem and users having served as both a customer of and consultant on SAP. A former IT analyst and outsourcing executive, he now runs his own consulting practice specializing in IT deal negotiation, RFPs and due diligence.


As part of his study of SAP implementation strategies and best practices, Mirchandani developed a comprehensive financial model and cost accounting that includes purchasing, configuring, operating and supporting SAP and its users — expenses that cross company and organizational boundaries including costs for internal IT staff, outsourcing and consulting firms, the hardware used to run SAP and related applications, along with software and support licenses. Mirchandani estimates the total spent in what he calls the SAP Economy totals $1 trillion since the end of the great recession, or about $200 billion per year. It’s a big number, particularly when you consider that the aggregate profits of the S&P 500 are about $250 billion (chart 9). Granted, the sum is spread across over a quarter million companies that Mirchandani calculates use one or more SAP product, but that still comes to almost $800,000 per year, two-thirds of which is spent on labor, not software or infrastructure, for the average SAP customer.

What’s wrong with SaaS?

Almost every key business application, including that bastion of fortress Microsoft, Office, has been disrupted by the cloud with the rental economics of SaaS proving to be irresistible for many organizations. However as I point out in the column, SAP is so entrenched in critical, revenue-producing business processes and IT infrastructure that the product and ecosystem have heretofore proven immune to the radical changes roiling the rest of the software industry. Indeed, SaaS has moved far past beyond consumer file sharing and collaboration services, spawning multi-billion dollar companies like Salesforce that cater to core business processes using cloud economics to lower IT CapEx and OpEx. Even as complex systems like CRM, HR and legal document analysis move to the cloud, SAP blithely rolls on soaking up a disproportionate amount of IT’s budget on pricey outsourcing and consulting contracts and immense, gold-plated on-premise systems.



One reason there isn’t a comprehensive, all-in-one cloud-based alternative to the full SAP suite is that the concept of vertically and tightly integrated software goes against the cloud ethos of lean, focused services extensible and integrable using published APIs that users can lash together to create software tailored for their specific needs. The cloud zeitgeist is about mashups, not suites. Most cloud early adopters have embraced the use of open APIs and data connectors to lash  together multiple SaaS offerings within a comprehensive service-oriented architecture. But as Mirchandani points out, “Contrast that with the view many SAP customers have that they bought the ERP suite to get out of the integration role.”

Source: MuleSoft

Source: MuleSoft

It’s not as if SAP couldn’t do an Adobe-like pivot and field a compelling SaaS suite since it’s got plenty of cash. Writes Mirchandani,

“In 2013, SAP reported over $3 billion in R&D expenditure. That was almost four times larger than the combined R&D spend of the seven leading SaaS providers — Salesforce, Workday, NetSuite, RightNow, ServiceNow, SuccessFactors and Taleo. (RightNow and Taleo are now part of Oracle, and SuccessFactors part of SAP.)”

It’s that the company isn’t particularly interested in the model since it caters to its largest customers; big enterprises that SAP executives believe aren’t keen on the idea of moving the infrastructure and data for key business processes into the cloud. As my original column concludes, large enterprises find it dauntingly complex and risky to redesign these mission critical apps around cloud services, hence the SAP gravy train will continue until more of them decide to redesign IT systems and services from scratch.

For the rest, those thousands of smaller SAP users, doing a full cost analysis of their SAP-related spending will prove both enlightening and alarming. If Mirchandani’s model is remotely accurate, such a TCO assessment could provide enough shock value to overcome IT inertia and seriously investigate cloud-based alternatives for ERP and other key business processes. It’s time for more businesses to extend SaaS from the front to back office.

Intel Still Rules the Cloud: Haswell Puts the Squeeze on ARM’s Server Plans

By | December 9, 2014
Update: A rewritten and condensed version of this report appeared as a Forbes column, available here.

Cloud software stacks, like those used at Amazon, Google, Facebook, Microsoft and other hyper scale operations, have radically altered how system architects design and build data centers. Out are big, gold-plated integrated hardware systems like fault-tolerant x86 boxes, mainframes and enterprise storage arrays. In are vast arrays of interconnected commodity pizza box servers with local disks, the Legos for building cloud-based applications. In this environment, size, power efficiency and cost are of prime importance as sophisticated cloud management software distributes workloads across many systems, automatically moving them as necessary without disrupting running applications or their users. As the saying goes, in the cloud you treat systems like cattle, not pets. Servers are disposable.

Disposability also means economization, in cost, space and power usage, which poses a threat to high-performance Intel processor and name-brand servers. But unlike with mobile processors, where Intel completely missed the boat and is still playing catchup, the firm’s reflexes in the data center, where it’s already the dominant platform, have been much more agile, actually anticipating the needs of cloud system designers. Yet there is room for low-end, consumer-oriented processors to disrupt the market, however early indications are that next-generation ARM-based products won’t be the catalyst for change.

What’s different about the cloud?

Cloud hardware is entirely virtualized, so given the speed of today’s processors it means each system can handle perhaps dozens of workloads. There’s no need to build servers with beefy, high-performance CPUs when a bunch of slower, more efficient chips can do the same job. Thus, it’s long been thought that cloud designs would evolve to look more like a colony of ants than a herd of elephants. When applied to systems and processors, the cloud philosophy places a premium on low-power chips that can be densely packed, not the highest performance per CPU socket.


The cloud design paradigm presented a challenge for Intel which evolved its CPU architecture from the days of monolithic client-server applications where workloads were confined to a single server and at best multithreaded enough to use multiple processing cores simultaneously. Prior to the rise of cloud services, massively parallel software was the stuff of government and university research labs. Thus, Intel has long focused its engineering prowess on maximizing performance at any cost. But the cloud changed the rules for system design by making compute density paramount. The relevant system metrics are now performance per watt and performance per cubic inch, not raw horsepower. Much like RISC instruction sets exploited hardware bottlenecks in traditional CPUs with a new, streamlined architecture to become the predominant platform for Unix boxes in the 90s, hyper scale clouds seemed to leave an opening for alternative processors better aligned with the needs of virtualized workloads and tuned for density, efficiency and cost.

Since the qualities of power efficiency and cost are precisely those of smartphones vis-a-vis PCs, ARM designs, which power virtually all of today’s mobile devices, including the iPhone, iPad and Galaxy lineup, are the natural ‘anti-Intel’. Although ARM servers have been tried before, notably by Calxeda, neither the market nor technology were ready for mass adoption, as evidenced by the firm’s humbling demise a year ago. For servers, ARM’s major drawback has been an antiquated 32-bit instruction set and no hardware support for virtualization that together made ARM systems infeasible to host cloud software stacks or VMware, the virtualization software of choice for enterprise IT. ARM addressed both of these limitations in its Cortex A-57 reference design. Although Apple introduced the first 64-bit ARM chip over a year ago in its iPhone 5s, commercially available ARM-64 products are just now coming to market. The first Cortex-A57 SoC specifically designed for server and embedded applications comes from Applied Micro. With pre-production development kits made available this fall, it’s finally possible to see how well an ARM-based system performs on actual server workloads.

Applied Micro X C1

APM X-Gene Motherboard with 64-bit ARM CPU

Unfortunately for ARM and its design partners, Intel hasn’t been standing still and has a substantial lead in process technology over any of the semiconductor foundries available to manufacture ARM-based designs that translates into fundamental performance advantages derived from the pure physics of semiconductor operation. Case in point is Intel’s recently released third-generation E-series Xeon server CPUs that push an already capable product to new levels of performance, efficiency and cloud workload management. Superior overall performance of the E5-v3 is well documented, however the open question is how it would compare to next-generation ARM-based servers in dense, hyper scale cloud environments. The release of Applied Micro’s kit provided the first clues.

Leave it to the Scientists: Analyzing ARM-64 on Real World Applications

With 64-bit APM development systems now available, the first independent tests were published by physicists at CERN this fall. The team of researchers benchmarked Applied Micro’s X-Gene system, with an 8-core Cortex-A57 SoC, against two Intel-based systems: a conventional 8-core Xeon and a many-core Xeon Phi. Since the Xeon Phi is designed for highly parallelized workloads, the interesting results for data center designers is ARM’s performance against Xeon. As expected, in raw horsepower Xeon mopped the floor with the ARM box, but things are closer when measuring performance per watt. However the CERN group’s results understate Intel’s advantage since they used a now-obsolete first-generation E5 Xeon (Sandy Bridge) in their tests, not the latest Haswell E5v3. The table below presents key specifications for each of the processors:

CPU family CPU model Cores Threads/core Freq. (GHz) TDP (W) Process Geom (nm) Launch date
Xeon E5 E5-2650 8 2 2 95 32 Q3’12
Xeon E5 v2 E5-2650 v2 8 2 2.6 95 22 Q3’13
Xeon E5 v3 E5-2630v3 8 2 2.4 85 22 Q3’14
ARM64 A57 APM Helix 1 8 1 2.4 42 42 Q3’14 samples 2015 prod.
  • used in CERN tests

When comparing the basic parameters, a couple of things stand out:

  • The two-generation process technology advantage enjoyed by Intel’s latest parts over the APM device, which is built by TSMC’s foundry
  • The APM device’s remarkably high power usage compared to mobile ARM processors that typically run 2.5 (phone) to 5 (tablet) watts. Indeed, Intel now has a gen-3 E5 part that comes within 10 watts (23%) of the APM device.

A mere 2-to-1 power advantage over Intel’s latest v3 parts leaves the APM SoC severely lacking in performance efficiency as we’ll see in the following benchmarks. Of course, APM and other ARM builders could reduce power by moving to a more advanced process node, although it’s unclear whether TSMC’s 20 nm process used by Apple would work for APM’s design. For now it’s just not competitive.

Extrapolating the CERN team’s test results to derive a comparison against Intel’s current-generation part is necessarily imprecise, but one way is to look at the relative performance differences between roughly equivalent Xeon products from the three E5 generations. SPEC benchmarks are widely available for all three and for overall system performance, the SPECint Rate is the best measure. The following table shows data for each generation:

Generation CPU model SPECint2006_rate Relative Perf.
Xeon E5 E5-2650 538 1.0
Xeon E5 v2 E5-2650 v2 683 1.270
Xeon E5 v3 E5-2630 v3 686 1.275
CMS XGene abs perf

CERN CMS Benchmarks

We can apply the relative performance measure to compare the APM device to a E5 v3 Xeon. CERN’s data found the first-generation E5  has about 2.5-times the overall performance of the ARM SoC on a mix of tests (called CMS) that are representative of the workloads for particle physics data analysis. This means a current-generation Xeon of roughly the same specs as the part used in CERN’s analysis would have over 3-times the raw performance of the ARM device.

Things tighten up when looking at compute efficiency, i.e. performance per watt. Here the APM system came within 10% of the Sandy Bridge Xeon, however since the Haswell v3 is both faster and more efficient, the gap widens considerably with the ARM device delivering only 65% of the Xeon’s compute efficiency. The chart below summarizes the data.

Intel ARM Perf comp

Intel Still Rules the Cloud

The first 64-bit ARM server processors illustrate the nearly impossible task of competing with Intel in the data center, a lesson AMD learned the hard way. Its tick-tock product development strategy means the steady delivery of improved performance derived either from new, denser fabrication processes or major circuit design changes. Although one can’t judge the merits of ARM’s 64-bit platform based on a single implementation, the first results show no benefit for cloud infrastructure where the Haswell generation Xeons deliver superior absolute and energy-adjusted performance while preserving the time-tested x86 instruction set.

Cloud builders seeking maximum density should stick with hyper scale 2U, quad node x86 systems like those used for VMware’s EVO:RAIL since it’s unlikely ARM systems will match their performance, density and flexibility any time soon.



Enterprise Cloud Use to Skyrocket in 2015: Will AWS Be Your Next Data Center?

By | December 3, 2014

As we close another year full of IT changes, there are more signs that enterprise adoption of public cloud services will…wait for it…soar in 2015. As I write in this column, big enterprise IT vendors used to charging 6- and 7-figures for hardware and 20% or more of that per year for ongoing support could be in for quite a shock. The current cloud enthusiasm could just the collective industry euphoria after the annual AWS re:Invent hype machine, but I doubt it; the cloud has gone mainstream. Enterprise IT’s understanding, acceptance and trust of the cloud reached a milestone this year, making the transition from FUD fixation to steely scrutinization of ROI.


Yet, what’s good for IT may turn out disastrous for big IT; i.e. old tech in Wall Street speak. What started as an annoyance primarily of interest to developers and startups has evolved into a serious disruption to their business models that is already starting to take significant chunks out of their top and bottom lines as companies move applications and even entire data centers to the cloud en masse. But it’s about to get worse. Big, established tech vendors are likely to face growing a tsunami of financial pain as the once hardly noticeable undersea earthquake of cloud acquiescence turns into a tidal wave as customers replace hardware and data centers with cloud instances and services.

I rundown the latest cloud market projections in the column projecting double-digit increases in IT cloud spending. However even the consensus rates of 20-30% annual growth may be too low if enterprises start moving entire data centers to Amazon or Azure. Think that’s not happening? Read on. But first, more about that vendor pain.

IDC 2018 public cloud

HP’s Fourth Quarter: A Study in Cloud Disruption

Look no further than HP’s just-released Q4 earnings to see the cloud’s potential to wreak widespread damage. The company’s enterprise group, which sells servers, storage and network equipment, saw revenue decline 4% year-over-year. But HP’s enterprise software group, which has only a fledgling cloud offering and does most of its business through IT outsourcing contracts, did even worse, dropping 7% year-year. Worse yet for HP, along with fellow travelers like IBM, Oracle and Cisco, even if it manages to grow services revenue through a more robust cloud offering, the gains still won’t replace the profits from lost hardware business since cloud services operate on thin margins. HP’s enterprise systems group operates at almost 15% margins; services, just under 7%. But services are a broad category that includes high-margin activities like consulting and deployment projects and given the high CapEx of hyper scale cloud data centers, it’s possible, if not likely that IaaS margins are even lower. In other words, HP needs to replace every dollar of lost hardware revenue with over two dollars of services to generate the same profit.

HP enterprise group Q414 HP enterprise services Q414

The reason the cloud does so much damage to the HP’s of the world isn’t just that hyper scale services like AWS, Azure and Google use hardware so efficiently, it’s that they increasingly bypass the middleman for servers, switches and storage shelves, dealing directly with Asian manufacturers. Cloud services design data centers, networks and virtual infrastructure much differently than enterprise IT; for example, see this Facebook post for details on its latest network design built for massive scale, redundancy and automation and thus ideally suited for cheap, commodity switches and servers. Cloud services see little or no value added by traditional big iron vendors. Why pay a 50% (or more) tax for features, support or gold-plated reliability when you won’t use and don’t need them?

Today’s Fork-Lift Data Center Upgrade: Straight to AWS

A recent conversation with 2nd Watch, a major AWS system integrator revealed that business is booming. 2nd Watch specializes in getting enterprise applications and infrastructure onto AWS, doing things like systems integration, management and monitoring by wielding cloud expertise that most companies could only dream of. The column goes into details about the company’s growth and finances, but what I found most interesting is what Aden calls lift and shift projects, i.e. wholesale migrations of entire data centers, now make up half of the company’s business. He says 2nd Watch has already decommissioned 30 data centers that typically hold between 200 and 600 servers and based on the interest expressed and leads generated at re:Invent, he’s planning to migrate between 15-30,000 servers next year.

cloud-migration What should be particularly scary for incumbent IT vendors is that just as they have made peace with the cloud through hybridization, advocating hybrid cloud designs that link virtualized corporate data centers with public cloud services, a small but growing segment of IT organizations are bypassing the private cloud piece altogether by going straight to AWS. Indeed, as I wrote earlier this fall, NetApp made hybrid cloud the centerpiece of its annual customer event. But NetApp isn’t alone, talk to any large IT vendor — Cisco, EMC, HP, VMware, pick your favorite — and they all say hybrid is the enterprise cloud paradigm of the future. It well may be, as I believe many (most?) companies aren’t in a position, either technologically, organizationally or even emotionally, to move their digital assets lock, stock and barrel to AWS or any other public cloud service. But it’s clearly too soon to anoint hybrid cloud as the de facto standard enterprise IT architecture for the next generation.

I will be watching closely to see if 2015 is the year the floodgates open and enterprises flock to the cloud in unprecedented numbers. Aden certainly thinks so, “This is happening so rapidly, traditional vendors probably don’t understand how it will affect them in 2015.” If he’s right, 2014’s growth problems at HP and company will serve as little more than a preview of the red ink to come.