IBM Launches HPC Cloud Offering

Born out of an initiative designed to bring efficiencies to the development of its own POWER7 processor family, IBM is commercializing a High Performance Computing (HPC) cloud offering tuned specifically to meet the needs of automotive and aerospace manufacturers as well as high-tech electronics companies.

The IBM Engineering Solutions for Cloud is designed to bring cloud computing resources to engineering organizations, particularly in these high-growth, high-complexity sectors, enabling them to scale HPC resources to accommodate expanding technical workloads in the areas of analytics and high-fidelity simulation. This instance of the cloud is different from IBM's SmartCloud enterprise-class cloud platform aimed at business in that it is optimized for handling the large-scale compute demands of highly advanced simulation software and the management and administration of HPC cloud platforms leveraging thousands, if not tens of thousands, of processor cores.

With digital prototyping and advanced simulation becoming more integral parts of the design process, IT costs for engineering organizations are skyrocketing. Global electronics companies and automotive OEMs typically have datacenters scattered around the globe to accommodate an extended ecosystem of design partners. Because the computing demands for these design and simulation applications are so complex, they end up with multiple datacenters stocked with expensive and often underutilized equipment so they can accommodate a "follow the sun" development strategy.

Mergers and acquisitions have also added to the datacenter and server sprawl. It's not uncommon for an electronics maker in the $1 billion to $20 billion range to have 50 datacenters around the world with up to 20,000 cores of high-performance compute power -- an expense that is increasingly difficult to justify and manage in today's challenging economic climate.

"All of the collaboration and complexities are pushing the limits of the engineering process," says Bruce Anderson, general manager, electronics industry, for IBM, in an interview. "You throw in all the software engineering that's going on and you end up with the worst of all worlds. As IT costs for engineering keep escalating, it's impacting electronics makers' ability to make a profit."

Given its experience creating a private HPC cloud for the POWER7 development effort, IBM is convinced other engineering organizations can benefit from the same approach. In the second quarter of 2009, 3,000 globally distributed engineers working on the POWER7 processor family moved off of local workstations and server farms and began accessing an IBM internally hosted private HPC cloud to do their advanced simulation and development work.

"All the computational power used to create POWER7 was in Building 45 in IBM's Austin, Texas, datacenter," says Carl Anderson, IBM fellow, in an interview. "An engineer could be working on a laptop in Bangalore, but all the compute and storage was in the cloud out of Austin."

The IBM engineering team saw two major benefits from the cloud-based HPC approach: First, it cut developer costs in half and reduced the design cycle for the POWER7 by up to six months. The other benefit, which was not anticipated, Carl Anderson says, was that the efficiencies of the designers and engineers went up. Instead of being chained to their workstations at the desk waiting for the results of simulations, which could take hours, engineers could run a simulation and view the results at home, at night, on the convenience of their laptops.

Having a cloud infrastructure specifically optimized for HPC tasks is critical to achieving any of the promised efficiencies, according to a paper written by a team of researchers at Lawrence Berkeley Lab as part of its Magellan Project. After running a series of benchmarks designed to represent a typical midrange scientific workload (and using fewer than 1,000 cores) on the Amazon EC2 cloud platform, the team found the interconnect capabilities of the offering limited the performance and caused significant variability.

Of course, the cloud raises all kinds of security issues for organizations, especially when you're talking about critical intellectual property. But making the HPC service initially available as a private cloud hosted by IBM should alleviate many of those concerns. Bruce Anderson says the offering will likely evolve into a public cloud service when there's an ecosystem to support it.

To accommodate the specific demands of engineering groups, the IBM Engineering Solutions for the Cloud will include IBM Rational Software and Systems Engineering Suite, the IBM Collaboration Hub, and 2D/3D accelerators. ISV partners like ANSYS, Cadence, EXA, and Magma are also lending their support to the HPC cloud environment.