Cars all have the same basic components that function in similar ways, but you wouldn't test-drive a sedan to make a buying decision on an SUV.

The same rationale should apply to CAD workstation performance. Benchmarks that use generic CAD models and datasets to characterize workstation performance will never give you a true picture of your real-world experience. That vague picture might be fine if your work isn't critical to the operation of your company, or if productivity and efficiency don't matter. But who has that luxury?

Faster response time = greater productivity

"When a computer and its users interact at a pace that ensures that neither has to wait on the other, productivity soars, the cost of the work done on the computer tumbles, employees get more satisfaction from their work, and its quality tends to improve. Few online computer systems are this well balanced; few executives are aware that such a balance is economically and technically feasible." 

The quote above is from a paper titled “The Economic Value of Rapid Response Time,” and it's just as true today as it was when it was published in 1982. It seems basic, but it is worth reiterating: The more computing speed you can pack into an engineer's day, the more benefits it reaps, including:

Beyond economic issues, there is the mission-critical nature of CAD/CAM work. When events such as an airplane crash or automotive recall happen, CAD applications are placed in environments that demand the best in terms of people, process, and products. There is no room for poor or mediocre workstation performance.

Differences that matter

CAD workstations are all doing the same basic things, but it is the way they do those things – based on the packages they run, the optimizations they provide, the specific models they are building, and the way they render those models – that adds up to performance differences.

Aside from the obvious differences in the objects being designed and engineered, there are many other differentiating factors affecting CAD performance, including:

Unique industry requirements – Product design is far from one approach fits all. Architecture, engineering, and construction (AEC) have requirements such as building information modeling (BIM) processes and construction documentation that are unique to the industry. Aerospace and medical device products need to account for certification requirements. Different industries require different levels of detail, accuracy, and technical specifications.

Different approaches to engaging the CPU and GPU – One of the fastest-moving areas of CAD innovation is how different packages handle rendering in order to deliver a finished photorealistic model in the least time possible. There is a delicate balancing act in managing the work performed by the CPU with that of the GPU, enabling their respective architectures to complement one another through careful partitioning of the different methods and stages of rendering. The approaches to rendering differ not just among CAD packages, but when newer versions of the same package are introduced.

Certification and optimizations – Graphics card vendors and workstation OEMs probably work closer with CAD software vendors than with any other type of ISVs. There are typically more certification requirements than with other applications, along with intense competition to make CAD packages run faster and more intuitively. Change is a constant.

Implementation of external referencing – Many CAD packages rely on externally referenced geometry to fully populate a model and streamline the modeling process. If your model has 100 sockets that are identical, why model or duplicate each individually? Instead, a part is saved to a file outside of the working assembly and a placeholder is imported into the main workspace that references that file. How a CAD package implements this functionality can make a difference in performance.

Support for different types of modeling – CAD packages typically go down different routes to the same destination in the type of modeling they support. Parametric modeling – a step-by-step process where an entire history of the model is recorded and can be adjusted at any time in the product development process – is implemented differently in Catia, Solidworks, NX, and other CAD packages. Direct modeling – which forgoes the object's history and allows the user to adjust faces, vertices, or sections of the model directly – also works differently depending on the package. Then there's the hybrid approach, using a combination of parametric and direct modeling. Same basic functionality; different implementations.

What happens after design is done – A lot of the major performance differences among CAD packages comes after the modeling is done. This includes functionality such as PLM/PDM integration, photorealistic rendering, management of different design iterations, and storage and retrieval of models.

The perpetual need to do more – In CAD/CAM/CAE, the prevailing trend is always to do more with larger and larger models. Benchmarks that measure basic functionality on small models are out of step with the innovation driving the industry toward integrating mechanical, software, electrical, and other elements in the same huge model. This leads to larger datasets that place pressure on the critical need for increased productivity and reliability.

Fortunately, there are choices

The complexity of measuring workstation performance based on professional CAD applications might be enough to have users throwing their hands in the air in despair. Fortunately, there are good options besides the generic benchmarks that are the equivalent of a plain brown wrapper.

The Standard Performance Evaluation Corporation's Graphics and Workstation Performance Group (SPEC/GWPG), a non-profit organization that's been around for more than 30 years, develops graphics and workstation benchmarks based on actual applications and real-world workloads, including ones for CAD/CAM. The really good news is that these benchmarks are free as long as your company is not a vendor of computer products or services.

Covering all the bases

If you want the best representation of total performance for a specific application, go for SPEC application performance characterization (SPECapc) benchmarks. These benchmarks require installing a specific version of the application you want to benchmark. They use models and workloads that represent those used by CAD/CAM professionals and provide a comprehensive picture of CPU, GPU and I/O performance.

Current SPECapc benchmarks for CAD/CAM cover Creo 3.0, Solidworks 2019, and NX 9.0/10.0. There are also SPECapc benchmarks for Autodesk Maya and 3ds Max.

Measuring graphics performance

If you don't have easy access to the application you wish to benchmark and you're interested primarily in graphics performance, you can run SPECviewperf. SPECviewperf measures the 3D graphics performance of systems running under the OpenGL and Direct X application programming interfaces. There is also a version for Linux.

SPECviewperf workloads, called viewsets, represent graphics content and behavior extracted from professional applications, without the need to install the applications themselves. SPECviewperf 13, the current version, includes viewsets for Catia, Creo, NX, Solidworks, and Autodesk Showcase.

Total workstation performance

If you are interested in total workstation system performance – CPU, GPU, I/O, and memory bandwidth – you can download SPECworkstation 3, which includes all of the CAD/CAM viewsets listed above, plus additional product development, media and entertainment, financial services, energy, life sciences, and general operations workloads.

Like SPECviewperf, SPECworkstation is a self-contained software package that doesn't require you to install or run the applications themselves.

Speed saves when applied correctly

There's a saying that speed kills, but it can also save – time and money. But in order for speed to deliver on its promise it must be applied in the proper way.

ISVs, workstation OEMs, graphics card manufacturers, and other component makers are in a perpetual race to improve the productivity of CAD engineers. Good CAD workstation benchmarking is a way to show how vendor innovations translate into real-world performance for very specific products and functionality, so you can make improvements in the areas that reap the greatest ROI.

RELATED ARTICLES:

Trey Morton is chair of the SPECapc subcommittee and a performance engineer in the Dell Client Solutions Group.