The Bug Hunters

Meet the working engineers who test your CAD/CAE vendor

Laurie Toupin -- Design News, April 6, 2003

Think you know the source of that engineering software code you use? Well, not so fast!

The software itself may have come from the vendor whose name is on the package, but many of the features you use every day were probably the brainchild of an engineer just like you working somewhere in the manufacturing world. Alpha testers and beta testers—those folks whom software companies ask to test their products before commercialization—are the real pioneers in engineering software. They find the bugs before you do, hopefully, and suggest features and fixes the code writers probably never thought of. Design News herewith pays tribute to these folks whose unpaid work on your behalf helps software suppliers help you be more productive.

Michelle Pillers, for SolidWorks

When Michelle Pillers, Managing Partner of Evolution Design Engineering in Kirkland, WA, started her own mechanical design company, she needed a CAD program. Initially a 2D CAD aficionado, Pillers decided to try 3D CAD packages for her more complicated consumer product designs. She liked what she saw in SolidWorks. In addition to her own business, Pillers wrote CAD reviews for various publications. CAD companies often sent her software to review. But SolidWorks went one step further and asked her to join their beta testing program. "They even flew me out to meet their developers," says Pillers.

"At the time, I was modeling handles for a number of travel mugs that were slated to be sold at Starbucks," she says. The desired shape required the smooth transitional shape of a loft, but needed a relatively sharp depression at the top for a thumb spot. A single body alone could not achieve the results she needed. Boolean-type operations would have been perfect, but SolidWorks did not have them at the time. She eventually used a cavity operation. "But the whole operation was very complicated and time-consuming," she says. This also made the result non-parametric, leading to issues when anyone wanted to make a change.

"Although it wasn't in time for my Starbucks project, SolidWorks did hear my cry for help and now includes the ability to maintain multiple bodies within a single file," says Pillers. "So the next time I run into a similar situation, the model will only take me a couple of hours, instead of a week."

Most Significant Suggestion:

In addition to the ability to maintain multiple bodies within a single file, Pillers was instrumental in developing the SolidWorks 2D program. For example, she was not satisfied with the way the software handled layering of imported AutoCAD files. "The developers spent a lot of time with me on this, and the results are evident," Pillers says. "Reading and writing DXF and AutoCAD DWG files is seamless now."

David Dearth, for MSC.Software

David Dearth has analyzed just about every FEA software package. "As part of my consulting business I've had to become fluent in every commercially available FEA code," says Dearth, president of Applied Analysis & Technology, an engineering consulting company located in Huntington Beach, CA. But he is a beta tester for MSC.Nastran. "Nastran is the de facto standard by which all other codes are measured," says Dearth.

When testing either Nastran or any analysis program, Dearth shies away from the tutorial problems that come along with any new software package. "Those tutorials are usually choreographed, reviewed, and rewritten by the company sales and marketing staff," says Dearth. Instead, for his evaluations, he runs through real life applications looking for three things: that the programs do all they claim to do; seamless integration and continuity in the software architecture, functionality, dialog boxes, input screens, etc.; and the ease with which he can generate either CAD geometry or analysis scenarios to tax the software to the limits.

In his own company, Dearth discovered that the best way to ensure the software performs correctly is to first put the program through all the normal operations, and then return to every past documented program bug. "It is uncanny how software bugs that are thought to be gone forever tend to crop up again," he says.

Dearth has also learned that every FEA software code will produce accurate solutions if used properly. "My most significant suggestion for anyone is to 'know thy code,'" he says. "Some of today's push-button FEA codes make the analysis process orders of magnitude easier than before. But the word easier is more a sales and marketing word. No FEA analysis is easy."

Most Significant Suggestion:

Dearth found a bug that would prevent installation of the program software if the computer ID had all numeric characters and no alpha characters. "A lot of times computers arrive from the distributor with all numeric computer IDs," says Dearth. The bug would have prevented installation of Nastran unless there was a least one alpha character. "That problem took some real detective work," he says. "No telling how many calls to the technical service center I prevented."

In his last beta test evaluation, Dearth discovered that all of the run code options needed for nonlinear transient analysis were not being written out correctly. The bug was the result of MSC improving their dialog boxes to make the nonlinear transient analysis less complex.

Paul Howard, for I-DEAS

Goodrich engineers use I-DEAS for the design and analysis of their aircraft wheels and brakes, which are used on everything from small business class planes to the space shuttle. So they were thrilled when what was then SDRC invited Paul Howard, designer, to be a part of their beta test program two years ago.

Testing, a two-week affair at the EDS headquarters in Milford, OH, involves an assigned project. "EDS purchases some off-the-shelf product for us to tear apart, model, detail, analyze, and assemble in the software," says Howard. "This year it was a mountain bike." In addition to the main project, the testers bring in their own special engineering challenges to run through the software.

Besides an in-depth look at the new version, Howard gets to hear things that are not out there for the general public to know about, he says. "I also gain an inside track to the developers," Howard says.

For the first time in several years, he is recommending that his company install I-10 as soon as it is released. Usually Goodrich waits several months to see what other companies find wrong with a new release. Howard says, "But during this round of beta testing I never did crash it."

Most Significant Suggestion:

Howard's major area of suggestions this year involved filleting. "My biggest problem is trying to fillet between two surfaces that don't touch and have different lengths," he says. "I want to be able to specify a plane where the fillet will stop at either or both ends and fill in as a solid." Currently the software will put in a surface (non-solid) fillet that matches the longer of the two surface lengths, he says. This requires Howard to do lots of trimming and stitching of surfaces to get the desired result. "While they made some good improvements in that area, it still was not up to doing some of the cases that I need it to do," he says. Maybe in the next release...

Ken Grundey, for Solid Edge

Ken Grundey, Engineering CAD Specialist for Pella Corporation, makers of Pella windows and doors, has participated in Solid Edge's beta testing program since prior to the release of version 1. "At that time, the software was sent to all Intergraph I/EMS customers. I decided to install it and started sending beta reports in," says Grundey. And he's been submitting suggestions ever since.

To test, Grundey uses his current workflows to catch regressions. "Regressions occur when features that worked in a prior version no longer work in the new version," says Grundey. He also modifies current workflows to incorporate applicable new features.

Being a beta tester benefits his company. "I can ensure that when the software is released, workflows that are important to my company continue to work," he says. It benefits him as well. "I learn what the new features are and how to use them long before the rest of the user population," he says. And in addition, Grundey gains a unique perspective of the company and its personnel. "This has resulted in numerous internal contacts, many of which I call friends," he adds.

Most Significant Suggestion:

Grundey's most recent suggestion involved a Broken View break line situation involving both horizontal and vertical breaks. Grundey suggested that the open space between the breaks should not contain intersecting break lines. This functionality was then added before the release. Grundey also suggested that the new symbol for the profile constraint known as the Connect looked too similar to the symbol used for the Symmetric constraint, which resulted in a new symbol for the Symmetric function.

Julian Holt, for Algor

When Algor wanted a rigorous beta tester, they approached Julian Holt, partner of CADFEM UK, a UK-based consultancy specializing in Finite Element Analysis. "In my opinion, there is no excuse for poorly designed software these days," he says. For the software to pass to his satisfaction, it must function as advertised, and be easy to use so that the operator uses minimum mouse clicks. "Ease of use is very important because people have much less play time and need to be working with the software quickly," says Holt. Over the years, Holt has learned that new features generally are added to releases before existing features are improved. "This is entirely understandable because the drive for new business has to take priority," says Holt. "But, the more users who ask for features, the better chance those features will be introduced." Holt particularly enjoys watching the software evolve and helping steer it in the right direction.

Most Significant Suggestion:

About five years ago Holt suggested introducing Sparse solvers, which has since been incorporated. He also suggested Algor add within the code the ability to perform Cyclic Symmetry analysis, which he felt would be a major time saver, but that has yet to be accepted.

Michael Best, for AutoDesk

Three short years ago, Robotic Technology Systems PLC (RTS Wright, Nashville, TN), manufacturers of integrated automation solutions, bought 70 seats of AutoCAD Mechanical and 30 seats of Autodesk Inventor, to enhance their 2D and 3D software packages. With the company making such a drastic transition, Michael Best, software support engineer for RTS, decided to apply as a beta tester on the Autodesk website for future AutoCAD and Inventor releases.

Along the way, RTS Wright and Best developed a partnership with Autodesk that enabled them to become an alpha site for AutoCAD and Inventor software builds. "As an alpha tester you work directly with the development team," says Best. Alpha testers are an elite group. Each tester reports to one person on the developer team. Issues found by testers and verified by the developers are reported and are addressed in future software builds.

To test, Best downloads the new software from an Autodesk secure website and applies it to in-house projects he's pulled offline. Not only does he check for bugs, but also he particularly looks at the enhancements to evaluate how they will benefit his users.

So far, the benefits have been quite rewarding. Last spring, after beta testing Revision 5.3 of Autodesk Inventor, Best felt so good about the program that he implemented the software five weeks before it was officially released.

Most Significant Suggestion:

Within the short time that Best has been alpha testing AutoDesk Inventor, he's made quite a few suggestions. "We suggested adding some enhanced check-in and check-out procedures, which were added in Release 6," he says. In addition, Best asked for better layer group capabilities with color options in the AutoCAD Mechanical product. AutoDesk responded with a mechanical extension, posted last summer. "We also suggested adding a PDM vaulting feature, which is being considered for future releases," he says.

Dennis Steffen, for PTC

Dennis Steffen, President of Plantation Key Design, a product design consultant company in Tavernier, FL, isn't a typical beta tester. But as author of Inside Pro/ENGINEER, he receives early releases. Steffen then meticulously compares the new release with the guidelines indicated in the structure of the book. Instead of feature suggestions, Steffen primarily debugs the new code. "I've been running Pro/ENGINEER for about 15 years. In fact, I use to work for one of the first companies in the state of Florida to buy the software," he says. "Because PTC developers come from a mechanical engineering background, they know how to design products and develop a corresponding CAD program."

Most Significant Suggestion:

Steffen suggested doing away with the option to have the "Intent Manager" function on or off. "I also have suggested a more flexible licensing format that would allow either a floating license scenario or permit a user to load up to two machines with one license," he adds.

Ted Harris, for ANSYS

"The ANSYS family of software has been closely tied to my livelihood for the last 15 years, both in industry and now in my role at an ANSYS support distributor," says Ted Harris, Lead Engineer, Tech Support & Training for Phoenix Analysis & Design Technologies. Harris volunteered to beta test using ANSYS's web-based sign-up process. To test, Harris reruns models that had problems in previous versions. "The first thing I do is check to see if known problems in prior versions have been fixed," says Harris. "Then I look to see if the new features work as documented. I try to find problems by using the code as a user would in day-to-day operation." The process is important to Harris as it pushes him to learn aspects of the code that he might not learn as quickly otherwise.

Most Significant Suggestion:

"We worked closely with ANSYS and our customers to enhance the graphical user interface," Harris says.