Mechanical Engineering Design, March 2005 -- "The Deciding Factor," Feature Article

mechanical engineering design

the deciding factor

Design engineers make decisions for a living. Research looks for ways to make the process go smoother.

by Daniel D. Frey and Kemper Lewis

Think about the activities that engineers perform as they create a new design—setting requirements, selecting design concepts, choosing components, optimizing parameters. Each of these activities involves choosing among possibilities and committing the resources needed to pursue the chosen path. In other words, design involves a lot of decisions. Each decision influences quality, cost, safety, and other valued attributes of the design.

According to no less an authority than the Accreditation Board for Engineering and Technology, decision-making defines engineering design. ABET says that design is "a decision-making process (often iterative), in which basic sciences, mathematics, and the engineering sciences are applied to convert resources optimally to meet stated needs." The natural conclusion is that to be good designers, engineers should be skilled decision-makers.

Pugh Concept Selection: New methods question it, but don't rule it out.

Numerous methods and tools have been developed to lead engineers to the decisions that will improve a product, and more methods are being developed all the time. Many newer techniques tap into resources that lie outside the realm of traditional engineering in the strictest sense. Statistical probabilities, marketing goals, and financial management are being brought to bear in the early stages of design to focus the process and increase the product's chances for success.

At the same time, researchers are reassuring us that design will never become a simple numbers game. No one can quite define, or replace, the intuitive gift of the professional, sharpened by experience.

Bottom-Line Decisions

One major method under research is known as Decision-Based Design, or DBD. At its heart are economic models used to estimate the market impact of engineering decisions. These estimates require understanding of consumer demand and how it is influenced by product attributes like weight and efficiency. A model of consumer demand is not enough, however. To support a design decision it is also important to account for the attitudes of the decision-maker regarding risk. With these factors to consider, it is helpful to have a consistent scheme working out all the implications. For this, DBD relies on probability theory and a concept called "utility," which any wise decision-maker should seek to maximize.

The DBD framework is centered on making unambiguous design alternative selections under uncertainty and risk. Essentially, DBD strives to select a design configuration and price to meet a business goal by balancing profit and risk. To accomplish this, alternative designs are generated and then the demand for each alternative is estimated by using modeling techniques and input from engineering, marketing, and management. Knowing the cost of each alternative and the demand at a given price, you can estimate the risk. Lastly, choosing a design comes down to selecting the alternative that has the best balance of upside and downside. If all the alternatives are not known initially, this framework can be carried out iteratively in order to identify the best configuration of design parameters (for example, materials, geometric dimensions, product configurations, modules, and components).

Successful implementation of DBD must include effective information transfer among engineering, marketing, and management—each of which plays an important role in the DBD framework. Marketing is able to contribute information about customer buying behavior as a function of a product's attributes. Engineering makes a product's technical attributes a reality and can influence cost and demand by making appropriate technical decisions. Management is able to contribute information regarding economic and risk models in order to assess the engineering decisions.

While the DBD framework strives to integrate information across critical corporate departments, it also emphasizes sound and rigorous decision-making models and methods. For instance, DBD research has brought into question some current practices such as Pugh Concept Selection, a popular engineering decision process frequently taught in undergraduate engineering curricula and widely used by industry.

How the customer will use a product is a defining criterion, but not the only consideration, that will shape a complex design.

Pugh Concept Selection is a means of narrowing a field of multiple solutions to the same problem specification. Sketches of each solution are used to label the columns of a matrix. Then several evaluation criteria are listed and used to label the rows of the matrix. A single concept is then selected as a datum to which all the others are compared. The marks in the matrix indicate if the concept is better than, worse than, or substantially the same as the datum. The marks of each type are summed at the bottom of the matrix to provide a guide to the relative merits of the various concepts. This process can be implemented by a team with a white board and scrap paper. In comparing the concepts against a datum, team members begin to share and explore information. Frequently, teams find they can combine two complementary concepts and retain the strengths of each one.

One concern over this method is that it cannot consider interaction among different criteria. Consumers may have a positive reaction to tailgate access and high trim lines when questioned about these design aspects separately, but may dislike vehicles that have both tailgates and high trim lines. In Pugh Concept Selection, the criteria are in separate rows and there is no explicit means to account for interactions. Researchers in DBD seek to make comprehensive demand models that take the full set of product attributes (including interactions) and estimate economic measures such as profitability.

The way that DBD relates product design decisions to a company's bottom line has proven appealing to many companies. The goal of the Customer-Driven Quality Methods program at the General Motors Research & Development Center is to develop a suite of analytical tools for balancing market appeal and technology innovation in new vehicle concepts. The vision for Customer-Driven Quality is very similar to the vision of Decision-Based Design. Both include a combination of models from economics and engineering, and address the management of uncertainty and the risk that accompanies it. Both ultimately are used to promote well-formulated and well-informed decision-making.

"We appreciate the research on DBD because it focuses so many high-caliber researchers on subjects that are important to us," said Alan Taub, executive director for research and development at GM R&D. This focus has produced tangible benefits for General Motors as researchers from the R&D center have worked collaboratively with researchers at Pennsylvania State University, Vanderbilt University, the State University of New York at Buffalo, the University of Illinois-Chicago, and the University of Missouri-Rolla to advance the CDQM program. The collaboration is likely to continue because of the importance of realizing results in this area. "Developing methods to deliver high-quality, affordable, 'gotta-have' vehicles with appropriate levels of technical stretch will always be a priority for us," Taub said.

A Driving Force

More specifically, general motors is developing a set of analytical tools to represent the performance bounds of its vehicle architectures. These tools may be used in advanced vehicle design to rapidly assess the technical feasibility of desired performance targets. Application of these tools affects the early phases of vehicle development by making the process of balancing marketing and engineering requirements more efficient and statistically reliable. It also drives development of new technology by identifying the white spaces in the vehicle portfolio that are most desirable from a market perspective.

Ford Motor Co. and J.D. Power & Associates are integrating customer preferences in the form of demand analysis to bridge the gap between market analysis and engineering modeling. J.D. Power is polling the buying public to learn the design attributes that are important to potential customers. The results of the poll will form the link between engineering design decisions and customer demand and eventually profit, thus improving the competitiveness of future vehicles.

Praxair, a Fortune 500 firm that produces and distributes atmospheric, process, and specialty gases, has customers in the food and beverage, healthcare, semiconductor, and metal fabrication industries. Engineers and managers at Praxair are using DBD-based decision support tools to adapt existing product lines to new customer applications, to speed products to market that will reliably meet or exceed market expectations, and to increase product reliability in a newly developed product line.

According to Mike Hammill, a Six Sigma Master Black Belt at Praxair, "These tools have helped to strengthen the understanding of the importance of the voice of the customer in the overall product development process and have acted as a catalyst for change in the way Praxair will do things in the future." He added, "The tools, especially those centered around gathering the voice of the customer and decision-making, have already paid dividends in the development of internal management systems."

More than 50 companies are currently involved in an ongoing workshop on Decision-Based Design, funded by the National Science Foundation. A Web site, dbd.eng.buffalo.edu, serves as a forum for the group.

People use simple heuristics—the experience of trial and error—to make decisions under uncertainty. What is surprising is how well these simple heuristics work.

A naturalistic approach to research on decision-making is currently emerging from psychology and is beginning to affect the practices of engineering design. The research approach is based on observation of decision-making in the real world. Important decisions are being made all the time. People decide who to live with, what to eat, and when to dash across the street through traffic. An ability to make such decisions influences the survival and reproduction of individuals and therefore influences the evolution of our species. A goal of the naturalistic research approach is to help us understand the decision-making apparatus we carry around in our skulls and help us use it more effectively.

One of the leaders in cognitive psychology and decision-making is Gerd Gigerenzer, director of the Max Planck Institute for Human Development in Berlin. His research group has shown that people (and animals) use simple heuristics—the experience of trial and error—rather than complex probabilistic calculations, to make decisions under uncertainty. It is not surprising that simple heuristics are used in situations where decisions have to be made quickly. What is surprising is how well these simple heuristics work.

Even when more time and information are available, it is often better to use a simpler, faster approach using less information. For example, if a patient complains of chest pain, a doctor typically evaluates dozens of risk factors to decide if further treatment is needed. But research has shown that a simple rule based on just four factors, including blood pressure and electrocardiogram readings, is far superior in both identifying real heart attacks and in avoiding unneeded treatments. This rule has been implemented in hospitals and is improving care and reducing costs.

Acting on Instinct

Another leading researcher in the naturalistic study of decision-making is Gary Klein, whose research is described in a book, Sources of Power: How People Make Decisions. Klein's research group, Klein Associates in Dayton, Ohio, has been studying experienced decision-makers in a variety of professions, including medicine, aviation, and engineering.

He has found that experienced decision-makers use what he calls a "recognition-primed" decision model. People look for similarities between the current situations and past situations and develop plans of action on that basis. One anecdote from Klein's book concerns a fire chief who sensed that a fire wasn't responding as it should. His instinct was to order the firefighters out of the building. The floor in the building subsequently collapsed, because the fire was in the basement, not in the kitchen as they were led to believe.

Klein writes that "recognition-primed" decision-making is a teachable skill and has helped improve decision-making in every U.S. armed service and at firms like AT&T, Johnson & Johnson, Procter & Gamble, and Westinghouse.

Herman Miller Inc. acted on instinct when it released its Aeron chair, which was its best seller.

The results from cognitive psychology have been brought to a broader audience in Blink: The Power of Thinking Without Thinking by popular author Malcolm Gladwell. This new book includes a number of stories engineers will find useful. For example, in the early 1990s, Herman Miller Inc. developed a radical new ergonomic chair called the Aeron. All the market research indicated there would be little demand for such a design. Focus groups appreciated the function of the chair, but rejected its styling. But the people in charge decided to trust their instincts and go ahead with the product. The Aeron became the best selling chair in the company's history. It may have just been luck, but perhaps the experienced professionals at Herman Miller knew something that the market research couldn't identify.

What is it about personal professional judgment that sometimes enables people to make the right decision? What could be missing from detailed, rational analyses of market demands? One possible answer is "tacit knowledge."

Donald Schön, a former professor at the Massachusetts Institute of Technology, studied design-related professions such as engineering and architecture. In his 1983 book, The Reflective Practitioner: How Professionals Think in Action, Schön argues that much of skillful professional work involves "actions, recognitions, and judgments which we know how to carry out spontaneously; we do not have to think about them prior to or during their performance."

This emphasis on tacit knowledge may have major consequences for the way professionals are educated. Schön felt that it was a key to "healing the splits between teaching and doing, school and life, research and practice."

What lessons might engineers extract according to research by social scientists like Gigerenzer, Klein, and Schön? One lesson is that we cannot afford to ignore our innate decision-making instincts.

The latest research in cognitive psychology does not suggest that our gut feelings are infallible, but it does show that our instincts for decision-making are an important source of skill that should be understood and trained. Another lesson is that we should not automatically assume that a mathematically sophisticated approach to decision-making is better than a simpler one. For example, Pugh Concept Selection is a simple approach to get designers to share knowledge and converge on a solution. Cognitive psychology seems to suggest it should not be abandoned just yet.

New research on engineering decision-making is aimed at improving professional practice. At present, there are significant controversies, but a few messages have emerged that are fairly consistent:

• Decision-making is an important skill for engineers to develop. Making decisions skillfully will improve the quality of your designs, the bottom line of your organization, and your career prospects.

• Decisions are determined by values and affected by uncertainties. As a consequence, engineers should be knowledgeable about economics, probability, and statistics. But decision-making should not be turned into a number-crunching exercise.

There are lots of decision support tools out there and more are sure to come from new research. The validity of some common decision support tools is currently being investigated. Design engineers should know the assumptions behind decision support tools and should never use a tool as a substitute for sound judgment or professional experience.

Daniel D. Frey is an ASME member and an assistant professor of mechanical engineering and engineering systems at Massachusetts Institute of Technology. Kemper Lewis, an associate professor of mechanical and aerospace engineering at the University at Buffalo-SUNY, is also director of education and training for the New York State Center for Engineering Design and Industrial Innovation.