"Ask the Supercomputer," Feature Article, April 2006

ask the supercomputer

Many complex fluid flows still can't be analyzed today. Maybe tomorrow.

by Jean Thilmany, Associate Editor

Comparing blood to water is the mechanical engineering equivalent of comparing apples and oranges.

The apple and the orange are both fruit. Beyond that, they don't have much in common. Same way with blood and water: They're both liquid, but blood contains solids—platelets— flowing within the liquid plasma. The mix of solid with liquid spells the difference between a fluid flow easily simulated on desktop computational fluid dynamics software and a flow analysis months or years in the making—and even then, done only on a powerful supercomputer.

Blood is considered a complex flow, water a more straightforward analysis. Although a number of CFD vendors now sell desktop software that mechanical engineers (or their bosses) can buy to model complex flows, many problems are still simply too hard for those applications.

Those hard-to-model fluid flows typically have more than one type of force acting on them or they contain a mix of solids and liquids. To be solved, they need the power only a supercomputer can lend, said one researcher, Marek Behr, who is the chair for computational analysis of technical systems at RWTH Aachen University in Germany.

CFD programs for these complex problems can take years to write, even with the supercomputer's aid. And some flows may never be modeled: They're just too complex for even the most advanced software, say engineers at Concentration, Heat and Momentum Ltd. of London, which makes fluid-flow and heat-transfer software and offers consultant services.

SLOW GOING

Behr and a colleague, Matteo Pasquali, an associate professor in the Department of Chemical and Biomolecular Engineering at Rice University, are now at work writing a CFD application that will help a heart-pump manufacturer analyze how blood would move through different configurations of the pump. The pair have worked the past five years to come up with a way to analyze bloodflow through a heart pump. They say they're not there yet, though they're close.

The work is slow going as the problem is huge; it needs to account for time, fluids, and solids, and bring in elements concerned with the chemistry and the biology of the fluid.
Their complex problem is solvable—but only with help from the Rice University supercomputer. The school recently installed a Cray XD 1 supercomputer. Rice had been home to a computer cluster powered by Pentium processors and, before that, to a cluster powered by Intel Itanium processors, all of which Pasquali and Behr used on their quest.

The impetus to write an application for bloodflow through a heart pump came after a chance meeting five years ago.
When Behr and Pasquali were both at Rice University, they were working together on ways to depict fluid flow around a rotor, like that of a helicopter or a submarine. One day, Sebastian Schulte-Eistrup, a researcher at Baylor College of Medicine, noted that their rotor looked like the rotor on a centrifugal blood pump that his Baylor team—led by Yukihiko Nosé, a professor of surgery—was trying to perfect. Baylor College of Medicine is across the street from Rice in Houston.

Blood coursing through arteries contains both solids—platelets—and liquids, making for a fluid flow that cannot be simulated very easily.

The Baylor team couldn't find a CFD application that met their needs. Instead, they built a prototype pump, ran tests, and built another prototype. Software to model how blood would move around the rotor would help enormously and do away with much of the prototyping, Nosé told them. Maybe the Rice researchers could lend their expertise?

Behr and Pasquali agreed. But they soon discovered that the difference between air and blood made their already complex rotor problem much more difficult.

"In blood, about half the volume is red blood cells," Pasquali said. "You can think about them as droplets—although they're flat when at rest, surrounded by plasma that's like water. Having those droplets as 50 percent of the volume makes blood behave in a way that's different from the way water or air would behave."

According to Behr, "Classic mechanical engineering materials don't have a timescale. Water doesn't. But blood has all these capsules and droplets that can be stretched. The timescale is the time it takes for one of these blood cells to relax back to its shape once it's stretched."

And if the pump rotor were to shear those droplets, hemoglobin could be freed from the cells and leak into the plasma. At a particular mix, that hemoglobin in the plasma becomes toxic to the patient. The Baylor researchers wanted an application that could analyze for shear, which was obviously important to pump design.

Baylor wanted to predict how much hemoglobin would leak out when blood flowed through the device to keep shear below a toxic level.

"You have to design a pump that pumps enough blood within its small footprint but doesn't damage the blood cell," Behr said. "You need the pump to be small because it's in your body. But it pumps several liters a minute—the heart pumps five liters a minute—and that amount of flow in a device that small makes for a great shear rate."

CFD programs that depict blood damage usually depict blood cells that stretch rather than break because stretching is easier to analyze for, he added.

Pasquali and Behr spent two years trying to turn the pump geometry and performance data Baylor provided into usable data. They converted the pump's computer-aided design information and input it into their homegrown CFD program, then came up with software tools to rotate one part of the computationally meshed pump element with respect to another.

"Even though the pump situation was similar to the rotors we were working on, the geometry was different enough that we had to spend time and we had to try enough meshes with enough resolution to get the right flow features," Behr said.
Along the way, the Rice researchers received funding from the National Science Foundation to develop a model to describe blood damage caused by shearing. The two continue to work on the Baylor problem, but have now turned attention to another, similar problem funded by a Houston-based maker of an axial pump.

SOME ARE STILL FEASIBLE

Behr's and Pasquali's tribulations demonstrate the difficulty many companies have with CFD applications. Still, desktop software has been keeping up with the times and many vendors do provide able applications for complex flows.

For instance, Comsol of Burlington, Mass., makes what it calls multiphysics analysis software, which can analyze for the multiple physical phenomena often encountered in real-life engineering analysis problems. The company boasts that doctors used its multiphysics software to study the upper part of a child's aortic artery. The blood vessels there are embedded in the cardiac muscle, with little room for them to expand, so when the heart beats it puts pressure on the artery, deforming it. Analysis needs to take into account pressure as well as fluid flow.

Ansys of Canonsburg, Pa., makes a CFD application that can also analyze for other forces that happen in tandem with flow. Fluent Inc. of Lebanon, N.H., recently purchased by Ansys, says its software can analyze for multiphase—or complex— flows that include time as a factor.

Engineers can easily analyze only a minority of flows with today's software—regardless of whether the application resides on a supercomputer or desktop, according to a statement from Concentration, Heat and Momentum Ltd. Most flows are rather difficult, and many are impossible—at least until researchers like Pasquali and Behr get to work on them.

Complex
fluid flows
typically have
more than one type of force
acting on them or they contain a mix of solids and
liquids.

Even in cases of great difficulty, CFD simulations can give valuable results if engineers can simplify the problem succinctly and know that answers won't be returned with 100 percent accuracy. Instead, they'll fall within a range of uncertainty. If a company understands that, it might just come up with a CFD program that fits its needs, especially if it has years to devote to writing specific software and the money to commission supercomputing time.

That could be the case with the heart pump maker Micromed Cardiovascular Inc., which approached Behr and Pasquali about a year ago. The researchers began collaborating with Micromed on an analysis program to depict blood shear around a rotor—similar to the software program for Baylor, but this time for an axial pump. Micromed is working on a heart pump suitable for children and needs an analysis program that can scale the pump.

"Their initial pump was designed for a six-foot-tall, healthy male," Pasquali said. "So the pump was sized for the flow this 180-pound guy who plays tennis would have. Most of the patients that need this pump aren't that person."

The two can't say exactly when their analysis program will be ready for Micromed. They've turned to the Cray to compute what Behr called a very computer-intensive problem, the largest he's ever run. The geometry of the pump is quite complicated and it has to be analyzed for flow and time.

"It's an equation with five million unknowns," Behr said.

For example, the actual pump has no flow—obviously— before the rotors start turning. It takes about five or 10 revolutions after bloodflow starts before that flow is quasi-steady.

"In real life, that takes a fraction of a second," Behr said.

Depicting those five to 10 revolutions for analysis takes 10 supercomputer hours.

So work continues. But Behr and Pasquali understand the importance of their task for the children who will need these pumps. They're not giving up on the complicated problem.

"Don't ask me for a timeline for this, but maybe someday one could go to a doctor who would look at the patient and run a certain number of tests, and then give specifications based on those tests and send back a pump appropriate for a person of that size and that age," Pasquali said.

That may be a tall order, but then, think of antibiotics, hip implants, and the eradication of smallpox. Many of the things we take for granted today were once impossible, too.