a touching sensation

Haptics technology lends the sense of touch to virtual reality. That might not sound very exciting, but it's being used to train surgeons and rehabilitate patients.

Haptics, a fairly recent enhancement to virtual reality technology, gives users the touch and feel of simulated objects they interact with, usually through a device like a specialized mouse or a haptic glove. The touch technology is making its way into key areas such as the automotive industry and medical practice, and a few companies are successfully marketing haptic devices, which trace their roots to academic research.

But even as the market for this type of touch capability looks ready to take off, some academics say the technology suffers from limitations that may keep touch from revolutionizing virtual reality applications.

John Hollerbach, a computing professor and an adjunct professor of mechanical engineering at the University of Utah, says haptic devices and robotic devices share the same drawbacks, particularly involving limits to the miniaturization of motors. But that hasn't stopped him from pursuing haptic research wholeheartedly. He's working on a project to add the sense of touch to the mechanical design process. If engineers could design by feel, rather than sight, he said, they might not need to build a physical prototype in many situations.

"Not only should you be able to look at the parts you're designing, you should be able to manipulate them by feeling them," Hollerbach said. "You build a physical prototype to figure out how humans interact with it. With haptics, you might want to build a virtual prototype and include the sense of touch to simulate how the driver would manipulate controls or put things in a glovebox. You'd be able to research reach and access."

Hollerbach has a haptic device that uses a commercial technology, the Sarcos Dextrous Arm Master, which is linked to the Alpha 1 computer-aided design system developed at the university. This haptic interfacing simulates forces of contact and gives users the ability to trace a surface or grasp and maneuver it, he said. The device allows users to simulate moving their whole arm to trace surfaces and squeeze objects.

One of the project's challenges, however, is that CAD systems aren't designed to interact in real time with haptic devices, and haptic devices aren't designed to operate over complex curved surfaces such as those typical of many manufactured parts. So Hollerbach and his students are having difficulties aligning the CAD and haptic systems to trace and to feel by virtual touch the parts that engineers are designing. They are making progress, however.

Eventually, automakers using the device could make a virtual prototype of their assembly lines and include touch in the prototype. That way, mechanics could use the prototype to learn how to assemble parts before the assembly line is actually created. They'd learn how to assemble the parts by feel, just as in real life.

And designers could ensure that the line runs smoothly before they build it, by operating it virtually, then checking by hand for interferences. For instance, a mechanic wearing a haptic glove could simulate reaching into a tight space and turning a screw. If the space were too narrowly designed for a human hand to perform the motion easily, the engineer would know to redesign that part of the line without ever having built a prototype.

Because haptic devices are robotic devices worn and manipulated by humans, they are limited in their usefulness by the same things that limit robotics, Hollerbach said. And that always comes down to actuator technology.

After all, motors don't follow Moore's law—which says the number of microcomponents that can be placed in an integrated circuit doubles every 18 months.

Haptic devices rely on very small motors that, in turn, rely on magnets to make them work, said Dean Chang, chief technology officer at Immersion Corp. of San Jose, Calif., a company that makes and sells haptic devices for markets that include medical, automotive, and gaming. The gaming industry is a widespread user of haptic technology. When you use a joystick to fire a shot during a video game and you feel the reverberation of the gun, the sensation is courtesy of haptic feedback.

Haptic devices that fit the hand, like the one sold by Immersion Corp. (top), or the force-feedback glove developed at Rutgers (bottom) give the wearer a sense of touch, as if one is actually squeezing a ball or tracing an object.

"But there are fundamental limitations on magnets," Chang said. "With haptics, you want to feel like you're cutting through tissue in a medical simulator and you need just the right amount of force for that. You don't want to feel like you're cutting through a rubber band. To generate that force, there are some minimal requirements. To exert the force, you need some kind of actuator.

"And that's the limitation. Magnets are what they are; it's not like you can do something to make them more magnetic," Chang said. "You can't change their properties and make them smaller."

The reliance on magnets will keep the price of haptic devices from falling as technology advances, as happens in the computer industry, where the price of computing power falls as chips get smaller. "Technological breakthroughs are going to bring down prices, but not like we've seen in other industries," Chang said.

Grigore Burdea, a professor at the Center for Advanced Information Processing at Rutgers University, said the actuators needed for the devices also keep them heavy and bulky. That's a drawback in an industry where users often have to wear the devices to feel the feedback. Burdea has been involved in haptic research for the past 15 years, after developing a force-feedback glove at Rutgers.

HAPTICS AND PHYSICAL THERAPY

Through the years, Burdea and his team have applied variants of that force-feedback glove to help people regain use of their hands after a stroke or surgery. The patients put on the force-feedback glove, which is connected to a computer, and simulate squeezing a rubber ball. The glove provides the feel of the ball.

"You can see the ball on the screen, but there's nothing in your hand," Burdea said. "The advantages of this are that it's easy to collect data while the patient is exercising. Eventually, we want to place this in people's homes so that people who've had a stroke don't have to go to a clinic. They often have trouble walking.

"We also have a philosophy that we want to make these kind of exercises fun, because they're very difficult to do," he added. To that end, the computer screen can simulate different backgrounds and games that the user can interact with and complete by use of the ball.

Burdea and his team have expanded the technology to what they call the Rutgers ankle, which aims to help rehabilitate the foot. A patient places his or her foot on a mobile platform and then performs exercises prompted on the computer screen. The platform resists the patient, with the resistance changing in accordance with how strongly the patient is pressing down.

The graduate students who helped Burdea create the ankle also thought of ways to make the simple exercise more enjoyable, and more challenging, for patients.

"They thought, let's have the patients power a plane with their ankle," Burdea said.

The Rutgers ankle simulates walking over different types of terrain, for patients undergoing physical therapy.

Because the ankle has different degrees of freedom, and can push, pull, or rotate, the plane motif is perfect. A flight simulator appears on screen while the patient is standing on the platform.

"Sometimes the physical therapist places hoops in front of the plane in different locations and the patient has to navigate through those hoops," Burdea said. "There's scoring, there's timing. We never get a complaint that it's boring."

When patients are able to fly the plane easily through different hoops, they can graduate to piloting through turbulence and fog.

"So you are really learning control, not just strength," Burdea said. "That's for people who fall. People who fall don't have good control."

A larger version of the Rutgers ankle allows patients to simulate walking, including walking over a variety of terrain like mud or gravel, or up stairs. The walking ankle beats simple treadmill walking—which is often used in physical therapy today—because it allows patients to master walking on an uneven terrain or on an incline.

But the very device that helps users simulate walking is also a barrier to its possible widespread adoption, Burdea admitted. All haptic devices, by their very nature, come in direct contact with users. How else could they feel the force feedback sense of touch? But that contact means that safety becomes an issue, Burdea said.

COST AS A BARRIER

"You can really get hurt with haptics if you don't pay
attention or if there's a malfunction," Burdea said. "Lawyers in the United States could get involved and make that an issue." He also cited cost as a reason that the devices might not become common in every hospital. Typically, the devices cost a lot because they're sophisticated new technology that also includes vision and sound technologies. Commercial devices sell for around $55,000 at least, and $100,000 is a common price, he said.

On the other hand, many applications today already rely on force feedback. Surgeons typically train on simulators that include the sense of touch, to simulate cutting through bone or to simulate navigating a medical device through the body, said Richard Stacy, vice president of Immersion Medical, in Gaithersburg, Md., a branch of Immersion Corp. Nurses and technicians learn how to give shots or insert IVs by using a company device that simulates the resistance of flesh as it is punctured. The simulator can be set to replicate a baby's skin, for example, or skin thickness on different parts of the body.

The advent of minimally invasive surgery has lent itself to developing many of the newer haptic medical training devices, Stacy said. For instance, another Immersion Medical simulator replicates the feeling of guiding an endoscope through a patient's colon. The endoscope is a long tube with a camera on the end that sends pictures of the bowel to a physician's computer as he guides it.

Haptics can simulate assembling a part to ensure that it is designed for easy construction.

"A key part of navigating the medical devices is what the surgeon or physician is feeling," Stacy said. "When they push the endoscope, it's not a straight shot. They have to navigate around a variety of curves, and the bowel is moving and constricting. If we can simulate that through haptics, that makes their training much more realistic."

Early medical simulator development—and by early, Stacy means the first half of the 1990s—was hampered by the cost of computing power, which has since declined greatly, of course, in part due to Moore's law.

"As the cost has come down, we've been able to achieve real-time haptic feedback and link it with graphics, so we can provide a real experience for physicians and surgeons that looks and feels like the real procedure," he said.

Some Immersion Corp. devices are finding their way into unique applications, like the Idrive in the BMW 7-Series. It is a means of controlling various functions, from tuning the radio to turning up the heat.

Immersion's vice president of sales and marketing, Joseph DiNucci, said the device should be thought of as a mouse that navigates the electromechanical systems in the car. Users may activate the Idrive and then scroll from left to right to adjust balance in the stereo speakers. They press again to find a radio station and they feel ticks in the drive as they dial, as if the device were mechanical, to guide them as they tune in a station.

The role of haptics is in the intuitive feedback that drivers learn to recognize as they use the device while driving, DiNucci said.

Hollerbach of the University of Utah said the future looks bright for haptics. The new generation of haptic interfaces growing at academic institutions today needs to make its way into the commercial realm, he said. "There are some decent devices in the marketplace, but there are some more that would really revolutionize the field if they were commercial-
ized," he said. "And that's something that will happen in the next few years. Of course, always there's the question of what the market will accept in terms of this type of com-
mercialization." Hollerbach expects the haptic drive, like the one already in use on the BMW, to find greater application in all cars, no matter what their cost.

Burdea predicts much more widespread use of haptics as this decade comes to a close.

"We're revolutionizing the technology now," he said. "As more companies commercialize it, the price will come down, too."

home | features | news update | marketplace | departments | about ME | back issues | ASME | site search

© 2003 by The American Society of Mechanical Engineers