"Murphy's Lawyers," Feature Article, May 2004

Murphy's lawyers

Watching for whatever can go wrong, because it will.

by Paul Sharke, Associate Editor

Steve Goldman, the late proprietor of Goldman Machinery Dynamics Corp. of West Nyack, N.Y., once wrote a book on vibration spectrum analysis, partly in response to what he called a "reluctance to travel." An engineer at Nash Engineering in the 1970s, Goldman began training technicians there in the fundamentals of machine vibration. He hoped that by their learning to analyze vibration spectra he could avoid having to go "throughout the known universe solving problems."

Training those future analysts bought him time to pursue machinery crises of his own choosing—because they were especially tough, politically sensitive, or right around the block. Meanwhile, a cadre of technicians roamed the greater globe. A book grew from his teaching.

Predictive maintenance, an umbrella term covering such techniques as vibration analysis, lube oil evaluation, and thermography, has since matured to the point that it wears its own designer label. In recent years, major manufacturers have been busily buying up PdM equipment makers.

GE Power Systems purchased Bently Nevada Corp. of Minden, Nev., in 2002, for instance. That same year, ABB acquired DLI Engineering Corp. of Bainbridge Island, Wash. A few years earlier, in 1997, Emerson purchased Computational Systems Inc. of Knoxville, Tenn. Clearly, the industry has cycled from pupa to adult.

A deteriorating bearing was monitored closely until it could be replaced during a scheduled outage.

Despite the progress in automation, communication, and data manipulation that has allowed PdM companies to "monitor tons of machines efficiently and accurately"—as DLI's senior engineer, Alan Friedman, put it—the technology has not displaced people from the picture altogether. What has happened, as if fulfilling Goldman's wish of long ago, is that the machinery experts now get to stay home.

"Today, I can sit in my office and monitor a water pump in Mongolia or an offshore windmill in Sweden," Friedman said. "I can add my 2 cents to an automated analysis, and the owner or operator and service center can be informed the minute I hit ÔAccept.' "

That's some distance from the way DLI employees worked in 1977, when the company was formed. Among its first clients was the U.S. Navy. Back then, engineers would have to haul dozens of cases of analog gear on board aircraft carriers. Everything from setting up the equipment to analyzing the data took intense effort in those days—a real "chore," Friedman explained.

Along came computers, of course, and a crackle of digital development in portable data collectors, bar code readers, expert systems, Web communications, and PC cards that followed. Core vibration technology relating to transducers and data analysis methodologies remained stable by comparison.

Soon, plants could afford to train and outfit their own predictive maintenance staffs. No longer did DLI's engineers have to hop aboard ships to gather vibration data. Instead, they could teach a ship's force how to gather data and how to run the collection through a computer expert system while at sea. The computer, in turn, would spit out detailed reports on the health of the ship's machinery. DLI engineers on land audited the calls of the expert system to ensure its accuracy.

Now, the trend is reversing. "Companies are finding it more cost-effective to funnel the data to the experts instead of maintaining their own experts on site," Friedman said.

Learning to See

The steam plant at the University of North Carolina in Chapel Hill consists of two coal-fired circulating fluidized bed boilers and one gas- or oil-fired backup boiler. Together, they deliver as much as 750,000 pounds of steam hourly to the campus and hospital. A small generator provides 28 megawatts of co-generated electricity.

Dwight Morgan, mechanical maintenance superintendent at the plant, said many of the station's 100-plus machines are enrolled in a vibration survey program, and have been in it for at least six years.

A single technician runs the quarterly survey and acts as the main communications contact with DLI, which analyzes the data and recommends repairs. Spectral data files grow quite large, so DLI acquires the university's machinery information by way of a file-transfer protocol site. Morgan mentioned two examples in which the PdM program made good on its promise of prediction.

The first was the case of an 800-horsepower primary air fan motor controlled by a variable frequency drive. Morgan called the fan "a single source of failure." If it went down, a CFB would go out—half the coal-fired capacity.

"You could feel the high-frequency vibration on the outboard motor bearing with your hands," he said of the fan. DLI engineers examined spectral signatures of the fan and predicted bearing failure. They advised university personnel to step up the frequency of monitoring bearings and to plan on replacing them. The plant suspected bearing fluting, a problem associated with VFD-controlled motors. It began taking measurements once a week.

In short, the plant avoided an unscheduled CFB outage and disruption of the steam supply on which the hospital depends. The fan continued running for several months until the maintenance department could replace the bearing at a planned outage. In the period between, personnel kept a close vigil on the bearing's health.

When the bearing finally came out, the plant's staff returned it to the motor vendor. Vendor personnel sawed the bearing open and found clear evidence of fluting. In all likelihood, the fluting stemmed from stray charges developing in the rotor by way of the variable frequency drive. These charges would discharge across the bearing and generate flutes on the race, Morgan explained.

The FMC trona mine in Green River, Wyo., is a locale where rotating machines and reliability engineers alike have to keep working in spite of harsh conditions.

Shaft brushes have since alleviated the condition, he said. The plant has also purchased a spare motor to have on hand for this critical service.

Morgan's other example involved an induced draft fan that was diagnosed as having loose anchor bolts. What makes the case stand out is the way in which the data collection box itself made the diagnosis immediately after the data had been gathered.

Further investigation by Morgan and his engineering staff discovered out-of-spec anchor bolts that were yielding before they could be turned to the desired torque. A solution is in the works, Morgan said.

These examples demonstrate the range of diagnostic possibilities within a PdM program: in one, a fairly easy call within the reach of a computerized diagnostic system; in the other, a complex bearing analysis that required the additional well-rounded judgment of human engineers. In both cases, identifying the problem led investigators not only to the source of vibration, but to the mechanisms that were causing the damage in the first place.

Richard Adler, a long-time reliability engineer at FMC Corp.'s trona mine in Green River, Wyo., has talked more than a few maintenance supervisors into letting him have the failed parts. He's compiled a list of resources on his way to fighting a battle against an attitude he sums up as, "If it breaks, it breaks." He wants to know why something breaks.
Surprisingly enough, sales skill tops his list. "A lot of engineers aren't taught to sell," Adler said.

That can be a peculiar handicap for a fresh-faced maintenance engineer hoping to be welcomed into a plant because he wants to solve problems. "He can be considered a real threat" by workers, Adler explained.

Adler can be found around repair sites poking his nose into machines as mechanics take them apart. He's the one asking for a look at damaged pieces as they come off the broken machine. He's not afraid to interrupt a few mechanics who are under fire to get the machine repaired and buttoned back up.

It was his first boss who taught him to look at damaged parts for clues to why they failed. None of his subsequent bosses directed him any further toward that responsibility, he said. It was something he kind of took upon himself.

Adler earned the trust of maintenance workers at the mine by serving as a shift relief foreman. It's a typical assignment for maintenance engineers, he said. For an engineer wanting to analyze failures, however, such a job should be only temporary. You can't push a crew and do engineering work at the same time, he explained.

Adler wended his way into failure analysis "one failure at a time," he said. As management began to understand failure analysis as a key to reliability, Adler began tackling tougher problems that many considered routine repairs.

Adler's work begins after the predictive maintenance has finished. He sees the stuff that doesn't show up in very many handbooks.

A new pump, for instance, runs just 36 hours before its discharge rate falls off and its outlet pipe starts shaking. Spectrum data shows vibration predominant at running speed, a strong indicator of imbalance.

A directive comes down to remove and repair the pump. When they've opened the housing, mechanics discover that 15 of the pump's 20 impeller vanes have nearly disappeared. Some have been digested completely. The culprit? An unintended introduction of acid into the pump stream.

The turbine generator at the University of North Carolina co-generation plant in Chapel Hill provides electricity to the campus and hospital.

Or, consider the case of the bearing that ran for 14 years before failing. The replacement bearing lasted 14 months. The mechanic who greased the original bearing overfilled it regularly—a maintenance no-no, as everyone knows. But, in this case, regular overfeeding kept trona out of the bearing better than any seal could.

After that first mechanic changed jobs, his replacement lubed the new bearing by the book, which cautions appropriately against over-greasing. The particular machine turned slowly enough so that churning the lubricant wasn't an issue there, as it is for higher-speed equipment. The invasive trona, which shows up everywhere in the plant, made its way into the new bearing and, soon enough, destroyed it.

Anecdotes? Yes, but here we are down in the basement where prediction and reality shake hands. "Engineering graphs are curves that are nicely fitted to a scattered array of data," Adler said. "Maintenance exists because equipment is pushed to the realm beyond that curve."

A Trend in Need

Needless to say, Adler opposes a cookbook approach to reliability engineering. Many of the machine failures that he has solved over the years have been cracked only through creative analysis. He cautions against relying on artificial intelligence alone to resolve machine breakdowns.

James Taylor, president of Tampa, Fla.-based Vibration Consultants Inc., is no fan of automated alerts and alarms either. He equates the current emphasis on machine data trending to "applying deodorant under one arm." By that, he means that predicting the frequencies that bearings can generate as they're failing, in order to monitor them, can miss things.

Taylor, who has written several books on vibration analysis, has a few anecdotes of his own.

A reliability-centered appropach aligns a plant's business objectives and maintenance strategy to decide upon the best way to monitor assets.

A crack, it seems, had developed along the inner race of a spherical roller bearing on a paper machine. A time-based waveform measured at the bearing showed two or three pulses for each revolution of the shaft. Taylor correlated these pulses to two or three rollers striking the crack every time it cycled through the bearing load zone.

A spectrum display of the same point plots vibration amplitude against frequency, telling the analyst what levels of vibration are happening at which multiples of machine speed.

In this case, the bearing generated no fundamental ball pass frequencies in spite of a defect on the inner race. Harmonics of the inner race ball pass frequency turned up in the spectrum, but at levels so low that any noise or looseness would have masked the signal, Taylor explained.

This signature characterizes a well-damped pulse, he said. There's very little energy generated as the rollers hit the crack.

Taylor pointed out that an automated vibration monitoring program using alert and alarm levels might never have found this very serious bearing defect.

For this reason, Taylor recommends analyzing every data point measured on a machine through the work of a skilled analyst, or the application of a rule-based diagnostic system. He discourages relying on predetermined alert and alarm levels to indicate trouble.

Toward a New Rule

According to Eric Huston, vice president of business development at SKF Reliability Systems of San Diego, only a few failures are genuine surprises.

"Detecting bearing failures has developed into a science," Huston said. PdM can provide "adequate warning" of imminent bearing failure in almost every instance, he added.

In some ways, though, the science has leapfrogged the need. Since the early 1990s, when a deluge of sophisticated, inexpensive, portable data collectors led to the practice of monitoring everything, the industry has gradually moved away from a broad-brush approach.

"That approach today is often considered overkill," he said. These days, PdM and its predecessor, preventive maintenance, form part of a strategy known as reliability-centered maintenance.

Briefly put, RCM attempts to align business objectives with equipment maintenance, Huston said. It looks at machinery from several perspectives. An assessment of each machine gauges how its failure could affect financial, environmental, or safety concerns at a plant, on a ship, or in a factory.

From that perspective, each machine can be fitted with some combination of four approaches to its maintenance. Inexpensive motors, for example, or machines that do not perform critical functions, might be allowed simply to run to failure. At one level beyond that, operators would watch a machine for signs of trouble. At a higher level of sophistication, a machine would be enrolled in a time-based maintenance program.

At the highest level, where a machine's failure could severely affect a plant's safety, environment, or profit, a full arsenal of PdM weapons, including periodic or continuous vibration monitoring, lube oil analysis, and infrared thermography, would keep its eyes on things. "The idea is to form a fusion between measurements and maintenance strategy, aligned with business goals," Huston said.

Part of that strategy has to deal with attrition and across-the-board staff reductions, where a great mass of hard-won, tacit knowledge can disappear with a single analyst's departure.

Some industries rely on plant staff for acquiring machinery data, but send the data outside for interpretation and analysis.

Many organizations consider the knowledge of their production machinery to be intellectual property, just as an old-car owner knows his auto's idiosyncrasies better than anyone else does and knows what he must do to keep it alive.

For these organizations, buying the technologies and investing in PdM training makes sense.

For other companies that recognize value in predictive maintenance but don't have enough machinery to justify full-blown, in-house programs, hiring an outside service to gather and analyze data is a better fit than maintaining a staff to do it.

A hybrid version of the two approaches increasingly makes a logical choice for many organizations. Coupling an employee who gathers data to an outside analysis service offers the best of two worlds. It minimizes travel expenses and guarantees a supply of experts.

One way of hanging onto expert knowledge is through the use of decision support systems. Not exactly expert systems, decision support attempts to capture the information that, 10 years ago, a maintenance engineer would have kept in his head, Huston said. Such systems are giving plants a structured way to classify the various failure modes of their equipment.

The coming of age of predictive maintenance has spawned other trends. Some companies are outsourcing PdM programs to better concentrate their own maintenance resources on improving reliability and equipment availability.

Others are passing on the complete responsibility for life cycle costs to the equipment makers, and demanding that they, in the case of compressor manufacturers, for instance, deliver a certain volume of compressed air over the course of a contract. Maintenance of the machines involved in producing that air then becomes the responsibility of the original equipment manufacturer.

Still other companies are moving to centralized diagnostics, Huston said. This trend prevails especially in large, decentralized industries, such as power generation or pulp and paper making.

It was power generators that first took up predictive maintenance a couple of decades ago, according to DLI's sales and marketing manager, Ronald Bodre. Today, as some of the pioneers in the field near retirement, electric utilities are puzzling over whether replacing lost machinery experts or outsourcing their tasks makes better sense. Other industries that adopted PdM later may soon have to face similar decisions.

There can be little dispute that human analysts are an important element in the PdM equation, according to Bodre. Typically, 80 percent of machines will pass a PdM survey, he said. An automatic screening program or a decision support system can quickly separate good actors and bad, freeing up human analysts to devote their time to figuring out how the other 20 percent are going wrong.

In the 10 or 15 years that it's taken PdM to enter the big time, machinery balance and alignment have steadily improved as equipment owners have grown more aware of these ailments and their cures, Bodre said.

Today, bearing troubles account for a big portion of machine maladies. Finding the root causes of stresses that lead to premature bearing failures has become a priority for many maintenance departments.