01 March 2003

Triple redundancy embraces network safety

By Ellen Fussell

Plant accidents heighten our awareness of safety, no matter what the cause, as evidenced by the 29 January explosion and obliteration of a pharmaceutical plant in Kinston, N.C. In fact, this month's issue of InTech explores how the promises of networking with a safety bus could help deter plant explosions.

While safety buses are on the horizon in the processing industry, experts admitted they are way down the road, so in today's plant, traditional networking is still the best way to stay safe.


Almost every DCS has its own safety system, "but when you sell a product, you have to be certified by independent agencies," said John Kolomiec, manager of TMR products at Triconex in Irvine, Calif. With a certification from TüV, Triconex can now sell TMR products to the nuclear industry because they passed yet another level of certification.

In fact, in August 2002, Triconex customer Florida Power received a technology transfer award from the Electric Power Research Institute for the first U.S. installation of a Triconex unit load demand system. Florida Power installed the system at its Crystal River nuclear power plant in Crystal River, Fla.

The federal Nuclear Regulatory Commission (NRC) issued a safety evaluation report that certified Triconex's Tricon Version 9 for use in 1E safety-critical applications in nuclear power plant instrument and control systems. This makes it the first TMR technology to receive NRC approval.

Triple modular redundancy (TMR) is one way to more effectively keep the plant safe-using traditional networking. "Say a control system fails, and a pump cannot shut off; a tank is filled with flammable liquid and it keeps filling because the pump doesn't shut down," said John Kolomiec, manager of TMR products at Irvine, Calif.-based Triconex. As a part of the entire safety aspect, Kolomiec said the company put a level switch in the tank, "and when the level switch triggers we shut down the pump. It's a watchdog over the control system to make sure it doesn't put itself in a hazardous situation."

TMR could be used in refineries or production offshore platforms, so it's crucial systems are able to react. Originating with the space industry, TMR is all about having three levels of redundancy. It's an architecture in which three isolated systems are tied together with diagnostics.

While TMR is effective, it isn't all that new. It still uses multiple controllers around the plant, which is "what people have been doing for the last 20 years, since DCS came out," said Paul Gruhn, president of L&M Engineering in Houston. "Unit 1 can know what's going on with unit 2, 3, and 4 because they're all sharing information."

Kolomiec admitted the menagerie of controllers in Triconex's system is in fact proprietary to the company and isn't new to the industry. Yet the thing that makes it work so well is the checks and balances, or redundancy, Kolomiec said. It goes through a series of cyclical redundancy checks, inverting the last few bits on a message and sending them back, which tells the sender the message was received on the other end, he said.


"There's no such thing as 100% diagnostic capability," Kolomiec said. "There's always a chance of something failing and you not knowing it." In safety systems, with redundancy on machines, there is no longer one channel but two channels. If something fails and isn't caught with diagnostics, high and low state changes occur between two systems, so it shuts it down. "And if something goes out of kilter, our system shuts down the process," he said. That's where TMR comes into play.

"We can operate in a mode of triplication. If we lose a component on one of those redundant levels, we can still operate on two channels," Kolomiec said. "If we lose a component on one of those other channels, we can operate on one channel. Our mode of degrading is called 3210." Dual systems are called 210, and other companies degrade to 320. "So we have another level of degradation we operate on. In our certification documents, there are 20 restrictions on how it can be used."

Controlling shutdown systems with TMR isn't reserved for Triconex, though. The Hidrojen Peroksit hydrogen peroxide plant at Bandirma in Turkey uses GE Fanuc's Genius modular redundancy to control its emergency shutdown system, according to a 28 October engineeringtalk.com article.

The plant required risk-free changes in making the switch to GE Fanuc's system. It also wanted to avoid interruptions from single system faults. A distributed control system (DCS) controls the entire plant, which handles 925 I/Os, and the TMR emergency shutdown (ESD) system handles 138 I/Os. The ESD system uses three GE Fanuc 90-70 programmable logic controllers whose CPUs are running identical software. A Genius bus controller connects all CPUs to the same I/O system through a Genius bus-a TMR arrangement-or what GE Fanuc calls its Genius modular redundancy systems.

The system controls the hydrogenation, oxidation, and filtration units-core components in the organic section and distillation unit. The operator hears sound and sees light to warn him of any malfunctions. The system also has a hard-wired emergency trip button in the workstations. But the control system is responsible for the plant if the DCS fails or there's a trip condition.

Plant managers insist the system reduces cost of ownership, even though fault-tolerant technology costs more than traditional simplex controllers. IT