01 February 2005

Process control adopts wireless

By Mark Nixon, Rusty Shepard, Aloysius K. Mok, Bill Bennett, and Deji Chen

Middleware approach precludes idiosyncrasies of wireless.

Wireless is all the rage. We do not need to repeat its benefit here. It suffices for us to say that those who ignore it risk missing the biggest opportunity.

In essence, wireless technology is about data transmission, and data transmission is a key part in process control. The benefit offered by wireless network is appealing for process control. An example is offshore oil drilling where systems on land communicate to systems on drilling platforms. In places where wire is difficult or expensive to deploy, satellite communication may be the best choice. Using commercial wireless network makes possible process control that was previously unachievable.

Communication bandwidth, disturbances, and cost have been very real impediments to moving monitor and control techniques into a much wider set of problems. Oil exploration, production, and distribution are areas where improved control would provide considerable benefits.

Challenges of delivery

There are three layers of networks in a distributed process control system. Level one is between the smart devices and the controllers; level two is among the controllers and the workstations; and level three is from the workstations to the outside world.

Level one runs the process. It has tight real-time requirement of high predictability and reliability. The network protocols are usually industry standards like HART, Foundation Fieldbus, DeviceNet, and the like. Level two supports user interaction, including configuration, control, and monitoring. It has less timing requirement than level one, but still requires good reliability. The network protocol could be proprietary or industry standards such as Ethernet. Level three is the gateway of the control system to other corporate systems, like accounting, inventory, and management decision systems. It would be nice to have good support at this level, but conventional networks like office network, intranet, and others are in use.

Wireless acts as a kind of network, and it can target any of the three network levels of a process control system. Level one requires reliable short-range data transmission. Data package sizes are small, usually less than 100 bytes. There is much debate on whether wireless can work at this level. Of different wireless technologies, ZigBee, Wi-Fi, Bluetooth, and Ultra Wideband work well at short range. One could implement fieldbus on top of them, or define new process control fieldbus standards for wireless communication. Self-healing mesh technologies such as those developed by Echelon are finding a place in this space.

Level two networks have longer transmission ranges and bigger data package sizes. They have stricter requirements than COTS networks. People in the process control industry are already using wireless technology at this level. Satellite, Wi-Max, and the like, support long-range wireless transmission; microwave and radio work for shorter distances. In comparison, bringing wireless to the third level draws far less concern from people of process control. Any conventional methods to replace wire line with wireless in non-process control world apply to this level.

Wireless as a new type of technology could introduce new support for a process control system. Wireless has applications in display and handheld functions as well. For example, people talk about a handheld device that could retrieve data wirelessly from field devices.

Wireless has its own challenges compared with wireline, such as bandwidth, delay, noise, security, etc. Data communication in process control introduces further challenges like delivery guarantee, delay guarantee, continuous data request, and asymmetric data traffic. When a wireless network is to do the process control work, even more challenges add up, such as reliability and cost structure.

The difference between wireline and wireless is big at the lower layer, but at a higher layer, the differences could be merely different parameter values, especially when both implement the same network protocols.

A distributed process control system

Network provides support

The approach we look at is replacing the wireline network in a process control system with a wireless one, with minimum possible change to the existing control system. We will look at replacing the level two networks.

We will assume the existence of the control system with wireline networks. This means building a new control system from the ground up. Assume the level two uses standard network protocol. If the wireline network uses proprietary protocol, the wireless version would have to implement the same protocol in order not to re-implement the control system. Assume a standard protocol enables us to apply COTS wireless networks. Since almost all COTS wireless networks support TCP/IP, we further assume, without loss of generality, the control system uses TCP/IP to communicate between the distributed workstations and controllers. In this way, the control system runs, but with flaws, if we substitute the wireline network with the wireless counterpart.

If the wireless network provides the same support as the wireline network, our job is over. Unfortunately, we have to continue to address the difference and its effect on the performance of the original system of wireline network. In the following, we shall define the framework to mitigate the flaws.

First, we'll define the middleware that captures the specifics of the wireless network. Then we'll look at how the specifics propagate during runtime. Then we talk about the necessary changes required for the control system. We think it is not possible without changes to the control system. Finally, we'll look at the angles to observe when configuring a control system with a wireless network.

We assume a basic commercial wireless network offering, on top of which we build a middleware for process control data communication. The middleware provides an application programming interface (API) to the process control system. The control system sends data to the middleware with transmission constraint requirements and little knowledge of the underlying wireless network.

The API provided by the middleware for the control system supports regular data communication, such as initialize, open, close, send, receive, acknowledge, and cancel. The middleware executes these commands by calling the corresponding functions at the lower layer. In addition, the middleware will monitor the performance of these calls. This registers as a set of parameters in the middleware. The difference of parameter values tells the difference of the underlying wireline and wireless networks. The parameters will keep track of the following, some of which are configurable:

• Flag indicating if the node is on the network
• Type of network connection
• Message timeout value
• Message round-trip delay
• Number of message retries
• Flag indicating encrypted data and type of encryption
• Flag indicating if the node has a redundant network connection
• Auto message timing indicating whether the low level automatically calculates the message timeout and re-tries based on round trip timing or the of configured values
• Flag indicating if there is failed communications or bad integrity
• License information
• Security information

Execution of the middleware

During execution, the middleware carries out the commands from the control system and maintains the parameters current. Ideally, the middleware would be able to adjust itself automatically by taking into account the response time, bandwidth, number of packets, number of unacknowledged packets, delay times, and cost—perhaps even choosing between transmission media based on time-of-day.

One major work the middleware should perform is to maintain network connections. Wireless network could be intermittent. The middleware should handle this gracefully. Depending on the configuration, the connection could be permanent between two nodes or could be ad-hoc whenever connection is available. In the case of redundant connection, the middleware should switch between primary and standby connections without the notice of the control applications. The middleware could add variable retry and timeout times for each connection to account for propagation delays. Alternatively, it could allow multiple outstanding messages for more efficient use of bandwidth.

During active data transmission period, the middleware should perform a suite of housekeeping tasks. These include:

Round-trip delay timing: The initial round-trip delay is the time between the synchronous request and the synchronous response during connection establishment. The round-trip delay should update over time by timing the delay between sending a message and the receipt of an acknowledgment. Round-trip delay value adds to the message header to communicate the initial value to the passive end of the connection and to keep both ends of the connection in agreement of the current value.

Send message processing: Remote connections need to have the ability to support multiple outstanding sent messages. We could add a window parameter that defines the limit on how many messages are to be outstanding at a time, change the message sending function to send all messages on the pending message queue up to the window limit, request an acknowledgment only on the last message sent, and add a timer to trigger sending messages that have queued while waiting for an acknowledgment on a previous message.

Receive message processing: When acknowledging a message, copy the time sent value from the received message into the acknowledgment message. This will work to calculate round trip times and will provide a mechanism for validating that the acknowledgment is associated with the acknowledged message. When an out of order message arrives, return an acknowledgment for accepted messages to that point. This will prevent messages that have already successfully logged in but not acknowledged from undergoing needless retransmission. When an acknowledgment comes back, the time sent value checks against the time sent value in the message undergoing acknowledgement. If these times match, the round-trip time averages into the round-trip time value.

Retry and timeouts: To support multiple outstanding messages, each message must have a timeout value, not one time for the connection. As messages receive acknowledgement, they drop from the acknowledgment timeout queue leaving the remaining messages to time out at their proper times. An acknowledgment may acknowledge multiple messages. All messages within the window that precede the sequence number undergoing acknowledgement get their acknowledgement and drop off the retransmit queue. The system ignores acknowledgments for messages with sequence numbers that are not in the window. Management of the retransmit queue must handle timing out multiple sets of messages sent at different times with each message given the same fixed amount of time to remain on the retransmit queue before timing out. The retry value should depend on the configured timeout or the round trip delay. If the round-trip delay is used, the retry value is a minimum, and the round trip delay multiplied by two. The timeout value for a link bases on the number of retries for a link. Slower, perhaps less reliable, links may use a larger count.

Message packing and unpacking: To better utilize the given bandwidth on a remote connection, it may be desirable to pack as much information as possible into each packet. This mechanism is only valuable if it is common for several small messages to be queued waiting for a message to be acknowledged or for the window to open up. To accomplish this, one may need the following enhancements: As messages drop from the pending message queue, if two or more messages will fit in a single message buffer, the system will allocate a large message and all available messages that will fit copy into the large buffer. As messages are received that contain more than one message as a result of being packed by the remote peer, new receive messages are allocated and data is copied from the packed message into individual message buffers to be processed. All messages contained in a single message will list under a single acknowledgment of the last sequence number on the packed messages.

Other optimizations: The extra middleware layer enables many kinds of optimizations. If several applications request the same data, we would get multiple requests for the same data sent over the wireless link. For remote network connections using satellites, modems, or other slow or delayed transmission media, we could collect up the runtime data on the remote side of the network over the satellite (or other slow connection) and distribute it out to the other side. This reduces the amount of message traffic sent over the slow network. The remote applications would then get their runtime data from the local middleware instead of requesting it over the satellite link.

Finally, the middleware should handle link integrity and security. Since we are allowing remote connections that may connect via satellite or microwave type connections, we will need to provide security measures to prevent someone from being able to sniff the network traffic and break into the system. Providing the option of encrypting messages on the remote network connections should is necessary.

The middleware is a new component in the system. It has to be developed. This may be extra work for the control system.

It is impossible to run the wireline version of the control system without any change on this framework of wireless. Even if the middleware supports the same set of APIs that the control system invokes, the control system still needs additional function to access the set of parameters in the middleware. Of course, if the control system and the middleware grow up together for the wireline version, switching to wireless could be much easier. The goal of the approach is to mitigate the difficulties introducing wireless to process control. For wireless network, the control system should configure and monitor the middleware parameters. The control system could rely on the middleware for wireless communication, but the more monitoring by the control system itself, the better control will be.

Between two dedicated nodes

Given a wireless control system, how one configures it makes a lot of difference. We have talked about how to develop a wireless control system. Here we discuss issues involved in deploying a wireless control system.

Consider a setup with only one wireless connection between a workstation and a controller. The workstations, however, connect using hardwire as are the controllers. This reduces the wireless exposure and should result in better data quality. Many real applications can only afford wireless communication between two dedicated remote nodes, which act as the gateway for all their local nodes.

Another important issue is the types of data transmitted across the wireless networks. As wireless is less reliable than wireline, a system needs to configure such that only necessary data passes through the wireless links. For example, configuration database can go where most of the download happens; historian data archives where the data source is; diagnostic data retrieved on demand; large quantity of non-time critical data transmitted between production phases, and other scenarios.

Dedicated wireless nodes

Now we'll apply this idea to analyze an oil extractor control system.

The mechanism works this way. A canister lowers into a well, and after some time it fills with oil. The entire assembly then returns to the surface. The collected oil then meters and pumps to a storage tank. It's an entirely automated procedure. Radio wireless links to a master communicator, which then employs cellular technology to communicate with operators.

Some of the system requirements are:

• Allow users to remotely command the oil extractor unit.
• Allow users to locally operate the oil extractor with a hand held unit.
• Detect abnormal situations like leaks, plugged lines, and salt-water.
• Provide auto calibration and metering.
• Upload measurements and events to host through cellular links.
• Provide security to prevent an unauthorized user from tampering with metering information.

The wireless network used by the oil extractor provides performance statistics and cost structure. For example, we found the communication over Global System for Mobil Communication (GSM) using Short Message Service (SMS) messages encounter transmission problems between 7 p.m. and 9 p.m. Monday thru Friday.

The pricing structure of satellite tends to be for bandwidth. For cellular, it is a combination of packets and bandwidth. One advantage of using cellular is the ability to send abnormal situation events to individuals or response centers.

The middleware dynamically adjusts to the network performance to send data with the best result. Some of middleware's considerations are:

• The effective bandwidth of a GSM or satellite network may be less than advertised due to overloading, noise, and other problems.
• In the case of satellite communications noise is a significant issue, while in the case of a cellular network traffic is very real issue.
• Since the delay times with satellite communications is significant, we could send many packets out and acknowledge in bulk.
• With cellular at 2-5 cents per packet, it makes sense to keep the packets as large as possible by combining several data packages.

Behind the byline

Mark Nixon is lead system architect and Deji Chen is a software engineer working at Emerson Process Management with Rusty Shepard and Bill Bennett. Aloysius K. Mok is associated with the Department of Computer Sciences at the University of Texas. This article comes from their ISA EXPO 2004 technical paper presented in Houston. Read the entire paper at www.isa.org/intech/feb2005/wirelessfeature.