The myth and magic of OPC
OLE for Process Control is a widely adopted industry standard for communication between applications in a control system: It’s a beautiful thing when it works
By Patrick Dixon and Michael Velarde
Many of you have dreamed of the day when a process control system would easily allow third-party applications to read and write data in a standard way without a custom, proprietary interface.
You remember the days when a vendor with a slick advanced control application spent weeks at your site just to get the darn thing to communicate with your digital control system. You remember two vendors sitting in a room with you trying to convince you the other guy is to blame for communications problems.
You remember staring at a protocol analyzer and digging through code and log files to figure out what is going on. You dream of an end to this nightmare.
Keep dreaming. The day has not quite arrived.
OPC (former known as OLE for Process Control) is a beautiful thing when it works. It is a widely adopted industry standard for communication between applications in a control system. Often times it works without a hitch, and applications automatically install and start talking with your system. It sits on a widely adopted standard.
The fundamental system enabler for OPC, Distributed Component Object Model (DCOM), is a field proven, stable, built-in capability of Microsoft systems, which are the predominant platform in process control at the control LAN layer and above. You get that warm and fuzzy feeling that you chose wisely in your selection of system design.
Unfortunately, there are times when OPC shows its uglier side. Communication does not work. There is little if any diagnostic information to tell you why. Possible causes run the gamut from something in the application, something in your particular system, or a gremlin within Microsoft Windows.
For example, consider the following message in the system Event Viewer:
DCOM Error None
10000 NT AUTHORITY\ANONYMOUS LOGON DECKER_OPC
“Unable to start a DCOM Server”
The error: “Access is denied.”
Does this help identify what is wrong and how to fix it?
If you like that, you may also see the following common OPC related error codes when debugging:
Since most of us have not memorized what these stand for, we have to look these up and see if there is a more helpful explanation. Here is what comes from one of the latest OPC debugging tools:
0x80040202: Unable to open the access token of the current thread
0x80070005: General access denied error
0x800706BA: The server is unavailable. Maybe the server is not enabled in the Firewall.
At least that last one gives you an idea of what to do.
Our purpose here is to share our experience with an OPC installation and to use that experience to seek better diagnostic support of this technology. Experience at the Austin Energy Decker Road power plant illustrates an OPC project and the real world challenges inherent in such projects.
At the application layer
OPC is a standard. It is not a product. It is not software code. It is a standard presented to developers of control systems and software applications for enabling communications between process control systems and process control applications.
It exists because there are features inherent in process control applications that other existing communications standards such as OLE, DDE, and others do not sufficiently address.
It provides communications at the application layer, not for field I/O and hardware devices. That is the realm of Foundation fieldbus, Profibus, Modbus, and others.
OPC is on and relies upon the Microsoft Windows architecture. It specifically is dependent on DCOM, a Microsoft mechanism for inter-application communication on a network.
OPC built on DCOM does not work on Unix, Linux, Apple, or other non-Windows systems, although plans for portability are being designed in future enhancement of the standard.
On one side of OPC, communications is an OPC server application. This application “exposes” data items in a process control system or application. By “exposing” the data, it makes it available to any OPC client application. The OPC client can browse these data items in real time and connect to them. The client can perform reads and writes once it has connected to the data item. OPC has provisions to make data items read-only and to limit “exposure” of specified data.
In addition, OPC has a security mechanism that relies on Microsoft system administration. Accounts and passwords in the operating system control access to data. Any OPC server that receives a request from an OPC client must authenticate that the client has access to read/write the data. The account and password of the requesting client application must match a user/account in the machine where the server resides. If there is no match, the server will not allow the client to connect.
A critical provider of power
Austin Energy’s Decker Road power plant has twin 400 mega-watt generating units (furnace, boiler, and turbine). The facility is 40 years old and had been operating on panel boards and vintage control systems. In recent years, the company has replaced and upgraded systems so they are the latest technology. Now they want to make use of this technology to open up the system for advanced control schemes and to reduce the effort to provide reports. There are two criteria for system design:
- Allow applications to easily read/write data without the high maintenance and cost of custom communication drivers.
- Security! This is a critical provider of power to the city of Austin and the Texas grid. This control system will not connect in any way to the outside world (Internet). In addition, functions such as plant operation, system configuration, and programming should only be accessible to the appropriate plant personnel.
This was an ideal opportunity to make use of OPC compliant systems and applications.
The design is to have a base control system for each generating unit running independently. On top of these systems is a fiber optic ring network, which ties in a plant historian and allows for specific data communication between the two systems. In addition, a Sequence of Events recorder connects on this network and provides inputs to the control systems. An OPC bridge also runs on this system and provides the bridge for all communications on the network. This ring is setup as a Microsoft Windows domain.
The last layer is an office network. This allows managers to access real-time data from the system and generate reports. It is imperative office personnel not be able to make changes that will effect operations.
The network diagram is here. In order to do this, the office network is setup as a separate domain. In Windows terminology, the plant domain and the office domain could be either separate domains in the same forest or separate domains in separate forests. Windows 2000 forces a two-way transitive trust between two domains in the same forest. A two-way trust enables users from either domain to access resources in the other domain. This is not the situation we want. We want the office able to get data, but not write (read-only). Therefore, we want to setup a relationship where the office system trusts the plant system.
Trust relationship for access
Both base control systems have an OPC server application that “exposes” process data to OPC clients. The plant fiber optic LAN connects by Ethernet to the base system node that hosts this OPC server application on each system. On the plant LAN is a server class machine running Windows 2000 server. This machine is the domain controller for the plant process control LAN. The base system nodes that connect to the plant control LAN are not members of the domain; they are members of their own base control system network. A machine cannot be a member of two domains at the same time.
This can present a problem for OPC based system design. Often, an OPC client must be in a domain that has a trust relationship to the server in order to get access and connect. In our case, we were able to connect with limited functionality. The ability to browse available data items in the server could not happen without an established trust relationship. We were able to connect to data items by hand typing the pathname in the client.
At this point, we are able to read data from both control systems through OPC. The historian can store data from both systems. For some specific cases, we are able to read data from one control system and write it down to the other.
The next step is to connect the office LAN and read process data for reports. Since we do not want office personnel to make changes in the control system, we intend to implement the office LAN as a separate domain in a separate forest from the plant process LAN. To place the office LAN in the same forest under a different domain would result in Windows automatically setting a two-way trust between the domains. Due to security concerns, we do not want this. Therefore, our choice is the only way to go.
We were able to setup the office LAN with its own domain and forest. We then tried setting up the one-way trust. Through several iterations, we were able to get this setup. We were able to verify the plant control LAN could access files on the office network, but the office network had no access to the plant LAN.
Our OPC client was unable to get good data from the OPC server in this configuration. We were able to connect and browse data items, but every data item would show “Bad data.” We thoroughly checked every consideration and were not able to find a way to get good values. There is no diagnostic that will tell you what the origin of the problem is. We ended up reinstalling the office LAN server to act as a separate domain in the same forest. This yielded the same results. Lastly, we made the Office LAN server a member of the plant LAN domain, and we were able to get good data. In order to address security, we decided to implement this through creating several user accounts in the office domain with restricted access. We also located the OPC client application outside of the office area.
What is most lacking in OPC is diagnostics information. If a client and server cannot communicate due to Microsoft security, there is no indication to the user that this is the problem.
This system has now been in place for over four years. Over the course of this time, there have been interruptions of data communications occasionally. Various individuals have performed maintenance, but there is a general fear and reluctance to touch it.
The system no longer uses the report LAN due to intermittent communication problems.
We installed a process historian on the control LAN. Austin Energy attempted to use OPC to bring in data points to the historian. This worked very well in some instances and failed in others. There have actually been intermittent interruptions in communications that have caused loss of historical data.
We brought third-party experts in to double check the configuration and set up of the OPC server. After reconfiguring the server, data flow will start off good and then fail for unspecified reasons. Austin Energy is currently looking into different options to harness these data points for the historian.
It is interesting to note security vulnerabilities have been associated with DCOM, which OPC relies on heavily. ISA TR99 states “communications like DCOM that dynamically open a wide range of ephemeral ports should be avoided.” It is also interesting to note current OPC standardization is moving away from the Microsoft Windows based architecture and protocols like DCOM.
OPC is a conceptual improvement upon custom communication drivers in process control systems.
What is necessary to make it a complete solution is diagnostic information that will help the user debug problems when they arise.
ABOUT THE AUTHORS
Patrick Dixon, PE (firstname.lastname@example.org) is principal of Dixon Process Automation Services. He has BS in paper science and engineering and 20+ years experience in industrial process control. He is a member of ISA and IEEE. Michael Velarde (email@example.com) is an engineering associate at Austin Energy.
OPC Foundation responds: We’ve solved the problem with United Architecture (UA)
Just prior to publishing the accompanying piece, InTech summarized the problems at the Austin Energy power plant and asked the OPC for comment. Here is their reply.
On behalf of the OPC Foundation, we are sorry that this user has had unfortunate difficulties and loss of data in his system.
We have addressed this problem with the OPC UA architecture, which we conceived five years ago. Vendors also solved this problem before OPC UA for COM came out using their own respective tunneling products.
When we first started OPC in 1995, the concept was to provide a standard interface for communication to devices on the factory floor and process control such that first-tier visualization applications had a standard way of reading and writing data in a highly efficient way.
We deliberately used the Microsoft COM/DCOM architecture, the best thing that we had at the time in 1995, and I can assure you that in 99% of the cases the system works fine.
However, that 1% is what concerns me. A majority of that 1% we can easily address using products from vendors that have certified their products for interoperability.
We do recognize the limitations of DCOM with respect to security and reliability. Many of the OPC Foundation vendors offer products that alleviate the headaches associated with DCOM.
The end user is right with respect to not knowing when the DCOM connection is broken. The delay for detection is unacceptable in industrial automation. The type of diagnostic information in this case is almost pointless, other than you would be able to look at either side of the equation and determine that the data has stopped updating and where the data dropped out.
Five years ago, we started the task, of developing a new architecture called the OPC Unified Architecture. The new sets of specifications technology and process focus deliberately on solving the problems of complete multivendor, multiplatform secure, reliable interoperability for moving data and information from devices on the factory floor to the enterprise.
In addition, because of our choices of scalability in our architectural implementation, we will withstand the test of time as technology continues to evolve. OPC Unified Architecture provides a migration plan for existing OPC products to migrate forward to the OPC UA paradigm.
Again, we’re sorry that this user ran into difficulties with OPC-based products, and we at the OPC Foundation would be more than happy to engage and help architect and resolve the problems that this end-user has had.
Please feel free to contact me.
Tom Burke (Thomas.Burke@OPCfoundation.org) is the OPC Foundation President and Executive Director. He has degrees in mathematics and computer engineering.