August 2008

Automation IT

Plays well with others

Engineering, IT can come together with shared goals, new skills, technological tools

Fast Forward

  • Engineers and IT finding more reasons to converge for enterprise success.
  • Human aspect means sharing goals.
  • Long road ahead, but not hopeless.
By Ellen Fussell Policastro

Joe Pasqualichio is a consultant at Booz Allen in McLean, Va. He is still a newcomer in the automation industry, but even he has noticed how engineering and IT have almost become synonymous in the IT consulting industry. "The only people who seem to be getting hired are those with a background or degree in engineering," he said.

We have seen over and over how developing software technologies and security needs on the manufacturing floor are forcing engineers and IT to meld their minds-and expertise-supposedly for the good of the entire operation. With the convergence of people and expertise comes technological convergence as well-all cascading into a new form of automation where the lines between engineering and IT are becoming more blurred. The key is learning just how to meld the two for the greatest manufacturing success.

In the past few years, Scott Sommer, automation technology manager with Jacobs Engineering Group in Conshohocken, Penn., has noticed a shift from an IT-versus-automation situation to one of cooperation and collaboration. Conferences such as this past spring's Manufacturing/IT Summit in Cleveland are propelling the changes and "bringing an awareness to the need, challenges, and benefits of cooperation and collaboration," Sommer said.

People who need people

The secret behind merging automation and IT involves how to work with people said Eric Cosman, engineering consultant with Dow Chemical, in Midland, Mich. "It's all about partnering skills and how to break out of the mode of trying to convert the other group to their point of view. That's a waste of time," Cosman said. "You have to be able to hold on to what you do as well as understand what the other person doesn't and put the two of them together. That's how you become successful."

A perfect example is how IT and engineering work together in the area of security. People have said much about "how we need to bring the IT people into the plant and get them to understand what a plant's like," Cosman said. "But the reverse is also true. Engineers have to understand the constraints the IT people work under-how you might be able to put some of that to use. Take support of systems on a large scale. IT has struggled and made some progress in how to do that because most large companies have thousands of computers they have to support. They've learned what works and what doesn't. Some of those lessons learned in supporting thousands of computers could be used in supporting hundreds of plant systems, such as software distribution methods," he said.

Sometimes it is better to have someone else explain how they would solve a problem, even if you disagree with the methods they have chosen, than to start with a blank sheet of paper, Cosman said.

"If you have one process system to support, you don't worry. You get software from the vendor and put on a control system, and you're done."

But if you are a large company with hundreds of systems to support, you have to be efficient because in engineering you do not have the luxury of having people for every single one of those plants. "IT people have been dealing with that for years," Cosman said.

Learning new skills

IT professionals have become increasingly aware of the time domain (milliseconds, seconds, or minutes) that automation applications require and the reliability required for process control systems. "For IT professionals, this means learning programming techniques geared toward higher-speed, periodic execution as compared to asynchronous program execution. This also means having to understand the structures of process control systems, which you cannot easily modify on the fly," Sommer said.

Automation professionals have become more aware of network management tools, such as SNMP and network switch configuration tools, and how to use network components to manage the flow of data. Sommer believes as wireless access to networks by computers, sensors, and other components become more common, both groups have to learn how to complete wireless site surveys, manage wireless access points and wireless channels, and maintain wireless systems.

There are more skill sets required when you go beyond networking machines. Plant-wide networking, best practices developed in the IT world, and point of information are all being driven by the need for information, said Gregory Wilcox, networks business development manager with Rockwell in Mayfield Heights, Ohio. Because you are deploying Ethernet on the manufacturing floor there are of course security threats and a need to leverage IT best practices, such as intrusion protection. "There could be cases where somebody puts in a dial-up modem and forgets," he said. In today's world of security, modems are numbers. How do you access remote activity, hire contractors in a secure fashion? Those are the things IT has developed and can help those in manufacturing accomplish."

In paying attention to security, segmentation is the first rule-identifying your network or systems, such as who needs access and what devices need to talk to each other," said Paul Didier, industry solution architect for manufacturing at Cisco in San Jose, Calif. "Look for choke points and manage and control traffic.

A good example from the IT world is the enterprise communicates with the rest of the world. They put together demilitarized zones to share data. People from the Internet manage that traffic, but it needs to be tailored. That's what we've done with architectural work with Rockwell," Didier said. The plant floor tends to be a sensitive space; there is a lot of expensive equipment. So you need to secure it properly, and you need to maintain the type of interaction you want between the plant floor and enterprise and introduce technology into that space. That's why IT has some good ideas and it needs to be well tailored for the automation and control space."

Technology convergence

Not only are minds and skill sets converging on the plant floor, but the proliferation of wireless instrumentation, expansion of distributed control networks, and focus on control network security mean automation hardware components and IT software applications also need to converge. Both groups are essentially using the same switch hardware, same security schemas, similar network applications, and similar wireless hardware. "It has become evident that each group is playing in each others' sandboxes," Sommer said, "and the line between the IT domain and the automation domain are becoming increasingly clouded."

Converging technologies means a boon to the manufacturing organization, since the natural progression is to a single technology platform for networks (switches, routers, cabling, fiber optics, connectivity, and firewalls) and to a single security and application layer, Sommer said. "Even if pieces of the network come from different suppliers, standards such as CAT6, SNMP, ISA95, ISA100, and WEP, can help the organization focus on the application of these standards rather than spending a lot of time trying to get differing technologies to play well together."

Converging technologies for IT and automation will help decrease training cycles, reduce the numbers of personnel required for support in a joint IT/automation support structure, and improve efficiency in the delivery of support services to the manufacturing organization, Sommer said.

Because the hardware, software, and application platforms each use are similar, both groups have a vested interest in conforming to a single standard for the plant, Sommer said. "IT and automation are not out to devour the other; both groups realize there are domains into which they do not want to venture. While a data historian may be hosted on a database server residing in the plant server room, IT does not generally want the responsibility to configure or maintain the ActiveX controls residing in the SCADA application. By the same token, automation is usually willing to have IT configure and maintain terminal servers for the SCADA system."

Didier said he believes engineers and IT professionals "see the value of having converged networks, not just getting the information out of the plant floor into the rest of the enterprise and dealing with globalization issues. It's also the value the information brings to plant floor to be able to more cheaply connect in a more standard manner," he said.

Bridging gaps

Jay Mellen, executive vice president of business development at As One Technologies in Minnetonka, Minn., sees an opportunity to bridge the gap between the automation domain and IT professionals' category expertise with tools to help increase a project's likelihood of success. "We're finding creating domain-specific, visual-based assembly for programming tools allows engineers in manufacturing to build some of the solutions they need themselves. If not, it at least helps them communicate better with IT staff and get what they need."

Mellen's team has developed an environment and run time that is distributable; it can operate on one PC or a thousand. "You build a solution in a development environment, then deploy in run time," he said. Say a user has 300 different pieces of equipment in a factory. Each one communicates to enterprise systems within the factory through a network of computers all running on run time, running a host of solutions that either gather data from control equipment directly or condition data after it's been received, then put that data into a KPI database. It compares run to run, lot to lot, and data all through different applications.

"We have a layer of logic between the plant floor equipment and the enterprise applications. Because of the way a run time is built and development environment is built, that layer of logic runs concurrently. So it's not a client server model where you batch everything up and run a queue at midnight," he said. "If an event takes place in a piece of equipment, that event is communicated in real time throughout the network of computers."

Some of the tools in the development environment are domain specific editors that let the engineers go in and build a solution that allows them to monitor or control tool parameters based on a set of rules they design. "That's a visual editor that's easy to understand. It's complicated to program, but because it's through software tools, we're trying to lower the bar for domain experts to program or help them speak to IT staff to communicate what they need the software to do," he said.

Education biggest need

One of the biggest needs Wilcox hears is the need for education, creating awareness and dialog with both sectors-IT securing capital assets, and engineering's 24/7 manufacturing. "The Cisco and Rockwell Automation education series is geared toward that," he said. Cisco also has acknowledged networks to provide education and webcasts for education. "Look at the design and implementation guide itself, it provides a wealth of technology and security information as well as standards from ISA and ODVA," he said.

 "We also set up reference architectures for manufacturing in Cisco labs," Wilcox said. "We validated and tested and provided documentation for customers. It came down to segmentation for security and traffic management-VLAN, quality of service, and a multi-tiered campus approach."

"We've also done work on taking Wireless HART, and tested how well will that would work with a typical 802.11 WiFi network," Didier said.  "We're testing these standards to see how they play with each other. What do we need to do to get them to co-exist?"

Sommer said he believes all IT and automation professionals should take courses in the implementation of  ISA95, managing the SNMP network, SQL Programming, implementing wireless networks, and firewalls, routers, and network security, to name a few. Companies such as ISA, WBF, Cisco, and Microsoft teach classes in these areas.

Convergence possible but slow

Since automation is not the type of industry that turns on a dime, the convergence will take a while, Didier said. "You don't rebuild plants every year just because of new technology. In five years, I think there will be a higher level of comfort in networking and a strong desire to start incorporating technologies further into places as companies globalize and acquisitions come about."

There are also personal and human aspects, Wilcox said. "In the maintenance workforce, 60-65% will be retiring by 2010. How do we replace skill sets? Those are the people who grew up with automation networks. We need to backfill with people who come from trade skills and Ethernet."

But to accomplish all this, there has to be an open dialog, Wilcox said. "Work with your counterparts on either side. Make sure each other's group is engaged in projects earlier and earlier in the life cycle."

"The IT guys should spend the day in a plant and see what it's like," Didier said. "Control engineers should open the door and let them in. Sit down and talk to them; view IT as if you're their customers. Work with them to apply these technologies in an appropriate way."

Separate but collaborative?

Cliff Pedersen, a chemical engineer with 39 years in process control and refinery operations, believes the jury is still out on convergence. "Convergence will work for some people if you have an IT department and [a chief information officer or] CIO who is knowledgeable about the manufacturing process because that generates the wealth," he said. Pedersen has seen IT departments and CIOs who are computer smart but not plant smart. "Collaborate by all means, integrate by all means, force them to work together if necessary, but keep them separate," he said. "In most cases, it's a mistake to put plant automation under IT and have the CIO drive it."

Some solutions to this problem include standards-based interoperability, Pedersen said. The key initiative there is open operations and maintenance initiatives, such as integrating and interfacing plant automation systems with ERP systems. There is also the use of service oriented architectures (SOAs) and how it needs to mature. "This means everything down the road becomes a service," Pedersen said. "If a plant manager in a refinery wants to know how the finery is running, the first question is what's the crude rate? Not what's the flow on TFIC 40-29? So the SOA will allow you to get information without using the computer jargon and the computer lingo."

The server and systems will be architected so services will be provided from anywhere. It does not matter where they are, and they will respond. "It's sort of like Ethernet on an application level. All applications should be able to integrate seamlessly," Pedersen said.

Look for shared goals

Cosman disagrees that IT organizations cannot understand the needs of process control systems. "We have a group of people who manage and configure our standard process control system we deploy in our plant. That group is made up of engineers and IT people working side by side in one team to a common goal, successfully. The IT people bring specialized technical skills that if we use them effectively, we in engineering don't have to take the time to learn skills, such as database technology and operating systems. It's not about preaching to them what's different, but it's about bringing them in and making them part of the team and finding a common objective," Cosman said. "People then work very well as a team.

"Some say IT and engineers are different species," he said. "That's wrong. We're not the only companies doing. A lot of people resist the idea that these two can work together. But I see it happen every day. I think more people are coming to that realization. I see fewer examples now where people say categorically these two should never mingle."

The secret is bringing both teams in and having a common objective, Cosman said. "If you don't get the fundamental human dynamic stuff right, you'll never get to the common technologies. Some people say, yeah, that's obvious. But if it's so obvious, why aren't more people doing it?"


Ellen Fussell Policastro is the associate editor of InTech.  Her e-mail is