1 March 2007
What Happens in Plant Stays in Plant
INL control system plant assessments reveal inconsistencies that prompt a primer on security measures
By May Permann, John Hammer, Ken Rohde, and Kathy Lee
While players in automation are becoming more aware computer systems are vulnerable to cyber attack, especially those controlling our nation’s critical infrastructure, some don’t always implement control system security procedures and devices consistently and effectively. In response, the Department of Homeland Security National Cyber Security Division established the Control Systems Security Center at the Idaho National Laboratory (INL) to help industry and government improve security in those control systems.
While significant differences between control system architectures permeate the industry, supervisory control and data acquisition (SCADA) systems, process control systems, and distributed control systems make up the general configurations. To remotely manipulate a control system, an attacker must gain access to the control system local area network (LAN), discover and understand the process, and then control the process.
Fortunately, most control system networks are no longer directly accessible from the Internet. It is becoming more of an industry practice to separate the business LAN from the control system LAN with some type of network isolation technology such as firewalls, which keep hackers out and isolate the LAN from worms and other maladies that may infect the corporate network. A firewall can also separate the control system network into sub-networks called demilitarized zones (DMZs) and control access between them.
Onsite assessment work provides an opportunity for cyber security professionals to meet with industry personnel to increase the security posture of a specific control system installation. Some security issues are unique to each implementation, yet there are commonalities among some of these installations.
When prioritizing security tasks, consider cost, probability, and consequence. Balance the risk of an intruder compromising your system with the risk of potentially degrading system operability. Above all, the control system must be reliable and perform its required mission. So build security into a system before putting it into production or adding security into an existing system in small increments. When adding security to a production system, test it on a backup system first to allow quick recovery to the previous configuration in the event any security measure affects system operation.
Security policies and training
Effective security policies and procedures are the first step to a secure control system network. It is possible to apply some of the same policies used for information technology security for corporate systems directly to control system networks. The SANS Institute provides free templates for security policies and can be a valuable resource for control system network administrators in developing their own policies.
Security policies must be practical and enforceable. To ensure they do not impact productivity, are not cost prohibitive, and do not lack support, include management and system administrator personnel. Network and system administrators have the technical knowledge but also need authorization and support from management to implement the policy.
In several cases those in charge of the control system network do not have adequate security training, usually because of insufficient funding or appreciation for its importance. Training provides an understanding of a network architecture’s security implications and how to design a more secure network. Network administrators require constant retraining to keep them up to date with the fast-paced changes of the network security field, including the latest network architecture designs and firewall and intrusion detection system (IDS) configurations.
Since new techniques are constantly under development to attack and defend computer networks, everyone should go through training to protect against e-mail attacks. If formal training is cost prohibitive, you can get some of this information from books, papers, and Web sites.
The INL onsite system assessments identified cases in which network administrators are competent at designing and maintaining reliable networks, but do not understand the security implications of their designs. Some examples are networks with redundant firewalls, IDSs, backup system networks, and DMZs for non-control system critical computers. At first glance, these appeared to be secure configurations, but a closer look revealed direct connections from the DMZ to the control system network circumventing the firewall, incomplete firewall rules, and non-validated IDS signatures.
Sometimes operators need to share IDs and passwords because of critical system operations. Unacceptable consequences might occur because of a locked user ID or a forgotten password. Typical continual manning of operating consoles provides additional physical security that reduces the need for distinct operator user IDs and passwords.
Don’t leave the default password from the manufacturer on your control system and networking equipment because they can give an attacker easy access to the equipment that controls the process. Unless your control system software requires it, always change default passwords to robust, unpublished passwords. Implement a password policy that enforces strong passwords to greatly impede password cracking and guessing. The SANS Institute’s password policy provides guidance on creating, protecting, and changing passwords.
Some have found passwords in control rooms on small pieces of paper on the bottom of the keyboard and in a drawer. If a password is too complicated and difficult to remember, or changes too often, users will undermine their security in order to remember them. Complex passwords protect against some of the advanced password cracking attacks, but they create a physical and social engineering vulnerability an attacker could exploit. Never use an auto-generated password, but create one from pass phrases or other memorable means.
Response procedures, patches
Incident response procedures tell employees what steps to take if a hacker compromises a computer on the network. They ask questions regarding what indications are present that an incident has occurred or is in progress, what immediate actions you should take, whom you should notify and in what order, how you should preserve forensic evidence (if you should leave the computer on), and how you can restore affected computers. The National Institute of Standards and Technology has developed a Computer Security Incident Handling Guide that provides guidance to security personnel in developing an incident response procedure.
Operating system patches repair vulnerabilities that could allow an attacker to exploit the computer, but unique challenges to consider include system functionality, security benefit, and timeliness. For security, download patches to a trusted server off the control network, and burn them to a CD. Then use the CD to patch the machines on the network. Other methods of patching could include the same process, but instead of loading each computer separately with the patch, the administrator could feed the new patch into a patch management server on a secure DMZ.
The system vendor should test operating system patches for compatibility with their system and supply the testing results to users. Make the results available as soon as possible after the patch release to limit the length of time the user’s system is vulnerable to the exploit. Always test patches on a backup system first, before implementing them on a production system. This testing period should be long enough and include full operational evolutions to make any side effects apparent. Cases exist in which the vendor tested and approved the patch, but at installation, it rendered the control system inoperable. Therefore, even if the vendor tests patches and updates, they should test them on the backup or test system as well.
Some control systems are still vulnerable to exploits that have had patches available for a long time. A widely available tool could exploit a system not patched for the overflow vulnerability, thus compromising the system. This particular exploit would allow complete graphical remote control of the system because the attacker has control of the HMI.
Services or applications running on a system open up different network ports to be able to communicate to the outside world. Each open port provides a possible access path for an attacker that can send exploits and receive data. An attacker can only gain access to, and receive information from, the control system through an open port. The more ports and services accessible, the greater risk of successful exploits due to service vulnerabilities.
New vulnerabilities appear everyday in computer applications and services, some of which see publication shortly after discovery. Yet some stay a close secret, allowing a few hackers to exploit computers at will, with no patches available to stop them. Decreasing the number of installed applications and services decreases the likelihood of an attacker finding vulnerability on the computer. Remove all unneeded applications and services. Allocate adequate resources to ensure complete patching of all services and applications is up to date using the patches process.
A common problem with applications and services is they run with system or root level privileges. In this case, an attacker can cause the application to crash; the exploit code will run with those same privileges, giving him full access to that device. A number of software products run with these super-user permissions by default, and yet, will function running as a less privileged user. You should therefore lower permission levels of applications and services only to what is necessary for their required functions. Another common problem is allowing users to operate a computer system (consoles, servers, and the like) with more permissions than necessary. Carefully evaluate user accounts for interactive logon for the lowest set of permissions necessary.
A related issue is file permissions. Share files with only the computers and accounts that require them. Restrict the read and write permissions to these shared files and directories to the minimum required for each user. Give each user and process the minimal privileges necessary for system operation. This is an emerging practice in the IT world that you can bring over the control system domain as well.
Cases also exist in which users need to restrict file access permissions to prevent information gathering. In our assessments, we discovered systems with information and vendor code shared to all computers on the control system LAN. In one example, even though the control system network was segmented, an intruder gaining access to the network would have access to all control system-specific information. In another case, a corporation housed the critical control system diagrams and documentation on an anonymous file transfer protocol (FTP) server for engineers’ easy use. They could view this FTP server from the Internet, making all the data accessible to anyone seeking the information.
Security perimeter defense
We consider any connection into the control system LAN part of the perimeter. Often users don’t document these perimeters well and neglect some connections. You should know all entry points into the control system LAN, and your security policy should strictly manage them. Most common entry points include vendor access, typically through a virtual private network (VPN) connection or by dialup modem. A corporate LAN connection is connected to the Internet, so it has the most potential for giving an attacker access to the control system LAN. Routinely check for unauthorized connections such as modems and direct cables to give operators remote access to the control system.
Know where the communication lines go for your control system. Only access the vendor VPN when you need it, and then immediately unplug it.
About the Authors
John Hammer, May Permann, and Ken Rohde are computer security researchers at Idaho National Laboratory (INL) in Idaho Falls, Ind. Kathy Lee is a SCADA security researcher at INL.