Comment

1 November 2005

Cyber assessment methods

Here is the plan for enhancing control system security.

By May Robin Permann and Kenneth Rohde

The terrorist attacks of 11 September 2001 brought to light threats and vulnerabilities that face the U.S.

In response, the U.S. government is directing the effort to secure the nation's critical infrastructure by creating programs to implement the National Strategy to Secure Cyberspace.

One part of this effort involves assessing supervisory control and data acquisition (SCADA) systems. These systems are essential to the control of critical elements of our national infrastructure, such as electric power, oil, and gas production and distribution.

Since their incapacitation or destruction would have a debilitating impact on the defense or economic security of the U.S., one of the main objectives of this program is to identify vulnerabilities and encourage the public and private sectors to work together to design secure control systems that resolve these weaknesses.

This article will discuss the vulnerability assessment methodologies used in ongoing research and assessment activities designed to identify and resolve vulnerabilities so as to improve the security of the nation's critical infrastructure.

Lessons learned of vendors

The National SCADA Test Bed (NSTB) is a program of the Department of Energy – Office of Electricity and Energy Assurance (DOE-OE) to improve the security of cyber assets in the energy sector. The Idaho National Laboratory (INL) SCADA Test Bed is a venue for assessing the security of various SCADA systems and system configurations. NSTB work at the INL focuses on outreach and awareness, vulnerability assessment and mitigation, standards development and best practices, and the creation of new security assessment tools. Information obtained through this program is shared with vendors and industry in order to enhance security by helping control system vendors and customers secure their own systems against external and internal cyber attacks.

Ongoing research and assessment activities have revealed an effective methodology for identifying vulnerabilities and developing assessment methods to secure SCADA and other control systems. This assessment methodology, which resulted from lessons learned when testing vendor systems, helps vendors, utilities, and others assess and enhance security measures on their own control systems.

Assessment methodology

The steps involved in this assessment methodology are:

• Developing an assessment plan
• Configuring the test environment
• Assessing the system
• Reporting requirements
• Using assessment metrics for scoring

Assessment plan development

Security assessments should have a detailed assessment plan that specifies a schedule and budget, targets and goals, expected deliverables, hardware and resource requirements, rules of engagement, and a recovery procedure. This plan should be the product of the team assigned to perform the assessment. Assessing the vulnerability of an information system can be a never-ending task as one digs deeper and deeper into the system, exposing vulnerabilities and finding new exploits. Specifying the time and resources to spend on any one task keeps the level of effort in check. It is important, therefore, that the assessment plan specify targets of evaluation (TOEs) to be tested. The Common Criteria for Information Technology (IT) Security Evaluation defines a TOE as an IT product or system with IT security functions. This includes operating systems, computer networks, distributed systems, and applications. For this application, a TOE is typically a subset of the SCADA or control system.

As an example, a TOE for a SCADA system might be the alarms and commands to and from the field components. One method of attack to change alarms and commands would be to analyze the network traffic to and from the Human-Machine Interface and develop a man-in-the-middle (MITM) style of manipulation. In an MITM attack, an attacker is able to read, insert, and modify messages between two parties without either party knowing the link between them is not secure. A functional description of the alarms and commands and their importance as well as ways of accomplishing an MITM attack on the test system could be included in the test plan to help testers focus the attacks.

Based upon priorities and resources, an allocated testing period would define the level of effort to give this task. The test plan would help the testers get started by suggesting methods of attack based on the combined knowledge of the cyber and SCADA test members. Data requirements that describe the information that the team must gather before attempting the TOE would, in many ways, define the difficulty of the attack and characterize the attacker. For this example, a detailed composition of the message format and the complete process for changing both an alarm and a command would be valuable information for achieving this TOE. Finally, a definition of a successful attack lets the test team know when they have accomplished the task.

Generally, the team should prioritize TOEs in the assessment plan based on how critical they are to the operation of the whole system, and defining a corresponding level of effort for testing each one is necessary. Prioritizing the list of TOEs allows the team to specify those needing evaluation, with the flexibility of having additional TOEs to assess should time remain. Flexibility in the assessment process, however, is extremely important, and allowing schedule or resource adjustments to account for unexpected findings during testing is good.

Along with accomplishing the TOEs, management may expect the team to produce other deliverables such as intermediate and final reports. The assessment plan should list the expectations up front.

The rules of engagement define the knowledge, funding, and timeframe of the attacker(s) as well as their skills and capabilities. Defining the attacker aids in the development of the types of attacks and from where they emanate. Possibilities include a nation state, a well-funded terrorist, a hacker who has penetrated the firewall or otherwise gained access to the SCADA network, or an insider.

Outlining a recovery plan is imperative in case vulnerability testing corrupts the system. This is possible by using tape backups, ghosting, or mirrored disks that were not in place during the test.

Modifying alarms and commands

Testing environment pattern

The ideal environment to assess the control system is a safe—non-production—configuration. However, it is also desirable to have all connecting components and functionality available in order to confidently assess full system interoperability. This includes everything from the Inter-Control Center Communication Protocol (ICCP) link to the Remote Terminal Unit (RTU) connectivity. ICCP is the international standard for real-time data communication between control centers. An RTU works in SCADA systems at a remote location to collect code and transmit data back to the control station, as well as receive and implement commands from the control center. Ideally, these components work with actual signals for the assessment, although emulators and simulators may be necessary.

The ability to establish a typical SCADA installation allows the assessment team to more accurately test and make recommendations on the configuration and deployment of a production system. Understanding the system configuration is imperative when deciding where the biggest risks in the system may reside and, therefore, which TOEs are top priority. Mirroring the connections to external systems is vital when replicating this configuration. The assessment team must know where and how a control system element typically connects to such things as the Internet, ICCP servers, and RTUs. Accessing any of these elements from outside the SCADA network, such as the Historian database, should undergo testing based on their position in the network. Knowing the accepted and normal operating procedures is important in order to assess the likelihood of an attack being successful. The goal of many attacks is to alter information and/or commands coming into or going out of the SCADA system. A completely functional system, or reasonable representation, provides a realistic assessment environment since complete determination of the results or consequences of an attack is dependent upon all of the interconnected functionalities within the system. For example, assessing a system without a redundant backup system with automatic failover does not show how a system that usually implements with redundancy responds to an attack.

Vulnerability analysis tools

Penetration testing must happen from a machine that is not part of the SCADA system unless otherwise defined in the assessment plan (i.e., an insider threat). This replicates a typical attack scenario where the attacker must penetrate the system from his own computer.

Sophisticated attacks are often system specific and tailored to the target computer's architecture. Therefore, the attacker needs a similar computer to create and test malicious code. For example, if 64-bit processors are at work in the target SCADA system, the assessment team may need equivalent hardware and software for developing exploits specific to that architecture. Although in a test environment there is direct access to the target system, don't alter its configuration. A separate machine should install required software such as compilers, debuggers, and other tools. It may not be feasible to obtain test computers for each represented hardware or software configuration on the SCADA system, but doing so would improve the assessment time and results.

Some open source and commercial tools useful for assessing SCADA systems are listed. Specific tools are mentioned because they are commonly available and are the best/only option without developing a tool for the specific application.

It is important to note that these scans and exploits are running in a controlled lab environment. Any of these used on a production system could cause it to malfunction or stop operating. Legal restrictions on using scanning tools and exploits across a public network or your own computing systems vary. Check with your company legal personnel and get permission from the appropriate company authorities before running any of the following attacks.

Nmap: Nmap is an open source multi-purpose network-scanning tool used mainly as a port-mapper. It identifies open ports on each machine. This information then identifies the services actually running on each port.

Nessus: The Nessus open source remote security scanner performs nearly 6,000 security checks against a target system, detecting vulnerable services running on the scanned hosts and providing a warning level and recommended fix for each possible vulnerability.

STAT Scanner: The commercially available STAT Scanner by Harris Corporation is useful in evaluating Windows-based systems in greater depth. The package provides superior detection of vulnerabilities on Microsoft operating systems, applications, and components. STAT Scanner provides powerful reporting capabilities and is available for other platforms.

Ethereal: Ethereal is a widely used open source network protocol analyzer that allows communications monitoring between the individual SCADA system components.

Ettercap: Ettercap is an open source suite for MITM attacks on switched networks. It features sniffing of live connections, content filtering "on the fly," and other capabilities such as active and passive dissection of many protocols (even ciphered ones) and password grabbing from specific applications.

Metasploit: The Metasploit framework is an advanced open-source platform for developing, testing, and using exploit code. It is a powerful tool for penetration testing, exploit development, and vulnerability research.

Debuggers: Debuggers allow one to evaluate a program during execution or a program's state at the moment it crashed. The main functions are:

• Execute a program, given any input parameters which affect its behavior.
• Allow user to step through the program instructions or stop-and-start the program at any given point.
• Allow user to examine and change values at stop points or after it crashes.

Software development tools: These facilitate software development, giving the programmer a user-friendly interface for writing, debugging, and compiling the code. They are useful in vulnerability assessments for analyzing code for safe programming practices and for generating exploit code with which to test the control system.

Published exploits: Rather than reinvent the wheel, it is most efficient to use any published exploits for the components in the target system. This is a quick approach for verifying vulnerabilities in a system.

In-house code development: When no public exploits are available for targeted systems, the ability to develop exploits in-house is very valuable because it allows the researchers to not only report discovered vulnerabilities, but provide working exploits back to the customer.

Fuzzers: Fuzzers are an automated way of finding vulnerabilities by feeding the target application with a wide range of invalid input. Input that causes the application to respond abnormally or crash then identifies vulnerabilities.

Static code analysis: Static code analysis disassembles binary executables, looking for vulnerabilities. A disassembler can translate the executable into a higher-level language that is easier for the evaluator to understand.

IDA Pro: Interactive disassembler (IDA) Pro is a Windows or Linux hosted multi-processor disassembler and debugger. This tool helps in vulnerability research for code analysis and software reverse engineering. It is one of the most advanced tools for hostile code analysis, vulnerability research, and reverse engineering. It allows assessment teams to evaluate software on the SCADA system for simple vulnerabilities.

Subject matter experts: SMEs are some of the most valuable resources in the vulnerability assessment process. Experts in both cyber security and the target control system are crucial to a thorough security assessment.

Books: In addition to some of the software development tools, books on Windows administration, Linux administration, scripting, and command-line references are very helpful in testing for vulnerabilities. Security books can provide a process and identify vulnerabilities to test for in particular operating systems and applications.

Hardware: Often times control systems use operating systems and hardware not in wide usage. Therefore, it is important the assessment team have access to some of these less common platforms so they can build their own testing environment for use during an assessment.

Logs: One should save system logs during testing because they can serve to indicate intrusions. Information gathered during testing can later work for discovering these attacks on a production system.

Detailed weekly reports

Detailed reporting of all tools used against the SCADA system and the associated steps, findings, and system response is invaluable. This includes archiving the test tools and scripts used. Documenting this information as quickly as possible assures one doesn't forget information and saves time when trying to duplicate the attack. This information can then lend help in the future for writing reports and validating which tests took place and whether the system was susceptible to them. Reports also provide a way to reproduce an attack when confirming the hole no longer exists in the next version of the software. The goal for this process should be to document to the level where someone skilled in the area could duplicate the results.

The following guidelines make reporting and future verification straightforward:

• Write detailed weekly reports.
• Keep the report writer involved during the assessment, at least at high level, so report writing is easier and more accurate.
• Keep documentation of testing.
• Archive testing tools and scripts for reproducibility.
• Use configuration management.
• Reboot and revert to the original configuration between each test.

Proprietary or otherwise sensitive data collected on specific systems is not for external release. Internal controls could include physical isolation during testing, assurance that anyone obtaining vulnerability information acknowledges applicable agreements, and security on computer systems holding the data.

It is important to have a way to quantitatively measure the security of a system to determine its risk level, measure risk reduction due to security enhancements, and evaluate how it compares to other systems. Since there is no tool to perform these tasks, the best way to do it is to seek the opinion of cyber security experts. This is somewhat subjective, however. The Department of Homeland Security is currently working on a risk-based decision methodology that can calculate risks associated with potential terrorist acts that utilize control systems.

Information technology-based scoring systems do exist. Experience using these tools shows it is important to decide on a scoring system before the assessment begins. This ensures one can gather the necessary data for more meaningful scoring.

Many hacking tools summary

Methodologies for performing vulnerability assessments of SCADA systems are developing through ongoing research, with the goal of improving the security of the nation's critical infrastructure. This experience helps to refine the assessment process and provide industry with better options to secure their section of the infrastructure.

Lessons learned in a laboratory environment can help out in other environments as well. Those sites that operate with backup and test systems can perform vulnerability assessments on these systems.

Many hacking tools and resources are available for sale or as free downloads off the Internet. These tools do require some computer skills and require the assistance of a qualified IT/cyber security professional.

Behind the byline

May Robin Permann (may.permann@inl.gov) and Kenneth Rohde work in information and communications systems and cyber security technologies at the U.S. Idaho National Laboratory. This article stems from the paper they presented at the 15th Annual Joint ISA POWID (Power Industries Division) / EPRI (Electric Power Research Institute) Controls and Instrumentation Conference, 48th Annual ISA POWID Symposium, Summer 2005.