Life-cycle and long-term migration planning
Successfully upgrading and replacing systems in a running production environment
by Leif Poulsen
Automation and manufacturing information technology (IT) systems are characterized by a relatively short life. Often the systems have to be upgraded or replaced long before the process equipment has reached the end of its life cycle. For most companies, it is a challenge to manage upgrading or replacing systems in a running production environment, and often the need for upgrade or replacement is ignored until it appears as an unpleasant surprise. This article shows how it can be done successfully with careful planning and organization.
Two main factors drive the need to upgrade or replace automation and manufacturing IT systems: the technical deterioration of the automation and manufacturing IT systems and changes to the requirements of the business processes such systems support.
The reliability of technical systems will decrease over time if companies ignore migration activities such as upgrading operating systems, database systems, and application software. The operational risk of failure increases accordingly. With careful planning, the operational risk can be kept at an acceptable level, while protecting existing investments and minimizing life-cycle cost. For a typical automation/IT system, only 20 to 40 percent of the investment is actually spent on purchasing the system; the other 60 to 80 percent goes toward maintaining high availability and adjusting the system to changing needs during its life span.
Along with assessing the necessary migration activities to cope with technical deterioration, it is also important to assess new challenges and new opportunities from a business point of view. The business environment is constantly changing, and opportunities for improving existing or exploiting new technologies must be considered. Typical business objectives, which may be important business drivers for migration planning, include speed to market, competitiveness, growth, quality, and compliance.
Long-term migration plan
Creating a long-term migration plan helps companies to keep system operational risk at acceptable levels while meeting changing business requirements. A migration plan addresses risk mitigation and timely support of business goals. It takes into account important constraints such as current Good Manufacturing Practice (cGMP) compliance, technology functionality and performance, system support, and plant downtime.
Drawing the future system landscape by making all these goals and constraints add up with minimum investment is the key to a good migration plan. Together with a phased and robust implementation plan, this ensures a painless transition.
The overall approach to long-term migration planning is illustrated in figure 1. The migration plan takes an organization to where it wants to be in five years by steps that match the needed changes with the resources available to implement the changes. This approach is based on architectural design principles as defined in The Open Group Architecture Framework (TOGAF) standard, which is widely accepted for developing enterprise architectures.
Figure 1. Overall approach to development of long-term migration plan (based on TOGAF)
We distinguish between the current architecture and the target architecture, which correspond to the description of where the company is now and where it wants to be, respectively. The migration plan goes from the current to the target architecture, maybe via some temporary transition architectures.
Each architecture must be described at a number of layers with appropriate mapping between business and technology, as shown in figure 1. We operate with the following layers:
- Business objectives are part of the overall strategy work. This is valuable for setting the proper direction of the planning process.
- The business model provides context to understand manufacturing and business processes. It normally includes a high-level description of the material/process flow.
- Describing manufacturing and business processes is essential to successfully applying technology and accurately estimating business value.
- Information, data, and documents are essential to linking processes and applications. The main concern is transactions and orchestrating information flows between applications.
- Application descriptions form high-level requirements and define interfaces.
- Define infrastructure, computers, and networks requirements (hardware, availability, and performance).
- Enabling services define how to secure efficient and successful operational control and support of the solutions.
Developing the migration plan
The development of the migration plan for a complete organization, site, or single facility may be a complex task that involves a lot of people. It is recommended that groups organize the development task in the five steps briefly described below.
Step 1: Mobilize
The purpose of the mobilize step is:
- to get a common understanding of objectives and goals
- to mobilize the project organization
- to detail the consulting plan, milestones, and deliverables
- to gather available inputs
- to ensure a proper understanding of concepts, practices, and theory
During this step, the following activities are conducted:
- planning meetings
- kick-off workshop
The output is:
- detailed consulting plan
- overall goals
- process overview
Step 2: Analyze
The purpose of the analyze step is:
to analyze the business and manufacturing processes in order to
- assess readiness for automation and manufacturing IT support
- clarify data and functionality needs for to-be architecture
- identify key benefits for setting goals and business justification
to establish an as-is architecture
- existing manufacturing processes with possible relation to automation, data collection, and manufacturing execution systems
- existing business processes with possible relation to manufacturing systems
- existing data
- existing applications and interfaces
- existing logical and physical infrastructure
- existing support services
During the analyze step, the following activities are conducted:
- process interviews and workshops
- plant tour to get information in context
- as-is systems and infrastructure architecture interviews and workshops
- scoring enabling services compliance and maturity
The output of this step is:
- as-is architecture
- analysis documentation
- ideas list of challenges and opportunities
Step 3: Target
The purpose of the target step is to identify the needs determined in the analyze step and describe the targets. The solution or target architecture that is the goal will describe:
- to-be business processes and functionalities
- target application types with supported overall functionality, users, information, and interfaces
- infrastructure needs and the revised supporting services
During the target step, the following activities are conducted:
- process improvement interviews and workshops
- architecture improvement interviews and workshops
The output of this step is:
- to-be architecture (presentation)
- short-description of application types
Step 4: Justify
The purpose of the justify step is to make preliminary business justifications based on rough cost and benefit estimation.
The gap between the as-is and the to-be leads into a number of project ideas. A justification of the collected ideas shall be elaborated to separate need-to-have from nice-to-have and to present and review project ideas with management.
During the justify step, the following activities are conducted:
- rough cost and benefit evaluation
- draft to-be presentation
The output of this step is:
- overall goals summary
- business case/ideas prioritization
- resource need
Step 5: Plan
The purpose of this step is to plan execution based on priority, resources, and dependencies:
- to plan the sequence in execution of the consolidated project portfolio
- to ensure resources and competencies for the next steps
- to initiate governance activities
- to complete the consultancy and hand over deliverables
During this step, the following activities are conducted:
- implementation planning
- investment planning
- risk assessment
The output of the plan step is:
- implementation plan
- employee load profile adhering to the plan
- project risk assessments
- rough cut investment plan
- final presentation
The following example shows how to apply the described approach in real life. For confidentiality, all data in the case has been made anonymous. However, it is about a fairly large site producing active ingredients for a number of pharmaceutical products. The site was established more than 20 years ago, and although some equipment and system upgrades have been made since then, a number of outdated systems still need to be replaced. The building management systems and the distributed control systems (DCS) are especially based on old technologies, which are now hard to support. Now the site also has to adapt to new business requirements, including retiring some products and launching new products. So there is an overall need to look at a migration plan that covers both technical and business requirements.
First, it is important to create an overview of what is currently installed in the different facilities. This information is often hidden in a huge number of documents (and maybe also in the minds of people) and has to be extracted and visualized to be the basis for the migration planning. To do this, we typically develop a Process Module Diagram, which shows the main equipment and the material flow in each facility. As separate layers on top of this drawing, we illustrate which systems are supporting which equipment. An example is shown in figure 2. The data about the installed systems is also recorded in a system repository (or can simply be saved in an Excel spreadsheet), which can be used for further analysis and planning.
Figure 2. Process module diagram with automation layer to show current system installations
Before discussing the migration planning, it is important to identify the main business drivers for the changes on the site. In this case, site management stated the main business drivers as follows:
- right first time, compliance
- time to market, flexibility
- sustain success, competitiveness, operational excellence
- right first time, quality
- volume growth
Such business drivers have to be turned into more specific business objectives that can be measured. We need this to prioritize potential change projects.
Next, we need to know more about how well the existing systems support existing and future business processes. We use a standardized reference model (based on the ANSI/ISA-95 series of standards) to explore this. It comprises 19 high-level business processes that are broken down to the relevant level of details to understand weaknesses and the need for changes from a business point of view.
In addition, we also have to assess the technical capabilities of the existing systems to support the business processes in the future. This is done systematically with the information described above in the system repository. For each system in the repository (in this case about 70 systems), the following aspects must be assessed:
- hardware condition (history of failure or mean time between failure, age of equipment, availability of spare parts)
- software condition (support from vendors, availability of documentation, availability of competences)
- system restore capability (redundancy, mean time to repair)
- business impact assessment (disclosure of information, data errors, nonavailability)
- indicative scoring (system reliability, system criticality, and life-cycle management score)
The technical assessment reveals a need to upgrade and replace a number of systems:
- The process control systems are based on one common outdated DCS platform and many different programmable logic controllers, of which some are also ready for replacement.
- The building management system is based on a newer platform but needs an upgrade to fit new requirements.
- A number of the support systems also need some upgrades, and some need to be replaced.
- The infrastructure that is common for all systems needs to be better segmented and protected to meet current security requirements.
By looking at the business objectives for the future, it is evident that none of the existing systems can fully cope with the future requirements. This knowledge led to several ideas for introducing new technology, including a new manufacturing execution system. The analysis ended up proposing 16 different projects, which can be implemented in steps to meet both technical and commercial requirements.
The technical scope and the cost of each project are estimated, and a one pager for management discussion is prepared for each project proposal (see example in figure 3).
Figure 3. One-page description of potential migration project
To prioritize among the proposed projects, an impact assessment is made for each project. The impact assessment includes both an assessment of the impact on the business objectives and an assessment of the impact on the reliability of the systems.
Figure 4 shows how the various projects affect the main business objectives, and figure 5 shows how the projects affect the reliability of the systems.
Figure 4. High-level business impact assessment of migration projects
Figure 5. High-level reliability impact assessment of migration projects The next step is to refine the budgets for the selected projects (figure 6) and to schedule the sequence for execution according to the priorities and technical constraints.
Figure 6. Consolidated overview of migration budget (actual values not shown for confidentiality)
Normally you have to simulate a number of implementation scenarios to assess the overall need for resources and capital for each project per period in the long-term migration plan (figure 7). One of the major constraints to consider is opening windows in the production schedule where you can modify or replace the systems. Often these windows are limited to holiday shutdowns, and this may be a bottleneck for execution.
Figure 7. Consolidated overview of migration schedule
Because the time for the change and cut over to new systems normally is very limited, you have to prepare thoroughly. Everything has to be planned at the necessary level of detail. An important aspect of the planning is the validation of the upgraded systems; good advice about this is provided in reference 3. In this case, the execution of the long-term migration plan has been organized in six separate implementation programs, as shown in figure 8.
Figure 8. Organization of migration projects in six streams
Part of the preparation is also careful assessment and mitigation of project risk. Figure 9 shows typical risks associated with migration projects.
Figure 9. Assessment of typical risks by migration projects
The approach to life-cycle management and long-term migration planning described in this article is business driven. It includes an assessment of current and future business objectives and a careful analysis of how the technical system must be maintained, extended, or replaced to best support these objectives. Further, the approach is based on system architectural design principles (TOGAF), which allows for stepwise implementation according to the availability of investment budgets and qualified resources. It includes a description and assessment of the current and future system architectures as key elements in identifying relevant migration projects. Finally, the approach is based on organizational change management principles, which ensure timely involvement of relevant stakeholders to make the implementation of the migration projects successful. The feasibility of this approach has been demonstrated through many practical examples, as described in this article.
ABOUT THE AUTHOR
Leif Poulsen (email@example.com), senior specialist, automation and IT, at NNE Pharmaplan, holds a master’s and a Ph.D. in process engineering and is certified as a professional enterprise architect according to the TOGAF 9 standards. At NNE Pharmaplan, Poulsen is responsible for the development of technology, methods, and competencies within automation and IT and works as a senior business consultant for customers worldwide. He is an expert on business analysis and conceptual design of automation and IT solutions, including how to deploy such solutions effectively in a GxP regulated organization. He is an active member of the ISA88 and ISA95 standards committees.
Thanks to my colleagues Henrik Pedersen, Jens Bruun, Carsten Holm Pedersen, and Gilad Langer for valuable input to this article.
1. TOGAF 9.1, The Open Group Architectural Framework, Open Group Standard, 2009–2011.
2. ANSI/ISA-95.00.01-2010 (IEC 62264-1 Mod) – Enterprise-Control System Integration Part 1: Models and Terminology, May 2010.
3. ISPE Guide GAMP® 5: A Risk-based Approach to Compliant GxP Computerized Systems, February 2008.