1 February 2007

People Power

To optimize systems, relationships remain key for steady improvements

By Aaron Wood

Communication is the key to establishing a continuous improvement environment.  That means before starting a system analysis, one must consider and evaluate the working relationships with the owners and users of the automation system.

Change doesn't happen instantly, it takes time. Developing these relationships has to remain a key aspect at all times. After all, improvements often occur via feedback, curiosity, and the usual give and take that happens on a daily basis.

The automation system must satisfy the customer. (In this case, we are talking about operations, management, and process engineering.) Automation professionals must build relationships with the customers to remain engaged and approachable. It is very important to value and listen to the customer. Ideas and opportunities evolve as a result of general established informal and formal interaction between the customer and the automation professional. Established communications will ensure the stage is set for "opportunity" information flow.

Talk it over

Often operations personnel will verbalize an issue or a general unexpected occurrence at a daily or production meeting. Operations and management may not sense a potential issue; while the automation professional may discover, after investigation, that a potentially significant software flaw is present or that an opportunity exists for improvement.

When someone learns of a possible solution to a problem or a potential improvement, they may not initially recognize all the advantages. One key is to not rely on input from one source to make or break an improvement implementation. Experience shows isolated cases of initial feedback may not be positive. Seeking multiple inputs provides more clarity for making sound project initiation decisions.

We may discover a potential opportunity and initially find out the same issue came up several years ago and people turned it down. Do not rely on previous efforts or opinions based on historical hearsay to stop an analysis effort. Rely on the facts.

System analysis may provide a sizable list of potential improvements. However, to get to that point, you have to ascertain what the potential benefits are and identify the real prospects. Potential opportunities and benefits should undergo comparison to develop a ranking that aligns with the customer. Keep in mind the costs may outweigh the benefits and could force you to cut opportunities. Along those same lines, you should estimate and quantify the costs as well.

The business case can include improvements to: deviations, cycle time, resource productivity, product loss or scrap, system uptime or downtime, raw material usage, safety incidents, and environmental releases.

Large complex automation systems present a challenge regarding the determination of an appropriate starting point. The user should base the initial system analysis on: system control philosophy consistency (batch vs. continuous control), operator interface consistency, automatic operations and batch integration, unit procedure complexity, operation and equipment module step applicability, abnormal event automation, and automation implementation process (potential risk mitigation). The general thrust of the automation improvement strategy should be to absorb or further automate manual manipulation into the batch model via new operations and modification of existing operations. The push of automation integration should result in reduction of required operator interaction and operator error.  

Batch vs. continuous

Existing building process control systems may contain a mix of batch and continuous applications of control software. Are similar processes or equipment controlled by different control philosophies such as batch vs. continuous control? Are the processes really batch, but you use a continuous control application instead? Continuous control applied to a batch process tends to require significant operator monitoring and action while batch control minimizes required operator interaction with the process. Improvement at this level can have significant impact depending on the breadth of the change.

A mixture of display graphic standards and presentations, especially for similar equipment or functions in a given area, can be an operator's worst nightmare. The more an operator can consistently recognize and execute given functions, the less operator entry/interaction errors will occur. 

Drying plant woes

In one real case, there was a drying facility that consisted of three batch-controlled rotary vacuum dryers (RVDs) and four continuous-controlled RVDs run by the same set of operators. The continuous control automation came online over 10 years ago for all dryers; while, batch control started up within the last five years for only three of the dryers. Significant differences existed in the operator interface graphics and operator/automation interaction control elements. A large portion of the operations staff were new, and the facility was understaffed. Operator errors associated with the continuous-controlled dryers resulted in 6.5 deviations/ dryer/year. 

The solution: The user upgraded the control software configuration for the four continuous-controlled RVDs to a batch control structure, utilizing the operations and batch recipe methodology and configuration already established for the existing batch-controlled RVDs. They also upgraded the operator interface graphics so all dryers matched from an operations perspective.

The results showed operator errors reduced to zero deviations/dryer/year. Also, they found unforeseen performance improvements. The transition time from the desired drying temperature endpoint to cool down improved an average 1.67 hours per batch, and the average dryer insertion duration reduced by 0.9 hours per batch. The drying facility had an approximate original capacity of 530 batches per year for the upgraded dryers.  The cycle time improvements increased capacity to 560 batches per year for the same dryers.


Automatic operations

Most processes fit under the automation umbrella, but manual operator interaction at some level always exists even if a facility is highly automated. Most non-automation personnel do not understand different levels of automation integration exist. For example, when a block valve connects to the control system and they see it displayed on a graphic, some personnel believe the valve is fully integrated and automated. From a system perspective, this may not be the case, especially if the valve works in conjunction with other automated elements to perform a function. One must identify where the operator performs manual operations within the automation. Are operators manipulating multiple automated elements such as valves or pumps to perform and complete a unit operation? Are operators performing unit operations outside of normal batch control? Automatic operations can perform the required sequencing and control element manipulation to provide the necessary function for a given unit.

In one real-life case, in order to execute a periodic tank bottom purge, operators were responsible for manipulating multiple valves at a periodic frequency to introduce a nitrogen purge into the bottom of a reactor for a given duration. Multiple instances of either not performing the function or not performing the function at the correct frequency were occurring.

By manipulating an equipment module, operators controlled recirculations of the reactor sample loop for a given duration. Multiple instances of not performing the function, not performing the function for the correct duration, and not initiating the function at the desired point in the process occurred.

Operators executed vacuum pump control by manipulating control or equipment modules to start and stop the vacuum pump and initialize the associated pressure control loops in order to start and complete vacuum pressure control for reactors. The operator might use a given vacuum pump multiple times for a reactor and process. Multiple instances of vacuum pump failure due to overpressure at the initial connection to the reactor were occurring. Inconsistent operator execution in starting and stopping vacuum pumps for given processes occurred.

Saving time, money

The solution ended up being these integrated into the batch control via new automated operations and phases.

The result showed zero instances of misoperations. The system took care of executing the function. Operators were not needed. Required operator time savings of an average three hours per batch added up to 1,100 man hours (or $40k) savings per year.  Incremental improvements can be significant, especially when the facility includes multiple reactors and process cells that utilize the same functionality.

In order to satisfy processing needs, an inefficient and complex series of automatic operations may occur by a unit procedure. This may be the result of the utilization of the existing automation "toolkit" of operations. Can a new operation method be more efficient? Could this simplify the configuration of the unit procedure and the subsequent processing steps? New operation or operation option development and configuration may be surprisingly straightforward, and you should examine them before proceeding with inefficient and cumbersome unit procedure sequencing if the situation is avoidable.

In one case, a unit procedure originally executed multiple iterations of vacuum distillations (endpoint by reactor weight) and operator prompts to confirm completion where the initial tank weight was unknown and might be variable.

The solution involved developing a new operation and phase option to incorporate a net weight distillation option.

The result showed unit procedure and process reduced from eight unit operations to one operation. This saved two hours of cycle time and simplified the process steps and unit procedure.

Operation and equipment module

Operators should evaluate the automation sequence of operations and equipment modules for unnecessary steps-steps that take time and not needed for given situations. Operations and equipment modules originally may have generically applied to different variations.  With that in mind, the process may include some unnecessary steps. This practice can consume precious cycle time. Upon further examination, operators can identify and remove unnecessary steps from the applicable operations or equipment modules.

In one case, the metered charge or transfer equipment module steps included waiting for the tank weight to settle prior to and after a charge or transfer even though the charge or transfer underwent measurement with a mass meter and did involve a tank weight measurement.  

The solution involved modifying associated equipment modules to remove this unnecessary step.

The result ended up reducing the cycle time by one to 10 minutes per charge or transfer. This adds up when 35 reactors are running and multiple charges or transfers execute on each reactor. 

Detecting an abnormal event

Abnormal events significantly impact cycle time. If abnormal events occur consistently and are essentially the same, this creates an opportunity for improvement in the automation environment, especially if it involves operator error. Can you detect the abnormal event? Can the response and recovery from the abnormal event be automated rather than rely on the operator to take manual action?

In one case, occasionally the product in the dryers would fail the lab assay at the end of the pre-defined batch recipe step to dry the product. At that point, operations would manually setup a new drying sequence (a re-dry). Due to the infrequency of this, operators would make errors in this manual setup or involve other personnel in order to setup the re-dry.

They ended up modifying the existing operation to allow for and specify the parameters for an automated re-dry after a failed lab assay result.

The result showed cycle time reduced by one hour for batches requiring re-drys. Deviations reduced from five per year to zero per year.

Risk mitigation

Are potential risks involved with the automation implementation process? Could one accidentally implement an unintended state for a control element during a download? We must guard against these potential situations. Can new steps minimize or eliminate the risk?

In one case, during the download of automation configuration for either the simulator or the production control systems, the automation system automatically prompted for an upload option. An upload is the collection of the on-line parameter values that are different than the hard-coded parameter values. If multiple downloads are necessary, one might accidentally request an upload and download unintended configuration data to the controller. At least one instance occurred where an upload accidentally occurred on the simulator system; it then transferred to the production system and downloaded to the controller.

The solution was to disable the upload function via the security features of the automation system for all users.

The results showed zero accidental uploads and subsequent downloads have occurred.

Existing automation systems often remain a mystery to operations and technical support personnel. The non-automation person may not have the control system knowledge and capability to detect issues and possible improvement opportunities.  The automation professional must be proactive and analyze their existing systems for improvement opportunities. At the same time, the relationship with the customer must grow, so the interaction and communication between the parties allows for a free interchange of ideas so any layer of the organization can make improvements. Each improvement individually may not have a huge impact on the business, but the incremental improvements can add up to significant results.  

About the author

Aaron Wood is a senior process control engineer at the Tippecanoe Laboratories at Eli Lilly & Company. His e-mail is WOOD_AARON_R@LILLY.COM.

Fast Forward

  • Automation improvement strategy works by building productive relationships.
  • If executed properly, change management will succeed over time.
  • Don't rely on input from one source to make or break an improvement implementation.



Models Unleashed: Virtual Plant and Model Predictive Control Applications

Preventive Maintenance, 3rd Edition

Bottom-Line Automation, 2nd Edition

Netting a model predictive combo

Charting the Future