ARTC COMPUTING 2000
CLOSING THE LOOP BETWEEN ERP AND DCS
By
Peter J. Lawrence
Resolution Integration Solutions, Inc.
ABSTRACT
With the successful implementation of many ERP (Enterprise Resource Planning) systems, and the wide acceptance of plant control systems, a significant functionality gap has emerged between the two systems that may be filled by the process plant equivalent of an MES (Manufacturing Execution System).
The reason for this gap is that: an ERP system will not need to deal at the specific operating instruction level; an ERP will not be dictating which tanks to use for blending, what operating conditions are required to achieve the required yield; it is unlikely that the ERP system is concerned with the specific tank location of the product; and etc.
Similarly, plant control systems require explicit instructions: line up a particular tank; optimize to a particular quality constraint; operating with the back-up pump, etc.
How are the details added to the ERP system’s material requests? What business processes and support systems are required to fill in the details such that they can be passed down to the plant control systems? How are deviations to plan, schedule, or operating instructions to be handled? Many refiners are seeking a solution to these questions.
At present, this business process is largely ‘manu-matic’: even though there may be analytical tools to support planners, schedulers, area superintendents, etc. (LPs, schedule optimizers, blend optimizers, unit simulators and optimizers, reconciliation, etc.), the information flow is manual.
The solution is not simple. It is not sufficient to ‘automate’ the business process by passing electronic files between the applications: terminology of each system differs, neither the underlying process model, the aggregation levels rarely agree, changes are rarely coordinated, consistency between the applications are rarely verified, etc.
This paper addresses these problems, and describes the solution implemented at several refineries, as follows:
· An analysis of the business information flow between the resource planning, refinery planning, scheduling, target setting, and operating activities reveals a helical information flow more reminiscent of DNA rather than a simple ‘feedback control loop’;
With the successful implementation of many ERP (Enterprise Resource Planning) systems, and the wide acceptance of plant control systems, a significant functionality gap has emerged between the two systems that may be filled by the process plant equivalent of an MES (Manufacturing Execution System).
Closing the loop between ERP and DCS has recently received considerable attention, particularly with the widespread deployment of ERP and Supply Chain applications. However, according to surveys only a few percent of companies that have installed ERP, have integrated down to the measurement.
The objective of this paper is not to look at the technical architecture but the business process architecture (BPA) that links these two areas. Following from an examination of the BPA, we then examine what impact does the business process architecture have on the technical architecture. Finally, has the BPA learnt some lessons that can be used to solve the technical architecture issues?
The benefits of ERP to DCS are frequently discussed but rarely enumerated. “$0.25 to $1 per product barrel for better integration of planning, scheduling, and control for gasoline blending” [1] seems to be consistent with our own observations. If the benefits are so large, why has the problem not been solved? Maybe we should look at the difficulties.
• Application solutions within a process plant have been deployed to solve a point-problem without reference to an overall business process architecture. This has resulted in an overly complex BPA, with both overlapping and missing functionality.
• Many of the applications have an implicit or explicit model of the process. Although much has been talked of the ‘single model’ concept for applications, this fails to recognize that there are legitimately different model metaphors to suit the purposes of the different parts of the BPA.
• Information is not required at the same level of detail for all processes within the BPA. This means that information must be aggregated before it can be transferred from one application to be understood by another. This implies that instructions have to be de-aggregated to be passed back, but this is a non-unique transformation.
Therefore, we can identify all of the problems why it is difficult to integrate between ERP and DCS [2] , yet process plants DO integrate between ERP and DCS, albeit without all the support that automation might offer. In other words, the manual business processes should offer us some clues how to better automate the integration.
There is a vast catalog of potential applications from which a plant may chose, each promising the solution to all of the sites problems. Indeed many of these do apply legitimate point solutions to particular problems. The difficulty lies in their deployment. No matter how hard a site attempts to specify the functional description of the application, the vendor will deliver what he has got, minimizing, hopefully, the customization. Yet, the application assumes a particular business process that might not match that currently used at the site. This application-specific business process is then superimposed on top of a complex business process architecture.
In fact, a site may well have 50 ~ 150 different applications deployed to support the business processes, each of which imposes its own business process, duplicates information, and has proprietary interfaces to other systems.
Some ERP applications have been sold as the ‘last application’ that you will need. In part they are, but there is still the need for other applications. Each application performs best in its area of competence. Outside of this competence area, it might fail to meet the minimum functionality that the existing business process can accept. Thus, inevitably there is a need for other applications but certainly not the 50~150 that a site may currently have, but maybe reduced 10 fold.
In other words, 15 800# gorillas are better than 150 80# chimpanzees.
Thus, the integration architecture reflects the ad hoc business process architecture. Ad hoc interfaces are developed to interface between each application system. The myth that “It’s easy to copy data between systems”2 is widely held. Yes, it is easy to move files of data, spreadsheets, or even messages containing data, but data is not what is required: information is required using the same terminology and model metaphor as the destination system. Thus, the interfaces are more than just a data transfer mechanism. They are systems that transform information from one form to another. These transformations involve temporal, material, and topological aggregation, and transformation of the terminology.
It is not surprising to find that the creation of these application interfaces often consumes more than 50% of the application deployment budget, which can be five times the cost of the application software! Consequently, a major proportion of the plants IT budget is spent maintaining these interfaces.
Copying data from one place to another is now relatively easy. Gone are the specialized transmission protocols, proprietary networks and other technologies that created the ‘islands of automation’. Similarly, the barriers presented by the different data formats are gradually disappearing. Binary file formats or non-existent APIs (application programming interface) are largely outdated, although there are still enough legacy systems to make this a challenge.
Thus, interface problems should be a thing of the past, but they are not.
Business units within an enterprise have, traditionally, been very self-contained. This phenomenon is not confined to the refining or process industry. Look at government (or better yet, don’t look there for an example to follow!). They are a classic example of one hand not knowing what the other is doing. In Britain once you could get a Government grant for reclaiming wetlands from the Department of Agriculture, and another grant for NOT reclaiming wetlands from the Department of the Environment!
Initially cooperation develops between departments out of sheer necessity. It usually takes the form of the interchange of COPIES of information (memos, declarations, reports, lists, etc in hard-copy or electronic format). Thus each business unit develops a vast filing system, 80% of which is data that has been copied from outside. Since it has been copied, the quality of the data contained therein deteriorates rapidly.
Applications model themselves after the business unit that they support. So guess what: they also copy the data from the applications with which they must interface. Not only is this a natural consequence of modeling the application after the behavior of the business unit, but it is codified in our development procedures: the first diagram of a functional specification is the context diagram showing how the data is going to be cloned from these other systems.
Thus, businesses have made the first step in creating, not islands of automation, but icebergs of information: each business unit/application has, on the surface some information (hardcopy/electronic) that it clearly owns, but has much more beneath the surface copied/stolen/borrowed from other business units/applications.
Herein lie the problem and the need for integration and cooperation.
There is another consequence of the evolution of isolated business units: they all start talking different languages! The same word can have a different meaning, or the meaning can be conveyed by different words or codes. A simple copy or reference to the information requires that it be accompanied by its complete meaning. For example, SRN could be either State Registered Nurse or Straight Run Naphtha. There is a difference. If you do not think this is important, then look at the Mars Explorer: information (thrust) was copied (by fax) without the complete meaning (units-of-measure). The consequence is that there is no longer a Mars Explorer!
If that is not enough problems, we finally have the problem of aggregation. Different business units view their business area with a different degree of summarization. Three dimensions to aggregation occur: temporal, material, and topological.
Temporal aggregation occurs when we take the fundamental time-series data and average, or totalize over a period. This period might be defined by business events such as the start and stop of a movement. Alternatively, the period might be defined by the calendar: an hour, a shift, a day, or a month.
Obviously, given only the average over a period it is impossible to determine what was the value at any particular time. This occurs when we are given the planned, say gasoline, production for a month. How do we decompose that ‘set-point’ into discrete blends? Like a stopped clock, the scheduled and actual gasoline production only equals the planned production infrequently.
The Temporal dimension can be described as follows:
Material aggregation occurs when we no longer wish to distinguish the different products. For example, the different gas oils from the different units might be reported as gas-oil production. Total gasoline in stock or shipped might be reported instead of the individual tank contents or shipments.
Obviously, given only the total gasoline stock, it is impossible to determine how much gasoline is in each tank. Yet, when decomposing a schedule into operating targets, we have to select specific materials to execute the blend.
The dimension of material aggregation can be described as follows:
Topological aggregation, the assignment of plant equipment into different physical groups, frequently differs between systems. For example:
The topological dimension might be described as follows:
3. Functional areas, as used in scheduling models.
A detailed process model views the plant in terms of equipment services whilst maintenance sees the physical equipment. Whilst even a detailed process model sees just a pump, a maintenance model might see both the in-service and spare pump locations, and would see at the location a specific pump, drive, etc.
Very often, the planning model does not match the physical plant. Simplifications are made in which entire units, such as treatment units, are simply omitted. Sometimes similar units, such as the crude units, are merged into a single logical unit. This contrasts with the material balance or reconciliation model that merges units if there are no measurements between them.
In general, the topological model between systems rarely agrees!
The differences between application systems might occur along one or more of these dimensions.
For example, unit yield accounting performs both temporal and material aggregation while unit material balancing only performs temporal aggregation. Conversely, oil accounting performs temporal, material, and topological aggregation.
It is important to recognize that aggregation is a one-way process. If I have information that is aggregated, I cannot de-aggregate that information. The aggregation process loses that information forever. This is the root of many of the integration problems that automation projects face. Paradoxically, human beings do a far better job.
Consider the following processes and the de-aggregations that are performed as the information is transformed from one ‘level’ to another. These comments are only generalizations as it is recognized that the actual aggregation varies from product to product and implementation to implementation. Additionally, the aggregation might not be uniformly applied throughout the same process. For example, the scheduling model might accurately model all processing units but simplify blending and products to ‘pools’.
Temporal
At a minimum, this will be a temporal de-aggregation of the plan and might involve some material and topological de-aggregation as well.
Material
The plan may well refer only to product categories, whereas the schedule needs to de-aggregate this into intermediates with, perhaps, differing specifications.
Topological
The plan might well be presented in terms of products, not actual streams from the unit. A classic example appears when a plan produces both Kerosene and Jet from the same unit during the same period. Consequently, the plan will not specify the individual tanks.
The schedule needs to be transformed before it is delivered as the operating instructions. This requires not only the addition of textual information, but also the addition of precise instructions.
Temporal
When the schedule is not presented based upon events, this will have to be de-aggregated into specific events with definitive start times and end times.
Material
Products will be specified by the qualities.
Topological
The schedule will be converted to specific actions, line-ups, and tanks (since the schedule might have referred to intermediate storage by pools of product).
When we represent the flow of information between applications, particularly the planning, scheduling, operating instructions, and operations cycle, we tend to use the control-feedback metaphor. Although this is hardly surprising considering, it is not an accurate representation of the business process architecture. For example, the ‘output’ plan from the LP is not usable as the ‘input’ to the scheduling application.
Another anomaly that occurs if we use this metaphor is that the business continues to operate if the ‘set-point’, the plan or schedule, is not produced. Where does the ‘set-point’ come from? In fact, we know that most of the business processes within an actual plant will continue to operate even if these links are broken. This suggests that the processes are largely autonomous; much like the way nature organizes its own control systems. These autonomous functions, working within constraints, have processes to check that they are meeting the overall goals. For example, conscious thought is not required to perform the action of walking. The brain presumably sets the overall direction, and then the sub-conscious and nervous system takes over working largely autonomously (remember the chicken without a head!).
So how does the manual business process solve this problem? Observation of this process has led us to a better understanding of:
• How the process should be automated.
• What, and where, aggregations should be used.
• The position and role of applications within the overall business process architecture.
• Gaps in the architecture that can be filled with some innovative applications.
Let us use the scheduling process as an example. The scheduling solution will be constrained by predefined material receipts and product liftings. These constraints will come from the upstream and downstream business processes. Note that these constraints should be specified with the same level of temporal, topological, and material aggregation.
From this, together with the unit performance model, the scheduling application or process will produce a schedule. The schedule will probably consist of a set of inventory projections and material movements between specified sources and destinations, each movement specified by its quantities and qualities, and either a calendar or an event period.
In principle, the schedule could now be transmitted to the next process, either operations or the conversions to operating instructions. In practice, another step is performed: the comparison with the plan to ensure that the schedule is meeting the plan’s overall objectives.
Since the schedule and the plan are using different levels of aggregation, they cannot be compared directly. Therefore, the first step is to aggregate the produced schedule to the same level of detail as provided by the plan. Manually this is performed by a spreadsheet. In a well-designed database, this can be achieved by querying the schedule and requesting that the database query perform the necessary aggregations. It might appear that this is an entirely new process: it is not. In fact, it is nothing more than unit yield performance monitoring.
Once the plan and schedule are comparable, any deviations can be used to adjust the schedule in such a way that the adjusted schedule better meets the plan: conventional negative feedback. This iterative process continues until the deviations are acceptably small. Then the schedule can be released to the next process.
What role does the actual performance play? Comparing instantaneous results with the schedule does not take into account process variations, but the actual performance report will not be using the same level of aggregation as the scheduling process. Therefore, the first step is to summarize the results as movement totals, average qualities, and total inventories. We are now in a position to compare the projected schedule with the actual results.
Although this comparison might be performed once a day, if they are performed more frequently we can immediately determine if one is not ‘conforming’ to the schedule. Any deviations can be used to adjust the scheduling model.
The same analysis can be repeated for many of the other business process cycles that exist in the plant. Within each application area, we see the same cycle of activities:
• Verification
Verify that the more detailed plan/instructions are meeting the intent of the plan/instructions received from ‘above’.
• Reporting
Aggregate the actual information to bring it to the same level of aggregation with other activities at the same ‘level’
• Comparison
Compare the target and observed information with the intent of improving the plan/instructions either immediately or in the long term.
Target Setting, the business process often found between the scheduling and operations process, sets specific targets recognizing other, often intangible, constraints coming from maintenance, equipment performance etc. These targets include specific equipment to be used, timings of movements, tank switches, etc. This process often performs further de-aggregations, especially when the scheduling application uses pools of material, and calendar-based temporal aggregation.
• Verification
This is often a ‘manu-matic’ processing involving adaptation of the previous days operating instructions to meet the intent of the new schedule.
• Reporting
This is nothing new: it is the daily performance report, but there is a strong case for making this report continuously available.
• Comparison
This is a relatively new process that we could call ‘Non-conformance monitoring’. This business process continuously tracks the performance against the targets. Instead of waiting until the next day, non-conformance monitoring can be automated so that the targets can be reset to reflect constraints that are more accurate or more accurate operating conditions. One site estimated that this application alone was worth $8~20MM/year ($0.11~ $0.28/bbl). Their estimates were justified when they broke 19 operating records in one month.
One of the key observations regarding how different applications function within the overall business process architecture is the importance of aggregation. The success of manual processes is because humans are very good at performing this aggregation. Skilled personnel within the plant have accumulated a detailed mental model of the plant that they use to perform this aggregation, spot anomalies, handle unforeseen events, etc. Unfortunately, as skilled personnel become difficult to retain or are lost through ‘right-sizing’, this knowledge is being lost. It is essential that these activities be automated so that the skill becomes an asset of the plant.
How can we put in place applications such that these aggregations can be automated? These aggregations are a process of transforming from one ‘level’ of detail to another. Each ‘level’ uses different degrees of temporal, material, and topological aggregation. To understand how to transform from one level to another, we need to understand the model metaphors that are typically used at different levels. What we mean by a model metaphor is the different ways in which the behavior of the plant is viewed.
The different levels of aggregation that are possible have led to several different plant model metaphors. These model metaphors are used in various applications such as real-time monitoring, oil accounting, scheduling, planning, etc.
The ‘Measurement Model’ is used extensively at the real-time level.
It has:
Individual measurements (quantities and qualities) at a point over time
But loses
Where the material comes from/goes
Category of the material
The measurement model is useful for unit monitoring and performance calculations.
The movement model is used to support the most scheduling, target setting applications, and yield accounting.
It has
Source and destination of a material movement
Identifies material by qualities
Total quantity and duration
But loses
Individual measurements
The material model is used at the planning and oil accounting level.
It has
Quantities and categories of materials
But loses
Precise source and destination
Skilled personnel, using their knowledge of the plant, can manually transform between one model-metaphor and another.
For example, a scheduled material movement may be specified by its duration, quantity and average qualities. Using only a real-time system, scheduled versus actual movement can be monitored but it requires a lot of plant knowledge: what real-time ‘tag’ should be used for each of the movement quantities and qualities; what was the actual start time of the movement so that the movement average can be determined, etc. When data has to be manipulated manually, it is surprising how much additional information is supplied by the user of that information to make it usable in an application.
For example, when a tag is retrieved from an RTDB the user has to deduce from the tag-name some or all of the following:
All of this logic has to be written into the applications or reports that are processing this data.
Successful integration of all plant applications will require that this be fully automated, but the additional knowledge that a skilled person uses does not normally reside in any of the existing applications. Therefore, we need an additional tool to assist in this process, and that tool is the database model but there is some confusion over the role that a data-model of a plant performs.
At one extreme, some regard a relational database as a ‘dumping’ ground for any data: if more data has to be stored then simply create another table to store that data. The database will certainly not lose this data, but users will not be able to create queries or reports on a database designed like this.
The cornerstone of a good plant data-model is the capture of the refinery configuration.
Therefore, there are database tables that:
What does this mean?
One advantage is that the production of a material balance for any unit in the plant uses exactly the same database report. The database knows what streams are flowing into or out of the unit from the information contained in the database. From this the database can deduce the mass- or volume-flows or whatever characteristic is required.
Another advantage is that if any changes occur to the refinery flow sheet, no reports whatsoever have to be re-coded. The changes have to be made once to the database and thereafter all reports will automatically reflect that change.
Does this mean that we do not need the 800# gorilla-solution? In part yes, but there are some constraints on how each of these individual applications is deployed to make integration possible:
We need to position each application clearly within the explicitly defined business process architecture.
When selecting and configuring each application, we must define the degree of the temporal, material, and topological aggregation that will be used so that this application is compatible with its peers in the same ‘level’ of integration. For example:
There is no point performing data reconciliation on a Coker working on an 18 hour coking cycle when we want the results for yield performance analysis.
The most difficult problem to solve is inconsistently applied degrees of aggregation. For example:
If the planning model merges all crude units into one, then it will forever be impossible to compare individual crude unit schedules with the plan.
Once we have met the constraints on the applications, we can deploy a plant data model, in the form of a database, that can automate the flow of information from one application to another performing the transformations and aggregations that are required to ‘match’ the temporal, topological, and material aggregations in the different applications.
The vast number of applications in a process plant has led to overly complex integration problems. These problems are not solely due to technology issues of data interchange formats, or islands of automation, but due to the incompatibility of the models within each application system. However, this does not mean that we have to have a ‘single plant model’ but applications with harmonious models: applications that understand their position within the overall business process architecture and agree with their peers on the degree of temporal, material, and topological aggregation.
[1] Planning, Scheduling, and Control Systems: Why can’t they work together
D.E. Shobrys & D. C. White; NPRA Annual Meeting, March 26-28th, 2000
[2] The Values and Myths of Plant-to-ERP Integration
B. Harkins; NPRA Annual Meeting, March 26-28th, 2000