Cost-Based Optimization of Integration Flows - Datenbanken ...

2.3 Integration Flow Meta Model
access to heterogeneous systems and applications. There, the proprietary external messages
and data representations are transformed into the described internal message meta
model. In detail, the group of interaction-oriented operators include the operators shown
in Table 2.1. In contrast to this, the data-flow- and control-flow-oriented operators are
used as local processing steps within the integration platform in the sense that they do
not perform any interactions with external systems. Both groups of operators are shown
in Table 2.2 and Table 2.3, respectively.
The instance-based plan execution has several implications for all operators. First, the
operators use materialized intermediate results in the sense of local message variables.
Second, the data flow is implicitly given by those input and output variables.
Moreover, we distinguish between external integration flow descriptions (e.g., BPEL),
internal plans (logical representation) and internal compiled plans (physical representation).
Here, the term plan is a shorthand for internal plans. In Section 2.4, we present
several use cases that are used as example plans throughout the whole thesis.
2.3.2 Transactional Properties
There are several transactional properties of integration flows that must be guaranteed
under all circumstances. In this section, we discuss different problems that can occur while
executing an integration flow as well as how they are typically addressed in integration
platforms and what we can imply for the cost-based optimization of integration flows. For
more details, see our analysis of problem categories [BHLW08a].
The most important risk of executing integration flows is the problem of message lost
when using a simple send and forget execution principle.
Problem 2.1 (Message Lost). Assume that the stream of incoming messages M is collected
using transient (in-memory) inbound message queues. If a server breakdown of the
integration platform has occurred, all messages not sent to the target systems will be lost.
This is a problem because often the messages cannot be restored by the source systems.
As shown in Section 2.1.2, this problem is typically addressed by persistently storing all
incoming messages at the inbound server side of the integration platform. The resulting
implication is that all integration platforms follow a store and forward principle in order to
guarantee that each received message will be successfully delivered to the external systems.
Thus, if a failure occurs, the stored messages are used to resume the state of execution.
A failure or server breakdown can occur at an arbitrary point in time. Thus, there might
be operators and interactions with external systems that have already been successfully
finished, while other operators have not. When re-executing the complete integration flow,
the following problem arises.
Problem 2.2 (Message Double Processing). Assume that a server breakdown during execution
of plan instance p 1 has occurred. Recall that typically, each interaction with an
external system is an atomic transaction. Thus, there might be successfully finished operators
and currently unfinished operators. Furthermore, if the external system does not
support transactional behavior, there might be partially successful interactions with external
systems. If we re-execute the plan instance p 1 with p ′ 1 , we might send the same message
twice to an external system.
In order to tackle this problem, specific recovery models for integration flows are used,
where we distinguish between two types. First, there is the compensation-based transac-
25