Cost-Based Optimization of Integration Flows - Datenbanken ...

3 Fundamentals of Optimizing Integration Flows
Assign (o1)
[out: msg1]
[5ms]
[max(60ms+3ms,
Fork (o-1)
26ms+6ms)]
[60ms,
5000]
[26ms,
1000]
Invoke (o2)
Invoke (o3)
[service s4, in: msg1, out: msg2] [service s5, in: msg1, out: msg3]
Join (o4)
[in: msg2,msg3, out: msg4]
Groupby (o5)
[in: msg4, out: msg5]
[50ms,
5000]
[35ms,
1000]
[max(60ms+W(o5)+3ms,
26ms+6ms)]
[60ms,
5000]
Invoke (o2)
[service s4, in: msg1, out: msg2]
[ W(o5),
|dsout(o5)| ]
Groupby (o5)
[in: msg2, out: msg4]
Assign (o1)
[out: msg1]
Join (o4)
[in: msg4,msg3, out: msg5]
[5ms]
Fork (o-1)
[26ms,
1000]
Invoke (o3)
[service s5, in: msg1, out: msg3]
[ W(o4),
1000 ]
Assign (o6)
[in: msg5, out: msg1]
[5ms]
Assign (o6)
[in: msg5, out: msg1]
[5ms]
Invoke (o7)
[service s5, in: msg1]
[40ms]
Invoke (o7)
[service s5, in: msg1]
[40ms]
(a) Plan P 3
(b) Plan P ′ 3
Figure 3.4: Plan Cost Estimation Example
Furthermore, we estimate the missing execution times using the monitored statistics of P 3
and the defined abstract costs of P 3 and P 3 ′ as follows:
Ŵ (o ′ 5) = |ds in(o ′ 5 )| + |ds in(o ′ 5 )| · |dsouto′ 5 |
2
|ds in (o 5 )| + |ds in (o 5 )| · |dsout(o 5)|
2
· W (o 5 ) = 2,505,000 · 35 ms = 35 ms
2,505,000
Ŵ (o ′ 4) = |ds in1(o ′ 4 )| + |ds in1(o ′ 4 )| · |ds in2(o ′ 4 )|
|ds in1 (o 4 )| + |ds in1 (o 4 )| · |ds in2 (o 4 )| · W (o 4) = 1,001,000 · 50 ms = 10 ms.
5,005,000
Finally, we can use the computed cost estimates, aggregate the plan costs, and compare
these costs as follows:
W (P 3 ) = 5 ms + max(60 ms + 3 ms, 26 ms + 2 · 3 ms) + 50 ms + 35 ms + 5 ms + 40 ms
= 198 ms
Ŵ (P ′ 3) = 5 ms + max(60 ms + 35 ms + 3 ms, 26 ms + 2 · 3 ms) + 10 ms + 5 ms + 40 ms
= 158 ms.
In our example, we would choose P 3 ′ as execution plan because, it is optimal, on average,
under the assumption of precise monitored statistics. Note that although o 2 and o 7 are defined
with equal abstract costs (Invoke), we adapt to the concrete workload characteristics
by weighting those costs with monitored execution times. In addition, the double metric
cost model enables us to use one single metric (the execution time) for data-flow-oriented
operators (e.g., Groupby), interaction-oriented operators (e.g., Invoke), and control-floworiented
operators (e.g., Fork).
To summarize, we proposed the first complete cost model for integration flows. This
double-metric cost model is self-adjusting because we weight the abstract costs with monitored
execution statistics. For this reason, over time, the estimates converge to the real
costs of the concrete application environment and hence, this cost model enables the adaptation
to changing workload characteristics. Further, the two metrics enable to integrate
the interaction-, control-flow-, and data-flow-oriented operators into a unique cost model
and thus, enable the comparison of plans with control-flow semantics.
44