download - Institut für Informationswirtschaft und -management (IISM)
download - Institut für Informationswirtschaft und -management (IISM)
download - Institut für Informationswirtschaft und -management (IISM)
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
Economic Models and Algorithms for<br />
Grid Systems<br />
KUMULATIVE HABILITATIONSSCHRIFT<br />
an der Fakultät für<br />
Wirtschaftswissenschaften<br />
der Universität Karlsruhe (TH)<br />
von<br />
Dr. Dirk Georg Neumann, M.A., Univ. Wisc.<br />
12 October 2007, Karlsruhe
Table of Contents<br />
Table of Contents ___________________________________________________________ i<br />
PART A: OVERVIEW ARTICLE<br />
TOWARDS AN OPEN GRID MARKET _____________________________________________ 1<br />
1 Introduction ________________________________________________________ 2<br />
2 Definitions _________________________________________________________ 3<br />
2.1 Cluster ___________________________________________________________ 3<br />
2.2 Distributed Computing ______________________________________________ 4<br />
2.3 Grid Computing ___________________________________________________ 5<br />
2.4 Utility computing ___________________________________________________ 5<br />
3 Markets as the Business Model for Grid _________________________________ 6<br />
3.1 Pricing strategies ___________________________________________________ 6<br />
3.2 The need for Self-organization ________________________________________ 9<br />
4 Related Work ______________________________________________________ 10<br />
4.1 Market-based Grid systems _________________________________________ 10<br />
4.2 Markets are dead, long live markets __________________________________ 13<br />
5 An Architecture for Open Grid Markets ________________________________ 14<br />
5.1 Architecture Rationale _____________________________________________ 14<br />
5.2 Open Grid Market Architecture _____________________________________ 15<br />
5.2.1 Layer 1 – Core Market Services _____________________________________ 16<br />
5.2.2 Layer 2: Open Grid Market _________________________________________ 16<br />
5.2.3 Layer 3: Intelligent Tools __________________________________________ 17<br />
5.2.4 Layer 4: Grid Application __________________________________________ 18<br />
6 Organization of the book _____________________________________________ 18<br />
6.1 Market Mechanisms for Grid Systems ________________________________ 18<br />
6.1.1 Bridging the Adoption Gap – Developing a Roadmap for Trading in Grids ___ 18<br />
6.1.2 A Truthful Heuristic for Efficient Scheduling in Network-centric Grid OS ____ 19<br />
6.1.3 Economically Enhanced MOSIX for Market-based Scheduling in Grid OS ___ 20<br />
6.1.4 GreedEx – A Scalable Clearing Mechanism for Utility Computing __________ 20<br />
6.1.5 Market-Based Pricing in Grids: On Strategic Manipulation and Computational<br />
Cost ___________________________________________________________ 21<br />
6.1.6 Trading grid services – a multi-attribute combinatorial approach ___________ 21<br />
i
6.1.7 A Discriminatory Pay-as-Bid Mechanism for Efficient Scheduling in the Sun N1<br />
Grid Engine _____________________________________________________ 22<br />
6.1.8 Decentralized Online Resource Allocation for Dynamic Web Service<br />
Applications ____________________________________________________ 22<br />
6.2 Economic Component Design _______________________________________ 22<br />
6.2.1 Self-Organizing ICT Resource Management – Policy-based Automated Bidding _<br />
_______________________________________________________________ 22<br />
6.2.2 A Decentralized Online Grid Market – Best Myopic vs. Rational Response ___ 23<br />
6.2.3 Economically Enhanced Resource Management for Internet Service Utilities _ 23<br />
6.2.4 Situated Decision Support for Managing Service Level Agreement Negotiations _<br />
_______________________________________________________________ 24<br />
6.2.5 Using k-pricing for Penalty Calculation in Grid Market ___________________ 24<br />
6.2.6 Rightsizing of Incentives for collaborative e-Science Grid Applications with TES<br />
_______________________________________________________________ 24<br />
6.3 Design Issues and Extensions ________________________________________ 25<br />
6.3.1 Distributed Ascending Proxy Auction – A Cryptographic Approach _________ 25<br />
6.3.2 Technology Assessment and Comparison: The Case of Auction and E-negotiation<br />
Systems ________________________________________________________ 25<br />
6.3.3 Comparing Ingress and Egress Detection to Secure – Inter-domain Routing: An<br />
Experimental Analysis ____________________________________________ 26<br />
6.3.4 A Market Mechanism for Energy Allocation in Micro-CHP Grids __________ 26<br />
7 Conclusion and Outlook on Future Research ____________________________ 27<br />
7.1 Concluding remarks _______________________________________________ 27<br />
7.2 Outlook __________________________________________________________ 28<br />
Bibliography ______________________________________________________________ 30<br />
Part B: Articles<br />
Article: “Bridging the Adoption Gap – Developing a Roadmap for Trading in Grids” 34<br />
Article: “A Truthful Heuristic for Efficient Scheduling in Network-centric Grid OS” _ 54<br />
Article:“Economically Enhanced MOSIX for Market-based Scheduling in Grid OS” _ 66<br />
Article:“GreedEx – A Scalable Clearing Mechanism for Utility Computing ________ 74<br />
Article:“ Market-Based Pricing in Grids: On Strategic Manipulation and<br />
Computational Cost” _______________________________________________ 95<br />
Article:“Trading grid services – a multi-attribute combinatorial approach ________ 125<br />
Article:“A Discriminatory Pay-as-Bid Mechanism for Efficient Scheduling in the Sun<br />
N1 Grid Engine” __________________________________________________ 154<br />
Article:“Decentralized Online Resource Allocation for Dynamic Web Service<br />
Applications” ____________________________________________________ 164<br />
ii
Article:“Self-Organizing ICT Resource Management – Policy-based Automated<br />
Bidding” ________________________________________________________ 172<br />
Article:“A Decentralized Online Grid Market – Best Myopic vs. Rational Response” 180<br />
Article:“Economically Enhanced Resource Management for Internet Service Utilities” _<br />
_________________________________________________________________ 91<br />
Article:“Situated Decision Support for Managing Service Level Agreement<br />
Negotiations” _____________________________________________________ 205<br />
Article:“Using k-pricing for Penalty Calculation in Grid Market” _______________ 214<br />
Article:“Rightsizing of Incentives for collaborative e-Science Grid Applications with<br />
TES” ___________________________________________________________ 224<br />
Article:“Distributed Ascending Proxy Auction – A Cryptographic Approach” _____ 235<br />
Article:“Technology Assessment and Comparison: The Case of Auction and E-<br />
negotiation Systems” ______________________________________________ 244<br />
Article:“ Comparing Ingress and Egress Detection to Secure – Inter-domain Routing:<br />
An Experimental Analysis _________________________________________ 284<br />
Article:“A Market Mechanism for Energy Allocation in Micro-CHP Grids ________ 307<br />
Lebenslauf ______________________________________________________________ 317<br />
Schriftenverzeichnis ______________________________________________________ 327<br />
iii
Part A<br />
Overview Article<br />
Towards an Open Grid Market<br />
1
TOWARDS AN OPEN GRID MARKET<br />
Dirk Neumann<br />
<strong>Institut</strong>e of Information Systems and<br />
Management (<strong>IISM</strong>)<br />
Universität Karlsruhe (TH)<br />
neumann@iism.uni-karlsruhe.de<br />
1 Introduction<br />
During the last years, the costs for ICT infrastructures have enormously exploded as a result<br />
of „one-application-one-platform“ style of deployment. This has left most of the ICT infrastructures<br />
with extremely low system utilization and wasted resources (Carr 2005). Examples<br />
can be drawn from virtually all sources: One recent study of six corporate data centers reported<br />
that the bulk of their 1000 servers just utilized 10% to 35% of their available<br />
processing power (Andrzejak, Arlitt et al. 2002). IBM estimated the average capacity utilization<br />
rates of desktop computers to just 5% (Berstis 2002). The Gartner Group indicates that<br />
between 50% and 60% of a typical company’s data storage capacity is wasted (Gomolski<br />
2003). Overcapacity can not only be observed with respect to hardware, but also to software<br />
applications. In essence, highly-scalable applications can serve additional users at almost no<br />
incremental costs – hence red<strong>und</strong>ant installations of the same application create unnecessary<br />
costs (Carr 2003; Carr 2005).<br />
In recent times, ICT is <strong>und</strong>ergoing an inevitable shift from being an asset that companies<br />
posses (e.g. computers, software) to being a service that companies purchase from designated<br />
utility providers. This shift will take years to fully unfold, but the technical building blocks<br />
have already begun to take shape. On the coat tail of this shift, the business model of utility<br />
computing or equivalently e-Business on-demand is more and more emerging.<br />
Utility computing denotes the service provisioning model, in which a service provider makes<br />
computing resources and infrastructure <strong>management</strong> available to the customer as needed, and<br />
charges them based on usage. This new business model relying on service-oriented architectures<br />
contributes to driving down costs – resulting from low system utilization and waste –<br />
and complexity by plug & play service provisioning in the ICT infrastructure.<br />
Typically, two general pricing models for utility computing are identified. The first pricing<br />
model is the subscription model. Accordingly, users are charged on a periodic basis to subscribe<br />
to a service. The second pricing model is the metered model, where users are charged<br />
on the basis of actual usage. While the first model gives rise to a waste of resources, since<br />
charging is independent of usage, the second, the metered model, is ultimately deemed the<br />
promising to remove the waste of resources (Rappa 2004).<br />
From the user’s point of view, the metering of usage can be compelling, as “one pays for<br />
what one uses”. For the service provider the metered model is appealing, as it offers an opportunity<br />
to sell idle or unused computer capacity. The metered model, however, is not without<br />
problems per-se: It has to be determined, which price is charged from a customer, when<br />
the meter is turned on (Rappa 2004). The price is essential, as it determines the incentive for<br />
resource owners to provide them as well as for consumers to demand them. If the price is too<br />
high, not all the offered resources are demanded contributing to idle resources. If the converse<br />
is true, not all tasks can be realized as the resource provision is too low, which is equally bad.<br />
2
In general, market-based approaches are considered to work well for price determination. By<br />
assigning a value (also called utility) to their service requests, users can reveal their relative<br />
urgency or costs to the service (Buyya, Abramson et al. 2004; Irwin, Grit et al. 2004). Despite<br />
their theoretically excellent properties only few market-based approaches have become operational<br />
systems, let alone commercial. The reason for this lack of market-based operational<br />
systems stems from the fact that many questions arise particularly concerning the applicability<br />
of markets, and their relevance to system design.<br />
This book strives for bridging this gap by providing economic models and algorithms for Grid<br />
systems in general, and for utility computing in particular. It is argued that those economic<br />
models and algorithms supporting technical Grid infrastructures may result in a widespread<br />
usage of utility computing. The results reported in this book have been generated within the<br />
EU f<strong>und</strong>ed project “Self-organizing ICT resource <strong>management</strong>” (SORMA) 1 . While the<br />
project is mainly focused on the technical implementation aspects, this book is devoted to the<br />
<strong>und</strong>erlying economic principles.<br />
This introductory chapter primarily aims at providing a common backgro<strong>und</strong> for <strong>und</strong>erstanding<br />
the selected papers of this book. Moreover, the paper promotes the idea of an open Grid<br />
market, which defines a blueprint for how to construct Grid markets. This blueprint follows a<br />
component-based architecture with well-defined interfaces between the different components.<br />
Most of the components embody economic models and algorithms. This paper introduces the<br />
architecture of an Open Grid Market and describes how the following papers fit into this architecture.<br />
The remainder of this chapter is structured as follows. Firstly, important definitions are introduced<br />
that are commonly used throughout the book (2.1). Secondly, it is motivated why markets<br />
are deemed promising for Grid computing (2.2). Thirdly, related approaches are described;<br />
it is analyzed why those approaches never made it into practice (2.3). Fourthly, the<br />
architecture of an Open Grid Market is proposed, which improves over related approaches<br />
(2.4). Lastly, the structure of the book is illustrated by referring to the overall Open Grid Market<br />
Architecture (2.5).<br />
2 Definitions<br />
The terms cluster, distributed computing, Grid computing and utility computing are often used<br />
as synonyms. Although these terms denote related terms, they impose different requirements<br />
upon both the technical infrastructure as well as on the <strong>und</strong>erlying economic models. In the<br />
following we will define those four concepts, point out their differences and give examples.<br />
2.1 Cluster<br />
A cluster denotes a group of tightly coupled resources that work together on a common problem.<br />
The resources(e.g. computers consisting of memory and computation power) are homogeneous,<br />
which are typically connected via local area networks (Buyya 1999). As all parts of<br />
a cluster are working on a common problem they can be viewed as a single entity. Clusters<br />
can be distinguished into three types:<br />
• Failover clusters<br />
Failover clusters aim at improving the availability of services that the cluster performs.<br />
This implies that they maintain red<strong>und</strong>ant nodes, which can be used once components<br />
of the cluster fail to provide the service (Marcus and Stern 2000). Failover clusters are<br />
frequently used for important databases, business applications, and e-commerce websites.<br />
1 (http://www.sorma-project.eu/)<br />
3
Examples of failover cluster solutions are Linux-HA (High-Availability Linux), Sun-<br />
Cluster, or Microsoft Cluster Server.<br />
• Load-balancing clusters<br />
Load-balancing clusters distribute the workload to a set of computers or servers to<br />
achieve a balanced workload over all nodes of the cluster (Kopparapu 2002). Albeit<br />
those clusters are specialized in load balancing, in many cases they embed failover<br />
properties as well; those clusters are dubbed server farm. Professional load balancers<br />
are the Sun Grid Engine N1, Maui and MOSIX (Barak, Shiloh et al. 2005) and are typically<br />
used as back-ends for high traffic websites such as Wikipedia, Online-shops or<br />
marketplaces.<br />
• High-performance computing clusters<br />
High-performance computing clusters are those clusters that aim at increasing the performance<br />
by exploiting parallelism achieved through splitting a computationally demanding<br />
task into several subtasks. Examples for state-of-the-art high-performance<br />
computing solutions are so-called Beowulf clusters (Bell and Gray 2002). Almost all<br />
supercomputers are clusters. For instance the largest super computer of the world, the<br />
BlueGene/L at Lawrence Livermore National Laboratory, California is devoted to high<br />
performance computing. It consists of 131072 processors attaining performance of<br />
more than 367 TFLOPS 2 at its peak. The investment costs for BlueGene/L were quite<br />
enormous exceeding $ 290 million.<br />
2.2 Distributed Computing<br />
Another computing model is referred to as distributed computing. In essence, it denotes a method<br />
of computer processing, where parts of the program are parallelized on different machines.<br />
Different to clusters, the involved resources are typically heterogeneous. In addition<br />
the resources are only loosely coupled for example over the internet. Distributed computing is<br />
characterized by the fact that there exists a central <strong>management</strong> component, which divides the<br />
program into smaller pieces and distributes those to the connected machines. The central<br />
<strong>management</strong> component thereby considers the heterogeneous resources when processing the<br />
distribution job.<br />
The most prominent example of distributed computing is SETI@home developed by the University<br />
of California at Berkeley. It is essentially a model of voluntary computing. End users<br />
can donate their idle computation time of their home PC to the search for extraterrestrial life.<br />
Subsequently, the results of the many PCs are automatically reported back and reassembled<br />
into a larger solution (Werthimer, Cobb et al. 2001). SETI currently combines more than<br />
1,594,396 active users from more than 208 different countries. The impact SETI has is quite<br />
enormous totaling average performance of 261 TFLOPS and up to 500 TFLOPS in its peak<br />
exceeding the performance of the largest supercomputer in the world. SETI has become the<br />
pioneer in voluntary computing – the BOINC framework is an open source software platform<br />
that is used for other distributed computing projects such as Folding@home and Cancer Research<br />
Project<br />
The advantages of distributed computing are impressive, as processor performances of supercomputers<br />
can be reached free of charge, just by utilizing otherwise idle resources. The potential<br />
of voluntary computing is huge, taking into consideration that home PCs are only utilized<br />
by 5 % of the time. Having in mind that investment costs accounts only for 20 % of the total<br />
cost of ownership leaving 80 % for administration, the computation achieved by the<br />
2 TFLOPS stands for Terra FLOPS, where FLOPS (floating point operations per second) denote a performance<br />
metric for processors.<br />
4
SETI@project would incur total cost of ownership of more than $ 1,450 million, which pertains<br />
to the BlueGene/L supercomputer.<br />
2.3 Grid Computing<br />
The term Grid computing was initially introduced by Foster and Kesselman in 1998. Accordingly<br />
a grid denotes “a hardware and software infrastructure that provides dependable,<br />
consistent, pervasive, and inexpensive access to high-end computational capabilities” (Foster<br />
and Kesselman 1998). Later on Foster refined this definition as “direct access to computers,<br />
software, data, and other resources, as is required by a range of collaborative problemsolving<br />
and resource-brokering strategies emerging in industry, science, and engineering”<br />
(Foster and Kesselman 1998).<br />
Obviously, the concept of Grid computing is very related to distributed computing. Both<br />
computing models involve heterogeneous resources that are loosely coupled. Typically organizational<br />
bo<strong>und</strong>aries are crossed. Different to distributed computing, Grid computing relies<br />
on distributed <strong>management</strong>. A key advantage of Grid systems is their ability to pool together<br />
resources, including both computational resources as well as hard- and software, that can then<br />
be shared among Grid users (Alkadi and Alkadi 2006; Knight 2006). This sharing has major<br />
ramifications for the efficiency of resources, as it<br />
• allows increased throughput because of statistical multiplexing and the fact that users<br />
have a different utilization pattern.<br />
• lowers delay because applications can store data close to potential users<br />
• offers greater reliability because of red<strong>und</strong>ancy in hosts and network connections<br />
All of this results in lower costs than comparable private system (Lai, Huberman et al. 2004).<br />
An example for the potentials of Grid computing offering is Novartis. For their research and<br />
development activities in creating new drugs, they connected their PCs within all branches of<br />
the company. Over night up to 2700 computers were used as number crunching machines.<br />
According to their press releases, the Grid helped reducing projects scheduled on a single supercomputer<br />
for six years to a total run time of 12 hours. Including the productivity gains Novartis<br />
claimed Grid computing to be responsible for savings up to $ 200 million over three<br />
years.<br />
2.4 Utility computing<br />
Utility computing 3 differs from the previous three concepts, as it does not denote an additional<br />
computing model, but a business model on top of Grid computing. More precisely, utility<br />
computing refers to offering a b<strong>und</strong>le of resources (e.g. computation, and storage) as a metered<br />
service. Comparable with a utility (such as electricity, water or gas) the user is unaware<br />
about where the resources are coming from. Hence, utility computing can be <strong>und</strong>erstood as<br />
the on-demand delivery of infrastructure, applications, and business processes in a securityrich,<br />
shared, scalable, and standards-based computer environment over the Internet for a fee.<br />
Customers will tap into IT resources and pay for them (Rappa 2004)]. Utility computing provides<br />
companies with the advantage of low investment costs to acquire hardware, as resources<br />
can be rented. As the terms utility computing and Grid computing is closely related, we will<br />
use them as synonyms throughout the remainder of the book.<br />
The most prominent example of utility computing is Sun’s platform network.com<br />
(http://www.network.com/). Network.com makes use of Grid computing technologies and<br />
offers computational resources for a fix price of $1 per CPU hour. Although utility computing<br />
has the potential to mark “the end of corporate computing” as Nicholas Carr terms it (Carr<br />
3 Synonyms are cloud computing or on-demand computing.<br />
5
2005), neither Sun’s network.com nor any other utility computing initiative has succeed to<br />
exploit the potential. Currently, Sun has had only one single customer, Virtual Computing<br />
Company (VCC) that bought over 1 million CPU hours from Sun. It is a well known fact that<br />
VCC paid a price way less than $1 per CPU hour. Rumors are aro<strong>und</strong> that Sun has bought<br />
VCC for marketing purposes. These rumors are to some extent confirmed by the fact that<br />
VCC also heavily invested in the build up of their own infrastructure, which is said to exceed<br />
the 1 million CPU hours by far.<br />
The reasons why utility computing has not yet taken off are manifold. Legal problems are<br />
among the most severe. For instance network.com conducts all computations within the USA.<br />
Requests from other countries could not be accepted due to the absence of legally binding<br />
contracts. Sun has now remedied this shortcoming, allowing users from over 25 countries to<br />
use network.com on-demand. Other shortcomings address the fact that $1 per CPU hour is too<br />
expensive. $1 was merely set by Sun as a marketing message that can be easily communicated.<br />
According to Sun officials, the margin for CPU hours is not yet skimmed off. Apparently<br />
$1 per CPU hour establishes the break-even point for Sun if their infrastructures are utilized<br />
by 30%. Amazon offers their idle CPU hours for 10 cents per CPU hour (Butler 2006).<br />
In their fee structure they also include a fraction for bandwidth usage, which adds to the 10<br />
cents.<br />
3 Markets as the Business Model for Grid<br />
The need for computational resources from industry – especially in the automotive or pharmaceutical<br />
area – is tremendous, but current providers are struggling to accommodate this<br />
need. It seems that the fix prices are still too high. Throughout this book it is argued that viable<br />
business models are missing for Grid computing. This hypothesis is in line with the<br />
“Forrester Research of Grid Computing” The451Group (Fellows, Wallage et al. 2007). Markets<br />
are known to work effectively in allocating resources (who gets which resources and<br />
when?), especially in cases where the resources are <strong>und</strong>er decentralized control, users and<br />
providers are selfish, and where demand and supply are fluctuating. This ability of markets is<br />
gained by establishing dynamic prices that reflect demand and supply situations. Hence markets<br />
seem to be the adequate model to determine accurate prices (which are unknown right<br />
now) for the users and providers to be acceptable. In addition, price signals help to identify<br />
priorities in the requests (Neumann, Holtmann et al. 2006).<br />
3.1 Pricing strategies<br />
In the following, it is explored what kind of pricing strategies are deemed promising for a use<br />
in Grid computing. As in any distributed system, a key problem to sharing resources is find a<br />
suitable allocation (i.e. who gets what?). Resource allocation is not a problem, if sufficient<br />
resources are available. Once, demand exceeds supply this becomes more problematic, as a<br />
scheduling unit in the Grid must decide, which demand will be supplied with the given resource<br />
endowment. Nowadays, resource allocation increasingly becomes an issue, because<br />
resource demands of data mining applications, scientific computing, rendering programs, and<br />
Internet services have kept pace with hardware improvements (Cherkasova, Gupta et al.<br />
2007). This is expected to further increase.<br />
Traditionally, Grid systems address the resource allocation problem with naïve heuristics<br />
(e.g., first-in first out) or with more complex idiosyncratic cost functions (e.g. throughput<br />
maximization) (Buyya, Stockinger et al. 2001; Buyya, Abramson et al. 2004).<br />
Today's schedulers have recognized the need of expressing values by providing a combination<br />
of approaches including user priority, weighted proportional sharing, and service level agreements<br />
that set upper and lower bo<strong>und</strong>s on the resources available to each user or group (Irwin,<br />
Grit et al. 2004; LSF 2005; Sun 2005).<br />
6
One example will show why value-oriented algorithms are needed. Suppose SME Integrated<br />
Systems needs to submit the accounting statement in time in order to avoid a huge fine. Nonetheless,<br />
Integrated Systems resource request cannot be computed right away, as the resources<br />
are blocked by a big rendering job by corporation MovieRender. MovieRender needs the results<br />
of the rendering in two weeks and is indifferent whether this rendering job is computed<br />
now or the next week. The resource allocation should (re)allocate resources to Integrated Systems<br />
which needs them urgently reducing the allocation to MovieRender, as long as the delay<br />
does not change MovieRender value (i.e. the job is conducted within the given timeframe).<br />
Proportional Share is an example of such a scheduling mechanism. In essence, Proportional<br />
Share attaches to any user a weight. The resources allocated to the users depend on their<br />
weighted shares. For example, if there are two users Integrated Systems and MovieRender,<br />
where Integrated Systems has a weight of 1 and MovieRender of 2, Integrated Systems is allocated<br />
1/3 of the resources, leaving 2/3 to MovieRender. This is effective, if MovieRender always<br />
performs more valuable computations than Integrated Systems. In our example, Integrated<br />
Systems clearly has suddenly a higher valuation and MovieRender has a low valuation.<br />
This is rather common, as the arrival process of important work is highly bursty (similar to a<br />
web-, email-, file server) (Lai 2005). As a consequence, the situation would be efficient, if<br />
MovieRender announces that his job is not time-critical and reduces its weight, while Integrated<br />
Systems increases her weight correspondingly. But, if MovieRender’s rendering creates<br />
a small value, then there is no incentive for MovieRender to lower his value, since it would<br />
reduce its own value without compensation.<br />
In other words, maximizing the utility (i.e. the sum of valuations) is possible only if the scheduler<br />
knows the attached valuations, or as in the case of Proportional Share, the exact relative<br />
weights. However, as in the example MovieRender has no incentive to reduce its weight,<br />
when it looses value by not getting its computation done. This is a common phenomenon in<br />
Economics, known as the “Tragedy of the Commons”, which states that the utility maximizing<br />
behavior of the clients results in an overall inefficient solution (Hardin 1968).<br />
Value-oriented approaches are, hence, not sufficient per se, to achieve an efficient solution.<br />
Only if all participants are willing to report their priorities and values honestly, these algorithms<br />
such as Proportional Share will work well. As the example shows, in most cases there<br />
will be no incentive for the participants, who act strategically in order to maximize their utility.<br />
One way to prevent strategic misrepresentation of the weights would be to install a system<br />
administrator that monitors the system and dynamically changes the weights of the users to<br />
maintain high efficiency. Even inside organizational units this is expensive and extremely<br />
error-prone. Once organizational bo<strong>und</strong>aries are crossed, such a system-wide monitoring is<br />
politically almost impossible to achieve. This is where markets enter the discussion.<br />
Markets have the ability to set the right incentives for users to reveal their true valuation as<br />
well as for resource owners to provide those resources that are scarcest in the Grid. By the<br />
dynamic adjustment of the prices, markets can react to the ever changing conditions of resource<br />
demand supply. This flexibility is one major advantage against long-term heuristics<br />
like Proportional Share.<br />
Proportional Share typically determines the proportional value for a longer period, say a week<br />
or a month. Obviously, Proportional Share does neither set the incentive to shift usage from a<br />
high demand resource to a low demand resource, nor does it reward shifting usage from a<br />
high demand period into a low demand period. For instance, the system as a whole may have<br />
high demand for a certain number of CPUs and low demand for memory. Since those resources<br />
are typically allocated separately, there will be no incentive in the system to substitute<br />
CPUs by memory (Sullivan and Seltzer 2000; Lai 2005).<br />
7
With the introduction of prices, incentives will be given to the users to substitute the scarce<br />
resource (e.g. number of CPUs) with less scarce resource (memory). For instance, a fixed<br />
pricing scheme which requests 10 € for one CPU and 1 € per Gigabyte of memory, sets incentives<br />
to reduce the number of used CPUs in favor of the cheaper memory. A fixed pricing<br />
scheme, however, is not enough to achieve an efficient allocation, as the following consideration<br />
will reveal:<br />
Suppose the fixed price is as in Figure 1 (c.f. Lai 2005). Demand changes over time – this will<br />
be depicted by the parabola – without loss of generality the costs of supplying resources are<br />
assumed to be zero. If demand is below the fixed-price (left end of the graph), no resource<br />
will be requested. This is because the value for the resource, represented by the demand curve<br />
is below the price. In our example, neither Integrated Systems nor MovieRender would buy<br />
the resources as they are too expensive. As a consequence, there is a loss of utility, which is<br />
denoted by the area below the demand curve. In the middle part of the Graph there are could<br />
be more buyers willing to pay the fixed price, as their demand exceeds the price. For instance<br />
both Integrated Systems and MovieRender have a valuation higher than the fixed price. If the<br />
seller allocates the resource efficiently to the user who values the resource most (Integrated<br />
Systems in our example), there will be unrealized profit for the seller indicated by the striped<br />
gray area, which refers to the difference between the demand curve and the fixed price.<br />
Figure 1: Fixed Pricing (Lai 2005)<br />
Unfortunately, there is no mechanism for the seller to find the user with the highest resource.<br />
In many cases, the buyer receiving the allocation is someone who is below the highest valuation.<br />
This is shown in the right part of Figure 1, where some user with inferior valuation (MovieRender<br />
in our case) receives the allocation. The difference between the demand curve of<br />
the highest valued user and the one who receives the allocation marks the efficiency loss or in<br />
other words waste. A market mechanism has the ability to set the prices in a way that most, if<br />
not the entire utility <strong>und</strong>er the demand curve can be realized; resulting in an efficient allocation,<br />
where a system designer cannot extract more utility. The gist of a market is to couple<br />
price discovery with the valuations of the users, which are expressed in the form of bids.<br />
For instance, in our example Integrated Systems would submit a bid with a high valuation to<br />
the Grid Market, reflecting the high demand for resources. MovieRender would only mod-<br />
8
erately bid for resources, since it does not matter whether the job is conducted now or at a<br />
later time. For Integrated Systems, it is inadvisable to submit a bid lower than its valuation,<br />
since this would lower the probability of winning. Conversely, bidding higher than the valuation<br />
would create the risk of paying more than the job is worth. For MovieRender, the same is<br />
true. As a result of the market mechanism, Integrated Systems receives the resources and pays<br />
a moderate price.<br />
3.2 The need for Self-organization<br />
In summary, markets in Grids represent a business model for the resource owners, as a way to<br />
get money for their resource provision. On the other hand, markets are adequate way to determine<br />
the most efficient allocation making the Grid attractive for users.<br />
Suppose Integrated Systems needs more than one resource for the completion of the accounting<br />
statement, say CPU cycles and memory. Then Integrated Systems needs to formulate bids<br />
for both resources. If the resource demand is varying over time, Integrated Systems would<br />
have to adjust the bids frequently.<br />
For long-term contracts, manual discovery of the bids, appears to be reasonable, it is ineffective<br />
if low-value bids must be frequently submitted. Especially, if the resources are purchased<br />
on-demand, manual bid formulation is too slow and creates too many transaction costs. To<br />
reduce overhead, it is advisable to automate the bidding process. If the automated bid generator<br />
is coupled with the ICT infrastructure and possesses business knowledge on how to submit<br />
bids, the entire bidding process can be virtualized.<br />
Once the bidding process is virtualized, the ICT system of an organization will self-organize<br />
its resource <strong>management</strong>. Overcapacity will be provided over the market while <strong>und</strong>ercapacity<br />
initiates a buying process. The idea of the self-organizing resource <strong>management</strong> in depicted in<br />
Figure 2.<br />
Supply<br />
Demand<br />
Supply<br />
Situation<br />
Information<br />
Policies<br />
Bidding<br />
Agent<br />
Bids<br />
Open<br />
Grid<br />
Market<br />
Bids<br />
Bidding<br />
Agent<br />
Information<br />
Policies<br />
Demand<br />
Situation<br />
Agreements<br />
Agreements<br />
Resource<br />
Owner A<br />
Resource Provision<br />
Payment<br />
User I<br />
Figure 2: Self-organizing Resource Management<br />
Depending on the current demand for resources created by the Grid application, the bidding<br />
agent of the users autonomously defines bids on the basis of user-specific policies. The bids<br />
are submitted to the Open Grid Market. Likewise, on the resource owner side, the bidding<br />
agents publish automatically available resources based on the policies. The Open Grid Market<br />
matches requesting and offering bids and executes them against each other. The allocations<br />
are formulated in agreements (or contracts). The fulfillment of the agreements will be automatically<br />
executed, while the Grid middleware is responsible for the resource provisioning<br />
and the payment system (such as PayPal) for the monetary transfer of f<strong>und</strong>s. Thus, to be effective,<br />
any Grid Market system needs to be composed of three components:<br />
• An Open Grid Market, which determines the allocation in an efficient way and offers in<br />
addition complementary market services. Openness is widely defined and refers to the<br />
9
participants (i.e. all participants are potentially admitted to the market) and to the communication<br />
protocols, so that the market can be accessed by any kind of middleware<br />
system.<br />
• Economic Grid Middleware, which extends common virtualization middleware (e.g.<br />
Globus, GRIA, UNICORE, gLite, VMWare, Xen) in a way that the allocations determined<br />
by the Open Grid Market can be deployed.<br />
• Intelligent Tools, which automate the bidding process.<br />
Any reasonable effort in establishing Grid markets ultimately need to involve all three components,<br />
as these are prerequisite for full automation of the market process (Neumann 2006).<br />
4 Related Work<br />
Since the 60s researchers have motivated the use of markets as a means to cope with those<br />
incentival problems in distributed computing. The first attempt has been made with auctioning<br />
off time slots of the Harvard supercomputer (Sutherland 1968). While this primer was purely<br />
paper-based and restricted to one single computer, subsequent proposals and prototypes offered<br />
automated trading in distributed environments. Despite the fact that the idea of using<br />
markets in distributed computing is not new, no implementation has made it into practice.<br />
4.1 Market-based Grid systems<br />
In the following the most significant efforts will be compiled. Essentially, there are several<br />
ways to classify market-based distributed computing environments. This section will use a<br />
classification along the lines of Buyya et al and will sketch the related work (Buyya, Stockinger<br />
et al. 2001; Yeo and Buyya 2004). The <strong>und</strong>erlying taxonomy is structured aro<strong>und</strong> the<br />
market mechanisms, which are incorporated in the distributed computing environments. 4 It<br />
will be differentiated into the following market mechanisms:<br />
• Posted Price<br />
In posted price settings the price is revealed openly to all participants. As we have<br />
shown in our preceding motivation, the posted price is inadequate from an economic<br />
point of view if demand and/or supply are strongly fluctuating.<br />
• Commodity Market<br />
In a commodity market, the resource owners determine their pricing policy and the user<br />
determines accordingly his/her amount of resources to consume. Pricing can depend on<br />
the parameters such as usage time (e.g. peak-load pricing) or usage quantity (e.g. price<br />
discrimination) (Wolski, Plank et al. 2001). In many cases, it is referred to flat rates,<br />
which boils down to fixed price until a certain amount of resources or a certain time is<br />
reached.<br />
• Bargaining<br />
In bargaining markets, resource owners and users negotiate bilaterally for mutually<br />
agreeable prices. 5 By gradually lowering their claims, resource owners and users eventually<br />
reach an agreement. Logrolling for several attributes can also be included in the<br />
bargaining protocol (Rosenschein and Zlotkin 1994; Kersten, Strecker et al. 2004).<br />
4 From an economic point of view, this taxonomy is not detailed enough. Nevertheless, for a rough classification<br />
of running resource <strong>management</strong> systems this taxonomy is despite its lack of details sufficient.<br />
5 A game-theoretic treatment of applicable bargaining models can be fo<strong>und</strong> at (Wolinsky 1988; De Fraja and<br />
Sakovics 2001; Satterthwaite and Shneyerov 2003).<br />
10
• Contract Net Protocol<br />
In the contract net protocol the user advertises its demand and invites resource owners<br />
to submit bids. Resource owners check these advertisements with respect to their requirements.<br />
In case the advertisement is favorable, the resource owners respond with<br />
bids. The user consolidates all bids, compares them and selects the most favorable<br />
bids(s).<br />
• Proportional Share<br />
Proportional Share assigns resources proportionally to the bids submitted by the users.<br />
• Auctions<br />
Auctions are mediated market mechanisms. An auctioneer collects bids from either one<br />
market side (buyers or sellers) or from both. According to the auction rules (which are<br />
known to all bidders) the auctioneer allocates the resources. Typical one-sided auctions<br />
are the English, First-Price Sealed Bid, Dutch and Vickrey auction. Two sided auctions<br />
are the Double Auctions.<br />
A survey using the above mentioned taxonomy over selected market-based resource <strong>management</strong><br />
systems are summarized in the following.<br />
Computing<br />
Platform<br />
Marketbased<br />
System<br />
Market<br />
Mechanism<br />
Description<br />
Distributed<br />
Computing<br />
SPAWN Auction The SPAWN system provides a market mechanism for trading<br />
CPU times in a network of workstations (Waldspurger, Hogg et al.<br />
1992). SPAWN treats computer resources as standardized<br />
modities and implements a standard Vickrey auction. It is known<br />
from auction theory that the Vickrey auction attains (1) truthful<br />
preference revelation and (2) an efficient allocation of resources<br />
(Krishna 2002). However, SPAWN does not make use of the<br />
neralized Vickrey auction, which can cope with<br />
ties. 6 Furthermore, the Vickrey auction can traditionally neither<br />
cope with multiple attributes nor with different time slots.<br />
Peer-to-<br />
Peer<br />
Stanford<br />
Peers<br />
Bargaining<br />
The Stanford Peers model is a Peer-to-Peer system which<br />
implements auctions within a cooperative bartering model in a<br />
cooperative sharing environment (Wolski, Brevik et al. 2004). It<br />
simulates storage trading for content replication and archiving. It<br />
demonstrates distributed resource trading policies based on<br />
auctions by simulation.<br />
Internet POPCORN Auction POPCORN provides an infrastructure for global distributed<br />
computation (Nisan, London et al. 1998; Regev and Nisan 1998).<br />
POPCORN mainly consists of three entities: (i) A parallel program<br />
which requires CPU time (buyer), (ii) a CPU seller, and (iii) a<br />
market which serves as meeting place and matchmaker for the<br />
buyers and sellers. Buyers of CPU time can bid for one single<br />
commodity, which can be traded executing a Vickrey auction<br />
repeatedly. POPCORN obviously suffers <strong>und</strong>er the same<br />
shortcomings as SPAWN.<br />
6 Users often demand a b<strong>und</strong>le of resources. Since they need more than one resource to complete a task, they<br />
value the b<strong>und</strong>le higher than the sum of its constituents. In an extreme case, the constituents may generate<br />
no value at all – only the b<strong>und</strong>le creates value. If the resources are sequentially auctioned, the user is facing<br />
the so-called exposure risk, once he received one leg of the b<strong>und</strong>le, he is in danger of running a loss, if he<br />
does not get the other leg.<br />
11
Computing<br />
Platform<br />
Marketbased<br />
System<br />
Market<br />
Mechanism<br />
Description<br />
Grids Bellagio Auction Bellagio is intended to serve as a resource discovery and resource<br />
allocation system for distributed computing infrastructures. Users<br />
express preferences for resources using a bidding language, which<br />
support XOR bids. The bids are formulated in virtual currency.<br />
The auction employed in Bellagio is periodic. Bids from users are<br />
only accepted as long as enough virtual currency is left (AuYoung,<br />
Chun et al. 2004).<br />
Grids CATNETS Bargaining In CATNETS, trading is divided into two layers, the<br />
application/service layer and the resource layer. In both layers, the<br />
participants have varying objectives which change dynamically<br />
and unpredictably over time.<br />
In the application/service layer, a complex service is a proxy who<br />
negotiates the access to b<strong>und</strong>les of basic service capabilities for<br />
execution on behalf of the application. Basic services provide an<br />
interface to access computational resources<br />
Agents representing the complex services, basic services and<br />
resources participate in a peer-to-peer trading network, on which<br />
request are disseminated and when an appropriated provider is<br />
fo<strong>und</strong>, agents engage in a bilateral bargaining (Eymann, Reinicke<br />
et al. 2003; Eymann, Reinicke et al. 2003; Eymann, Ardaiz et al.<br />
2005).<br />
Grids G-<br />
Commerce<br />
Commodity<br />
market,<br />
auction<br />
G-Commerce provides a framework for trading computer resources<br />
(CPU and hard disk) in commodity markets and Vickrey auctions<br />
(Wolski, Plank et al. 2001; Wolski, Plank et al. 2001; Wolski,<br />
Brevik et al. 2003). While the Vickrey auction has the<br />
abovementioned shortcomings in grid, the commodity market<br />
typically works with standardized products. Additionally, the<br />
commodity market cannot account for the complementarities<br />
among the resources, as only one leg of the b<strong>und</strong>le is auctioned off,<br />
exposing the bidder to the threshold risk.<br />
Grids Nimrod/G Commodity<br />
market<br />
Grids OCEAN Bargaining/<br />
Contract net<br />
Nimrod/G enables users to define the types of resources needed<br />
and negotiate with the system for the use of a particular set of<br />
resources at a particular price (Buyya, Abramson et al. 2000). This<br />
requires from the system to conduct resource discovery, which can<br />
become quite complex, as the numbers of resources can be large.<br />
Also, does the system need to support price negotiations, which<br />
may be complex as well. Both resource discovery and negotiation<br />
can become very cumbersome, if the users demand b<strong>und</strong>les instead<br />
of single resources.<br />
OCEAN (Open Computation Exchange and Arbitration Network)<br />
is a market-based infrastructure for high-performance computation,<br />
such as Cluster and Grid computing environments (Acharya,<br />
Chokkaredd et al. 2001; Padala, Harrison et al. 2003). The major<br />
components of the OCEAN’s market infrastructure are user<br />
ponents, computational resources, and the <strong>und</strong>erlying market<br />
chanism (e.g. the OCEAN Auction Component). In the OCEAN<br />
framework, each user (i.e. resource provider or consumer) is<br />
represented by a local OCEAN node. The OCEAN node<br />
ments the core components of the system, for instance a Trader<br />
Component, an Auction Component, or a Security Component.<br />
The implemented OCEAN auctions occur in a distributed Peer-to-<br />
Peer manner. The auction mechanism implemented in the OCEAN<br />
12
Computing<br />
Platform<br />
Marketbased<br />
System<br />
Market<br />
Mechanism<br />
Description<br />
framework can be interpreted as a distributed sealed-bid<br />
nuous double-auction (Acharya, Chokkaredd et al. 2001). A trade<br />
is proposed to the highest bidder and the lowest seller. Afterwards,<br />
the trading partner can renegotiate their service level agreements.<br />
The renegotiation possibility one the one hand allows to cope with<br />
multiple attributes and with the assignment of resources to time<br />
slots. Nonetheless, the negotiation makes the results of the<br />
rent auction obsolete. Neither the auction can enfold its full<br />
tial, nor can the negotiation guarantee to achieve an efficient<br />
cation, as competition is trimmed.<br />
Grids Tycoon Proportional<br />
Share<br />
P2P clusters like the Grid and PlanetLab enable in principle the<br />
same statistical multiplexing efficiency gains for computing as the<br />
Internet provides for networking. Tycoon is a market based<br />
distributed resource allocation system based on an Auction Share<br />
scheduling algorithm (Lai, Rasmusson et al. 2004). Tycoon<br />
distinguishes itself from other systems in that it separates the<br />
allocation mechanism (which provides incentives) from the agent<br />
strategy (which interprets preferences). This simplifies the system<br />
and allows specialization of agent strategies for different<br />
applications while providing incentives for applications to use<br />
resources efficiently and resource providers to provide valuable<br />
resources. Tycoon’s distributed markets allow the system to be<br />
fault-tolerant and to allocate resources with low latency. Auction<br />
Share is the local scheduling component of Tycoon.<br />
Table 1: Summary of market-based systems (adapted from (Yeo and Buyya 2004))<br />
Those market-based resource <strong>management</strong> systems, however, are purely prototypical implementations.<br />
Apparently, only two prototypes implement all three components being CAT-<br />
NETS and Tycoon. Tycoon developed by Hewlett Packard offers the most comprehensive<br />
realizations of the system. However, Tycoon still fails to meet the challenge, as the supported<br />
economic models and algorithms are too simplistic and the deployment requires rebooting the<br />
resources before provisioning. CATNETS offers very interesting features but lacks comprehensive<br />
support (e.g., monitoring, multi platform deployment).<br />
All other approaches neglect the provision of decision support. In ordinary commodity markets,<br />
bidders have to come up with their bids themselves by rational deliberation. In Grid<br />
markets this can become extremely cumbersome, as demand and supply fluctuates dynamically<br />
and is in most of the cases unknown. In addition, the complex auctions make it difficult<br />
even for humans to devise suitable strategies.<br />
4.2 Markets are dead, long live markets<br />
Despite the shortcomings of providing intelligent tools, economic middleware extensions and<br />
a Grid market, there are several other reasons why previous systems have not (yet) made it<br />
into practice. Those other reasons partially stem from the fact that several prerequisites on the<br />
technical and economic side have not been given. In recent times, advancements in Grid<br />
Computing and Economics have been achieved so that the prerequisites as well as the environment<br />
have changed by now. Those advancements favor the application of market-based<br />
platforms for Grid systems. Among the most important changes are the following three<br />
(Shneidman, Ng et al. 2005):<br />
• Increasing Demand for Resources: In the past, demand for computational resources<br />
was not an issue. As a consequence, there was no scarcity of resources that would have<br />
13
equired value-based scheduling mechanisms. Thus, past market-based systems hardly<br />
(if ever) saw real field tests, and contention was often artificially generated. This has<br />
changed due to the increasing demand for resources that exceeds supply. A deployed<br />
market-based Grid system could solve real resource conflicts. The confrontation of<br />
market-based Grid systems is essential, as the real data will give researchers the insights<br />
to evaluate (and to adjust) their market-based resource schedulers.<br />
• Improved technical infrastructure: Past systems had to deal with limitations in technical<br />
Grid infrastructures. For example, the enforcement of the allocation decision was<br />
not possible. Market mechanisms without proper enforcement are useless, as the agreements<br />
are not binding. Recent advancements in the area of operating systems and Grid<br />
middleware have overcome this and other technical problems.<br />
• Advanced economic models and algorithms: Market-based Grid systems exclusively<br />
use simple market mechanisms. Simple market mechanisms, however, cannot attain<br />
economic efficiency, as they are unable to capture complementarities (i.e. one resource<br />
creates value for a user only in connection with another resource). During the past decade,<br />
advances have been made in the theory and practice of more complex market mechanisms.<br />
Current mechanisms can support combinatorial bidding, which allow the expression<br />
of multiple resource needs. As such, those bidding languages can represent any<br />
logical combination of resources, such as AND, OR, and XOR. On the other hand, these<br />
combinatorial mechanisms come at an additional cost of computation. Solving those resource<br />
allocation problems becomes fairly computationally expensive. Hence, the developments<br />
in market mechanisms need to be complemented by standard solver, which<br />
can determine the allocation within an acceptable time frame. In recent times, off-theshelf<br />
solvers such as CPLEX have been developed that can reach both efficient allocation<br />
and acceptable computation time.<br />
The key to the successful deployment of market-based Grid systems is a holistic and interdisciplinary<br />
approach. On the one hand, it is important to have a strong grip on the technical infrastructure.<br />
On the other hand, it is also important to make use of economic design knowledge.<br />
The concept of an Open Grid Market is expected to fill the current gap paving the way<br />
towards widespread Grid adoption (Neumann 2008).<br />
5 An Architecture for Open Grid Markets<br />
"As the size and complexity of software systems increases, the design problem goes beyond<br />
the algorithms and data structures of the computation: designing and specifying the overall<br />
system structure emerges as a new kind of problem" (Garlan and Shaw 1993). To address this<br />
complexity, an architecture-based system development approach seems promising. The architecture-based<br />
approach focuses on the organization and global control structure of the system,<br />
on protocols for communication, synchronization, and data access (Garlan and Shaw 1993).<br />
Key element of this approach is the architecture whose importance goes far beyond the simple<br />
documentation of technical elements. The architecture rather serves as the blueprint for both<br />
the system itself and the project developing it.<br />
As an Open Grid Market appears to be a very complex system, the architecture will be presented.<br />
Before, the rationale of the Open Grid Market will be given, revealing why “openness”<br />
is important in the Grid context.<br />
5.1 Architecture Rationale<br />
As presented in the related work, there are several ongoing attempts to establish a marketbased<br />
Grid system. The approach pursued here is different, as it does not strive for "yet another"<br />
Grid market system. Instead, a market system is envisioned that is open, with respect to<br />
14
1. the communication protocols being used, and<br />
2. the <strong>und</strong>erlying local resource managers of the traded resources.<br />
This devotion to openness has major ramifications on the scope of the market: Any potential<br />
Grid user can access the Open Grid Market via open communication protocols. Furthermore,<br />
resource providers with different virtualization platforms and resource managers can easily<br />
plug in the Open Grid Market. The idea is set up a flexible market infrastructure, which can<br />
access resources over all kind of virtualization platforms by means of wrappers.<br />
The openness is intended to offer the possibility of loosely integrating emerging Grid markets.<br />
For example the Open Grid Market needs to combine platforms like network.com or Amazon’s<br />
elastic cloud but should also be capable of accessing Beowulf or MOSIX clusters. This<br />
approach should attract already existing resource providers such as Sun to plug in the Open<br />
Grid Market. Obviously, competition on such Open Grid Markets encourages markets to integrate<br />
and weed out too complex or inefficient platforms.<br />
5.2 Open Grid Market Architecture<br />
The architecture of the Open Grid Market describes the functionalities of the entire market<br />
system in terms of its functional entities, their responsibilities and their dependencies (see<br />
Figure 3) (Neumann, Stoesser et al. 2008).<br />
Figure 3: Open Grid Market Architecture<br />
Boxes represent functional entities that can be encapsulated in a corresponding component.<br />
Arrows represent dependencies, where an arrow from an entity A to an entity B means that<br />
entity A either receives information or consumes a service from entity B.<br />
The architecture is structured along four layers. Ultimately, it is the Grid application and the<br />
available resources that create demand and supply for computational resources and thus give<br />
15
ise to the establishment of an Open Grid Market. Accordingly, Grid applications as well as<br />
the resource mark the fourth and most ‘low-level’ layer. Layer 3 consists of the intelligent<br />
tools that translate the resource demand or supply by the applications and machines into bids.<br />
Layer 2 concerns the market functionalities, whereas layer 1 comprises the core market services<br />
comprising the basic communication infrastructure that is common to all potential markets.<br />
The intuition for dividing layer 2 and 1 is very simple. While the core market services<br />
establish the common trading platform every market uses, the Open Grid market components<br />
can be different from market to market.<br />
5.2.1 Layer 1 – Core Market Services<br />
As aforementioned, state-of-the-art Grid middleware does not provide all the infrastructure<br />
services necessary for supporting an Open Grid Market. Layer 1 extends the standard Grid<br />
middleware by additional infrastructure services so that the Open Grid Market can be hooked<br />
up with the Grid middleware:<br />
• Trusted market exchange service: All communication among market participants –<br />
users and their applications as well as and providers and their resources – is mediated<br />
by this service, which assures that information is routed to the designated recipients in<br />
a secure and reliable way. The trusted market exchange service also enforces policies<br />
defined on specific messages for logging, encryption, and signing.<br />
• Logging: All transactions executed on the market are registered in a secure log for auditing<br />
purposes.<br />
• Market directory: The market directory is a market-enabled extension to the commonplace<br />
service registries in (Grid) middleware. The technical information on Grid<br />
markets directories are enriched by economically relevant information like pricing or<br />
quality of service parameters. The enriched information serves as input to the trading<br />
<strong>management</strong> component on layer 2.<br />
• Market information: The market information service allows market participants to<br />
publish information and to gather information from other participants (e.g., prices, resource<br />
usage levels). Participants can query the service or subscribe to topics. Information<br />
queries consider either instantaneous measures or their history.<br />
5.2.2 Layer 2: Open Grid Market<br />
The Open Grid Market in layer 2 defines the arena, where the published resources are assigned<br />
to the Grid applications, following certain market mechanisms.<br />
• Trading <strong>management</strong>: The trading <strong>management</strong> component is the access point for<br />
the users to the Open Grid Market where they can find the offered resources and submit<br />
their according bids. More specifically, the trading <strong>management</strong> matches the technical<br />
descriptions of the requests obtained from the bids to suitable technical descriptions<br />
of the offered resources collected from the associated Grid market directories.<br />
Subsequently, the trading <strong>management</strong> manages the bidding process according to given<br />
market mechanisms (e.g. MACE auction (Schnizler, Neumann et al. 2006)). If the<br />
bidding process finishes successfully the corresponding bid and offer are submitted to<br />
the contract <strong>management</strong>.<br />
• Contract <strong>management</strong>: The contract <strong>management</strong> component transforms corresponding<br />
pairs of bids and offers to mutually agreed contracts. One important part of these<br />
contracts are service level agreements (SLAs) which define the agreed terms of usage<br />
of the resources and the pricing. The contract <strong>management</strong> also initiates the enforce-<br />
16
ment of the contract, especially the allocation of the sold resources (aided by the<br />
EERM) and the payment process.<br />
• SLA enforcement and billing: The SLA enforcement and billing component is responsible<br />
for the surveillance and enforcement of the contracts it receives from the<br />
contract <strong>management</strong>. The component keeps track of the actual usage of the resources,<br />
makes comparisons to the SLA and (if appropriate) initiates the billing and clearing<br />
according to the results of the comparison.<br />
• Payment: The payment service offers a unified interface to payment, isolating the rest<br />
of the components from the particularities of the payment mechanism. The payment<br />
service also generates appropriate logging/auditing information.<br />
• Security <strong>management</strong>: The security <strong>management</strong> component is intended as the entry<br />
point for a single sign-on mechanism and is responsible for a tamper-proof identity<br />
<strong>management</strong> for the consumers, the suppliers and the constituent components of the<br />
Open Grid Market. Thus, all layers provide security connectors that build the technological<br />
bridges from the respective layers to the security <strong>management</strong>.<br />
• Economically Enhanced Resource Management (EERM): The EERM component<br />
provides a standardized interface to typical Grid middleware (e.g. Globus Toolkit or<br />
Sun Grid Engine). The EERM can shield clients from resource platform specific issues<br />
and also enhance or complement the <strong>management</strong> functions provided by job scheduling<br />
and submission systems. The EERM’s main duties include (i) resource <strong>management</strong><br />
including the <strong>management</strong> functionalities to achieve the expected service levels<br />
and notify of deviations. The resource <strong>management</strong> furthermore coordinates independent<br />
resources to allow co-allocation – if not provided by the fabrics (ii) resource<br />
monitoring (iii) accessing resource fabrics via standardized interfaces to create instances<br />
of resources and later make use of them by the application (Macías, Smith et<br />
al. 2007).<br />
5.2.3 Layer 3: Intelligent Tools<br />
The market participants including the Grid applications and resources are supported by intelligent<br />
tools for an easy access to the Open Grid Market.<br />
• Consumer preference modeling: This component allows for the users to describe<br />
their economic preferences that will determine their bidding strategies on the Open<br />
Grid Market, e.g. they could define if they prefer cheap over reliable resources. One<br />
approach would be to provide the users with a GUI in the form of a simplified ontology<br />
modeling tool to instantiate a given consumer preference modeling language.<br />
• Demand modeling: The user needs a tool to specify the technical requirements of her<br />
Grid application. The technical approach for this component is similar (i.e. ontologybased)<br />
to the preference modeling.<br />
• Business modeling: Analogously to the consumer preference modeling, the providers<br />
have to specify their business models to determine the generation of their offers on the<br />
Open Grid Market. For example one part of such a description could be a pricing<br />
model that specifies if the consumer has to pay for booked time-slots or for the actual<br />
usage. As the example indicates, the models specified by means of this component depend<br />
on the implemented market type.<br />
• Supply modeling: The resource modeling component is the correspondent of the demand<br />
modeling component. Aided by this component the providers can technically<br />
specify their offers.<br />
17
• Bid generation: The bid generation is the intelligent (i.e. agent-based) component that<br />
generates and places the bids of the consumer on the Open Grid Market. For this purpose<br />
it considers the user preferences, the technical requirements and the current state<br />
of the market and derives the bids. The bids are submitted to the trading Management<br />
component of the Open Grid Market. The bid generation component could be implemented<br />
with the help of a rule engine for logical inference over the mentioned inputs.<br />
• Offer generation: The offers are assembled from the technical resource descriptions<br />
and the business model of the respective provider by the offer generation component.<br />
It also publishes the offers at the Grid market directory.<br />
5.2.4 Layer 4: Grid Application<br />
Layer 4 pertains to the Grid applications and Grid resources for trade. At the provider side a<br />
provider IT specialist makes use of the intelligent tools in layer 3 to model the provider's<br />
business strategies and the offered Grid resources.<br />
On the consumer side it has to be distinguished between the Grid application's end user(s)<br />
and the consumer's IT support staff who will use the intelligent tools to model an application's<br />
resource requirements and the consumer's preferences.<br />
6 Organization of the book<br />
This book is devoted to the development of economic models and algorithms for the Open<br />
Grid Market. As such, the book mainly deals with layer 2 and 3 – as those areas comprise the<br />
components, where economic principles can help to improve the entire system. This book<br />
contains eighteen different papers, which are structured along three main sections: “Market<br />
Mechanisms for Grid Systems”, “Economic Component Design”, and “Design Issues and<br />
Extensions”. The papers present joint research conducted with colleagues at the <strong>Institut</strong>e of<br />
Information Management and Systems within the scope of the<br />
• EU-f<strong>und</strong>ed project “CATNETS” (http://www.catnets.org/),<br />
• EU-f<strong>und</strong>ed project “SORMA”, (http://www.sorma-project.eu/),<br />
• BMBF sponsored project “SESAM”,<br />
(http://www.internetoekonomie.uni-karlsruhe.de/)<br />
• project “Billing the Grid” f<strong>und</strong>ed by the Landesstiftung Baden-Württemberg as Landesschwerpunktprogramm<br />
(http://www.billing-the-grid.org/), and<br />
• SSHRC f<strong>und</strong>ed project “Electronic Negotiations, Media and Transactions for Socio-<br />
Economic Interactions”<br />
(http://interneg.concordia.ca/views/bodyfiles/enegotiation/index.html.)<br />
6.1 Market Mechanisms for Grid Systems<br />
The first section deals with different market mechanisms for scheduling Grid resources. Apparently,<br />
it is devoted to the economic modeling of the trading <strong>management</strong> component in<br />
layer 2 of the architecture. All papers focus on the design of market mechanisms for different<br />
market segments of the Grid market. This section consists in total of eight papers, where the<br />
first paper is an overview paper and the subsequent seven papers deal with different market<br />
mechanisms.<br />
6.1.1 Bridging the Adoption Gap – Developing a Roadmap for Trading in Grids<br />
The paper “Bridging the Adoption Gap – Developing a Roadmap for Trading in Grids” argues<br />
that the technology of Grid computing has not yet been adopted in commercial settings<br />
18
due to the lack of viable business models. While in academia Grid technology has already<br />
been taken up, the sharing approach among non for-profit organizations is not suitable for<br />
enterprises. In this paper, the idea of a Grid market is taken up to overcome this Grid adoption<br />
gap. Although this idea is not new, all previous proposals have been made either by computer<br />
scientists being unaware of economic market mechanisms or by economists being unaware of<br />
the technical requirements and possibilities.<br />
This paper derives an economically so<strong>und</strong> set of market mechanisms based on a solid <strong>und</strong>erstanding<br />
of the technical possibilities. More precisely, the paper analyzes the characteristics of<br />
the object traded in Grids. This trading object is closely related to the deployment of software<br />
applications via the EERM. The deployment directly on physical resources or via raw application<br />
services has major ramifications for the trading object and consequently for the requirements<br />
for market mechanisms. Physical resources are essentially commodities, whereas application<br />
services can be both standardized commodities and unique entities.<br />
Based on this analysis, a two-tiered market structure along the distinction between physical<br />
resources and application services is derived, where each tier demands different market mechanisms.<br />
The first tier comprises the markets for physical resources (e.g. CPU, memory) and<br />
raw application services. The second tier comprises the markets for complex application services.<br />
Subsequently, existing Grid market mechanisms are classified according to this market structure.<br />
At the core of this paper, we argue that there is no single market that satisfies all purposes.<br />
Reflecting the distinct requirements of different application models and deployment modes,<br />
a catalogue of co-existing market mechanisms is needed.<br />
This paper provides the outline for the following papers, which present the different mechanisms<br />
of the catalogue and derive their economic properties.<br />
6.1.2 A Truthful Heuristic for Efficient Scheduling in Network-centric Grid OS<br />
The paper “A Truthful Heuristic for Efficient Scheduling in Network-centric Grid OS” argues<br />
that a network-centric Grid OS coupled with market-based scheduling can increase efficiency<br />
in Grid and cluster environments by adequately allocating all available resources. This distinguishes<br />
network-centric Grid OS from state-of-the-art Grid middleware, which rely on batch<br />
processing of idle resources only. While there are highly sophisticated market mechanisms<br />
available for Grid middleware where raw services are being traded (e.g. MACE), there are no<br />
mechanisms available for network-centric Grid OS which rely on interactive application<br />
processing.<br />
The paper strives for filling this need by defining a market mechanism as a multi-attribute<br />
exchange in which resource owners and consumers can publish their demand and supply for<br />
resources deployed as raw services. Exact market-based scheduling mechanisms share one<br />
deficiency: they are quite complex. While this may not be a problem in a cluster setting with a<br />
small number of users, it may prove crucial in an interactive, large-scale Grid OS environment.<br />
Hence a greedy heuristic is designed, which performs a fast scheduling while retaining<br />
truthfulness on the request-side and approximating truthfulness on the provisioning-side of the<br />
market.<br />
In summary, the contributions of this paper are threefold: Firstly, the paper designs a multiattribute<br />
exchange that can be used for Grid OS. Secondly, a greedy heuristic is employed to<br />
solve the scheduling problem. Thirdly, an adequate pricing scheme is developed which assures<br />
that reporting truthfully is a dominant strategy for resource requesters and payments to<br />
resource providers are approximately truthful.<br />
19
6.1.3 Economically Enhanced MOSIX for Market-based Scheduling in Grid OS<br />
The paper “Economically Enhanced MOSIX for Market-based Scheduling in Grid OS” is a<br />
follow-up of the previous paper. Essentially, the paper describes the implementation of the<br />
greedy heuristic in the state-of-the-art network-centric Grid OS MOSIX (i.e. Multi-computer<br />
Operating System for Unix). The paper is a result of the co-operation with Amnon Barak and<br />
Lior Amar (both The Hebrew University of Jerusalem) who co-author this paper.<br />
By introducing market mechanisms to MOSIX, end users can influence the allocation of resources<br />
by reporting valuations for these resources. Current market-based schedulers, however,<br />
are static, assume the availability of complete information about jobs (in particular with<br />
respect to processing times), and do not make use of the flexibility offered by computing systems.<br />
In this paper, the implementation of a novel market mechanism for MOSIX, a state-ofthe-art<br />
<strong>management</strong> system for computing clusters and organizational grids is described. The<br />
market mechanism is designed so as to be able to work in large-scale settings with selfish<br />
agents. Facing incomplete information about jobs’ characteristics, it dynamically allocates<br />
jobs to computing nodes by leveraging preemption and process migration, two distinct features<br />
offered by the MOSIX system.<br />
The contribution of this paper can be summarized as follows. Firstly, a market mechanism is<br />
proposed, which is specifically tailored towards the needs and technical features of MOSIX so<br />
as to generate highly efficient resource allocations in settings with large numbers of selfish<br />
agents. Secondly, the implementation of the proposed market mechanism in MOSIX is presented<br />
which serves as a proof-of-concept.<br />
6.1.4 GreedEx – A Scalable Clearing Mechanism for Utility Computing<br />
The paper “GreedEx – A Scalable Clearing Mechanism for Utility Computing” extends the<br />
mechanism proposed in the paper “A Truthful Heuristic for Efficient Scheduling in Networkcentric<br />
Grid OS” by a more elaborate pricing scheme. The newly developed market mechanism<br />
is dubbed GreedEx, an exchange for clearing utility computing markets, based on a greedy<br />
heuristic.<br />
This market mechanism can be used to perform this clearing in a scalable manner while at the<br />
same time satisfying basic economic design criteria. Besides its computational speed, the<br />
main characteristic of GreedEx is its ability to account for the inter-organizational nature of<br />
utility computing by generating truthful prices for resource requests and approximately truthful<br />
payments to utility computing providers. In general, however, the computational speed of<br />
heuristic comes at the expense of efficiency. Consequently, a numerical simulation is presented,<br />
which shows that GreedEx is very fast and, surprisingly, highly efficient on average.<br />
These results strengthen our approach and point to several interesting avenues for future research.<br />
Obviously, the contribution of this paper is the design as well as analytical and numerical<br />
evaluation of GreedEx that achieves a distinct trade-off, as it conducts fast near-optimal resource<br />
allocations while generating truthful prices on the demand-side and approximately<br />
truthful prices on the supply-side. GreedEx may be used for two purposes: Firstly, utility<br />
computing providers (such as Sun) may choose to run a proprietary GreedEx-based marketplatform<br />
to allocate their scarce computing resources to resource requests in an efficient manner<br />
and to dynamically price these requests. Moreover, a GreedEx-based platform may be run<br />
as an intermediary market which aggregates demand as well as supply across multiple resource<br />
requesters and utility computing providers, thus increasing liquidity and efficiency.<br />
20
6.1.5 Market-Based Pricing in Grids: On Strategic Manipulation and Computational<br />
Cost<br />
The paper “Market-Based Pricing in Grids: On Strategic Manipulation and Computational<br />
Cost” uses the GreedEx mechanism and shows in several propositions the impossibility of a<br />
perfect heuristic that achieves truthfulness on both market sides. In addition, the paper proofs<br />
propositions on split- and merge-proofness as a measure of fairness. In addition, it numerically<br />
shows that manipulation of individual agents very rarely pays off but is very often punished<br />
by the mechanism.<br />
In summary, the paper make the following contributions: an in-depth analysis of these pricing<br />
schemes with respect to the distribution of welfare among resource requesters and providers,<br />
the incentives of single users to try to manipulate the mechanism, and the computational impact<br />
of the pricing schemes on the mechanism is provided. In addition, basic design options of<br />
how the mechanism can be integrated into current Grid schedulers are outlined.<br />
6.1.6 Trading grid services – a multi-attribute combinatorial approach<br />
The paper “Trading grid services – a multi-attribute combinatorial approach” proposes the<br />
derivation of a multi-attribute combinatorial exchange for allocating and scheduling services<br />
in the Grid. In contrast to GreedEx and other approaches, the proposed mechanism accounts<br />
for the variety of services by incorporating time and quality as well as coupling constraints.<br />
The mechanism provides buyers and sellers with a rich bidding language, allowing for the<br />
formulation of b<strong>und</strong>les expressing either substitutabilities or complementarities. The winner<br />
determination problem maximizes the social welfare of this combinatorial allocation problem.<br />
The winner determination scheme alone, however, is insufficient to guarantee an efficient<br />
allocation of the services. The pricing scheme must be constructed in a way that motivates<br />
buyers and sellers to reveal their true valuations and reservation prices. This is problematic in<br />
the case of combinatorial exchanges, since the only efficient pricing schedule, the VCG mechanism,<br />
is not budget-balanced and must be subsidized from outside the mechanism.<br />
The main contribution of this paper is the development of a new pricing family for a combinatorial<br />
exchange, namely the k-pricing rule. In essence, the k-pricing rule determines the price<br />
such that the resulting surpluses to the buyers and sellers divide the entire surplus being accrued<br />
by the trade according to the ration k. The k-pricing rule is budget-balanced but cannot<br />
retain the efficiency property of the VCG payments.<br />
As the simulation illustrates, the k-pricing rule does not rigorously punish inaccurate valuation<br />
and reserve price reporting. Buyers and sellers sometimes increase their individual utility<br />
by cheating. This possibility, however, is only limited to mild misreporting and a small number<br />
of strategic buyers and sellers. If the number of misreporting participants increases, the<br />
risk of not being executed in the auction rises dramatically. As a result, the k-pricing schema<br />
is a practical alternative to the VCG mechanism and highly relevant for an application in the<br />
Grid. The runtime analysis shows that the auction schema is computationally very demanding.<br />
However, the use of approximated solutions achieves adequate runtime results and fairly mild<br />
welfare losses for up to 500 participants. Comparing these results with an existing Grid testbed<br />
(PlanetLab) demonstrates the practical applicability of the proposed auction.<br />
21
6.1.7 A Discriminatory Pay-as-Bid Mechanism for Efficient Scheduling in the Sun N1<br />
Grid Engine<br />
The paper “A Discriminatory Pay-as-Bid Mechanism for Efficient Scheduling in the Sun N1<br />
Grid Engine” regards resources as traded object. This work was jointly conducted with Simon<br />
See from Sun Microsystems, who also co-authored this paper.<br />
In essence, this paper proposes an extended model of the Sanghavi-Hajek pay-as-bid mechanism<br />
a promising addition to the N1GE scheduler. Current technical schedulers require an administrator<br />
to specify user weights based on these users relative importance, regardless of the<br />
dynamic demand and supply situation, leading to inefficiencies.<br />
To this end, the contribution of this paper is twofold:<br />
• Mechanism design: A discriminatory pay-as-bid market mechanism by Sanghavi and<br />
Hajek is presented. Conditions are analytically derived <strong>und</strong>er which this mechanism<br />
outperforms market-based proportional share – the currently most prominent grid<br />
market mechanism – with respect to both provider’s surplus and allocative efficiency.<br />
• Integration into Sun N1GE: It is further showed that this mechanism is not a purely<br />
theoretical construct but that it can be integrated into state-of-the-art grid schedulers to<br />
economically enrich the current allocation logics. The basic design considerations are<br />
conducted for the case of the N1 Grid Engine (N1GE) the scheduler of Sun Microsystems’s<br />
grid platform.<br />
6.1.8 Decentralized Online Resource Allocation for Dynamic Web Service Applications<br />
The paper “Decentralized Online Resource Allocation for Dynamic Web Service Applications”<br />
deals with the definition of an online market mechanism that allocates requests to resources<br />
once the requests are submitted to the system.<br />
Economic scheduling mechanisms introduce economic principles to Grids and thus promise to<br />
increase the systems’ overall utility by explicitly taking into account strategic, interorganizational<br />
settings. Current economic scheduling mechanisms, however, do not fully satisfy<br />
the requirements of interactive Grid applications. In this paper, a video surveillance environment<br />
is introduced and it is shown how its functionality can be encapsulated in Grid services<br />
which are traded dynamically using an economic scheduling mechanism. Subsequently,<br />
a mechanism from the general machine scheduling domain is introduced. The applicability of<br />
the market mechanism to the Grid context is showcased by a numerical experiment. However,<br />
the mechanism suffers from some limitations due to the distinct properties of Grids. These<br />
limitations are pinpointed: extensions to the basic mechanism are introduced, which may remedy<br />
these drawbacks.<br />
6.2 Economic Component Design<br />
The second section deals with economic models and algorithms for the component Bid Generation<br />
(6.2.1 and 6.2.2) on layer 3 and for the components EERM (6.2.3), Contract Management<br />
(6.2.4), SLA enforcement (6.2.5), and Payment (6.2.6) on layer 2.<br />
6.2.1 Self-Organizing ICT Resource Management – Policy-based Automated Bidding<br />
The paper “Self-Organizing ICT Resource Management – Policy-based Automated Bidding”<br />
is devoted to the bidding agent. In essence, it is a premise of this paper that markets can <strong>und</strong>ertake<br />
this task better than any other coordination mechanism. This is especially the case if<br />
the bidding process is fully automated, such that the market itself will handle any unforeseen<br />
problem in the resource <strong>management</strong> process.<br />
22
By bringing autonomic and Grid computing as well as economic principles closer together,<br />
market-based approaches have the potential to achieve an economy-wide adoption of Grid<br />
technology and will thus leverage the exploitation of potential Grid technology offers. To<br />
achieve this potential, many Grid Economics related obstacles must be overcome: markets can<br />
only work well if bids accurately reflect demand and supply. If this is not the case, the market<br />
price cannot reflect the correct situation. Thus, the market price looses its capacity to direct<br />
scarce resources to the bidders who value them most.<br />
This paper is unique by recommending an autonomic computing approach for bidding in Grid<br />
markets. By defining policies the business models of the service consumers and providers can<br />
be represented. The proposed bid generation process is currently quite simple and referring to<br />
observations and straightforward rules.<br />
6.2.2 A Decentralized Online Grid Market – Best Myopic vs. Rational Response<br />
The paper “A Decentralized Online Grid Market – Best Myopic vs. Rational Response” establishes<br />
a bridge between the trading <strong>management</strong> component and the bidding agent. More precisely,<br />
the paper connects the automated bidding approach with the decentralized online mechanism<br />
proposed in “Decentralized Online Resource Allocation for Dynamic Web Service<br />
Applications”.<br />
To this end, the contributions of this paper are twofold. Firstly, the theoretical analysis of Decentralized<br />
Local Greedy Mechanism is based on the assumption of simple agents behaving<br />
according to a so-called “myopic best response strategy”. By means of a simulation with<br />
learning agents, it is shown that an analysis using myopic best response strategies overestimates<br />
the performance of the mechanism and is thus not an appropriate solution concept for<br />
modeling real-world market-based scheduling mechanisms. As a byproduct of this analysis,<br />
the performance benefits of market-based schedulers as opposed to purely technical schedulers<br />
which are solely based on system-centric measures are demonstrated. Secondly, limitations<br />
of the mechanism for the practical use are pointed out. Furthermore, remedies that may<br />
help to mitigate these drawbacks are suggested.<br />
6.2.3 Economically Enhanced Resource Management for Internet Service Utilities<br />
The paper “Economically Enhanced Resource Management for Internet Service Utilities”<br />
local resource <strong>management</strong> systems are equipped with economic principles. The results reported<br />
in the paper have been achieved in cooperation with Jordi Torres and Jorid Guitart<br />
(both Barcelona Supercomputing Center) who co-author this paper.<br />
In this work various economical enhancements for resource <strong>management</strong> are motivated and<br />
explained. In addition a mechanism for assuring Quality of Service and dealing with partial<br />
resource failure without introducing the complexity of risk modeling is presented. It is shown<br />
how flexible dynamic pricing and client classification can benefit service providers.<br />
Various factors and technical parameters for these enhancements are discussed in detail.<br />
Moreover, a preliminary architecture for an Economically Enhanced Resource Manager integrating<br />
these enhancements is introduced. Due to the general architecture and the use of policies<br />
and a policy manager this approach can be adapted to a wide range of situations.<br />
The approach is evaluated considering economic design criteria and using an example scenario.<br />
The evaluation shows that the proposed economic enhancements enable the provider to<br />
increase his benefit. In the standard scenario a 92% of the maximum theoretically attainable<br />
revenue with the enhancements can be achieved in contrast to 77% without enhancements. In<br />
the scenario with partial resource failure the revenue is increased from 57% to 85% of the<br />
theoretical maximum.<br />
23
6.2.4 Situated Decision Support for Managing Service Level Agreement Negotiations<br />
The paper “Situated Decision Support for Managing Service Level Agreement Negotiations”<br />
tackles the interface between contract <strong>management</strong> and the bidding agent. This paper reports<br />
about the results achieved during the cooperation with Rustam Vahidov (Concordia University,<br />
Montreal) who co-author this paper.<br />
This work proposes the application of the situated decision support approach to managing<br />
automated SLA negotiations. The framework is based on the model for situated decision support<br />
that effectively combines human judgment and autonomous decision making and action<br />
by agent components. The key idea behind the approach lies in the managing the fleet of negotiating<br />
agents by the use of a “manager” agent and human decision maker. It is shown<br />
through simulation experiments how this approach performs <strong>und</strong>er a set of simplifying assumptions.<br />
6.2.5 Using k-pricing for Penalty Calculation in Grid Market<br />
The paper “Using k-pricing for Penalty Calculation in Grid Market” is concerned with the<br />
SLA enforcement component, by proposing penalty schemes for SLA violations. This research<br />
has been conducted in cooperation with Omer Rana and Vikas Deora (both Cardiff<br />
University) who also co-author the paper.<br />
Essentially, the paper considers the important role of the design of service level agreements<br />
(SLAs) to distribute risk in Grids. SLAs are crucial as they are contracts that determine the<br />
price for a service at an agreed quality level as well as the penalties in case of SLA violation.<br />
This paper proposes a price function over the quality of service (QoS) on the basis of the<br />
agreements negotiated upon price and quality objective. This function defines fair prices for<br />
every possible quality of a service, which are in line with the business of the customer and<br />
incentivize the provider to supply welfare-maximizing quality. Therewith, penalties can be<br />
calculated for every possible quality level as the difference between the agreed price and the<br />
output of the price function for the effectively met quality. A price function according to the<br />
k-pricing scheme is presented for a single service scenario and for a scenario with multiple<br />
interdependent services.<br />
6.2.6 Rightsizing of Incentives for collaborative e-Science Grid Applications with TES<br />
The paper “Rightsizing of Incentives for collaborative e-Science Grid Applications with TES”<br />
addresses the payment component. The work has been conducted within the scope of the research<br />
project “Billing the Grid”, which strives for designing and implementing a billing system<br />
for academia.<br />
For business Grids the payment component is comparatively easy, as money can be used as<br />
common denominator. In academia, the use of money is banned. Currently resources are used<br />
purely on availability. This raises incentive problems, as there is no fair allocation that takes<br />
the priority of a job into consideration. Researchers are using Grid resources regardless of<br />
whether others need them more urgently. In addition, researchers often tend to not contribute<br />
their own resources to the Grid, due to missing incentives. Thus, researchers are demanding a<br />
Grid infrastructure, which sets the right incentives to share supporting a fair allocation of resources.<br />
The most common instrument to establish incentives is the use of money. However,<br />
the constraint in academia rules out the use of money.<br />
In this paper, a mechanism called Token Exchange System (TES) is proposed. Considering<br />
the requirements from the particle physics community this mechanism combines the advantages<br />
of reputation and payment mechanism in one coherent system to realize a fair allocation.<br />
It enables to build up a Grid infrastructure with an incentive mechanism which is scalable,<br />
24
incentive-compatible and does not require a face-to-face agreement like the current system.<br />
Transaction between institutes can be accomplished without burdening their financial endowment.<br />
Simulations demonstrate that TES has a better performance than a common payment<br />
system combined with a reputation mechanism.<br />
6.3 Design Issues and Extensions<br />
While the first two sections are focusing on economic models and algorithms for the components<br />
of the Open Grid Market architecture, section 3 broadens the scope by addressing (i)<br />
design issues that helps to introduce the results in practice and (ii) extensions of the results<br />
concerning the applicability of the economic models and algorithms in other domains.<br />
6.3.1 Distributed Ascending Proxy Auction – A Cryptographic Approach<br />
The paper “Distributed Ascending Proxy Auction – A Cryptographic Approach” strives for<br />
decentralizing auctions through the use of cryptography. As Grid computing is characterized<br />
by decentralized <strong>management</strong>, the use of auctions is questionable as they are centralized in<br />
nature. The results of the paper have been conducted with Michael Conrad and Christoph<br />
Sorge from the Telematics area within the scope of the interdisciplinary research project “SE-<br />
SAM”.<br />
The paper presents a secure mechanism for distributed ascending proxy auctions. It fosters<br />
privacy and overall correct conduct to a degree that enables the trusted application of an asynchronous,<br />
iterative, second-price auction in systems as open as P2P networks.<br />
The approach produces several desirable properties for the auction process: It eliminates the<br />
dependency on one single auctioneer, and the winning bidder can hide his true valuation, respectively<br />
his highest bid. Using an encrypted bid chain for bidding ensures that only a fraction<br />
of information is revealed to each auctioneer. With any new bid chain each bidder can<br />
freely decide, which auctioneer groups to trust. Robustness is achieved by forming groups of<br />
auctioneers, where even one group member suffices to decrypt a bid step.<br />
In contrast to previous distributed second-price auction mechanisms, this approach is suitable<br />
for iterative open-cry auctions. It is never necessary for all participants to be online at the<br />
same time, which is what makes other approaches highly vulnerable, as it allows the blocking<br />
of an auction fairly by attacking any participating node. In the proposed protocol, all a bidder<br />
needs to do is to convey his bid chains, whereas the auctioneer groups must be accessible during<br />
the whole auction process. However, one single obedient auction group member being<br />
online at a time suffices to conduct the auction with a sufficient standard of security.<br />
Obviously, this paper showcases how inherently centralized auctions can be decentralized<br />
through rigorous use of cryptography. The chosen auction is in particular difficult to decentralize.<br />
It is assumed that the mechanisms presented in section 6.1. are much easier to decentralize.<br />
6.3.2 Technology Assessment and Comparison: The Case of Auction and E-negotiation<br />
Systems<br />
The paper “Technology Assessment and Comparison: The Case of Auction and E-negotiation<br />
Systems” investigates whether the use of auctions or negotiations are better suited for a use in<br />
situations, where the trading object is defined by more than one attribute. The paper introduces<br />
an Information Systems framework, which explains the antecedents of system performance<br />
emphasizing the role of mechanisms. A laboratory experiment shows how the framework can<br />
be used. This work helps to argue why the trading <strong>management</strong> uses auctions rather than negotiations.<br />
The paper has been conducted with Gregory Kersten, Rustam Vahidov and Eva<br />
25
Chen (all from Concordia University, Montreal) within the scope of the Canadian SSHRC<br />
f<strong>und</strong>ed project “Electronic Negotiations, Media and Transactions for Socio-Economic Interactions”.<br />
The purpose of this paper has been to propose a model for studying the impacts of electronic<br />
exchange mechanisms on key variables of interests, both objective, as well as subjective ones.<br />
The proposed TIMES model provides a framework that allows studying of types of exchange<br />
mechanisms in their various implementations within different task, environment, and individual<br />
contexts. These mechanisms could range from the simplest catalogue-based models to<br />
advanced auction and negotiation schemas. Thus, the model can accommodate continuity in<br />
the key design principles of the mechanisms, as opposed to considering them as distinct<br />
lasses. Therefore, one of the key contributions of the model is that it enables the comparison<br />
of various exchange structures in terms of the same set of key dependent factors.<br />
6.3.3 Comparing Ingress and Egress Detection to Secure – Inter-domain Routing: An<br />
Experimental Analysis<br />
The paper “Comparing Ingress and Egress Detection to Secure – Inter-domain Routing: An<br />
Experimental Analysis” addresses the issue of protocol security and adoption. The paper concentrates<br />
on the protocol of secure Border Gateway Protocol. It is assumed that the results can<br />
also be transferred to security mechanisms in the Open Grid market. The work has been conducted<br />
together with Christoph Goebel (Humboldt University Berlin) and Ramayya Krishnan<br />
(Carnegie Mellon University, Pittsburgh) who co-author this paper.<br />
Starting point of the paper is that the global economy and society is increasingly dependent on<br />
computer networks linked together by the Internet. The importance of networks reaches far<br />
beyond the telecommunications sector since they have become a critical factor for many other<br />
crucial infrastructures and markets. With threats mounting and security incidents becoming<br />
more frequent, concerns about network security grow.<br />
It is an acknowledged fact that some of the most f<strong>und</strong>amental network protocols that make the<br />
Internet work are exposed to serious threats. One of them is the Border Gateway Protocol<br />
(BGP) which determines how Internet traffic is routed through the topology of administratively<br />
independent networks the Internet is comprised of. Despite the existence of a steadily<br />
growing number of BGP security proposals, to date neither of them is even close to being<br />
adopted.<br />
The purpose of this work is to contemplate BGP security from a theoretical point of view in<br />
order to take a first step toward <strong>und</strong>erstanding the factors that complicate secure BGP adoption.<br />
Using a definition of BGP robustness we experimentally show that the degree of robustness<br />
is distributed unequally across the administrative domains of the Internet, the so-called<br />
Autonomous Systems (ASs). The experiments confirm the intuition that the contribution<br />
ASs are able to make towards securing the correct working of the inter-domain routing infrastructure<br />
by deploying countermeasures against routing attacks differ depending on their position<br />
in the AS topology. It is shown that the degree of this asymmetry can be controlled by the<br />
choice of the security strategy. The strengths and weaknesses of two f<strong>und</strong>amentally different<br />
approaches in increasing the BGP’s robustness termed ingress and egress detection of false<br />
route advertisements and are compared against each other. Depending on the comparison the<br />
economic implications are discussed.<br />
6.3.4 A Market Mechanism for Energy Allocation in Micro-CHP Grids<br />
The last paper of this book “A Market Mechanism for Energy Allocation in Micro-CHP Grids”<br />
strives for transferring the lessons learnt to other Grid markets. This paper addresses the<br />
26
Energy market and motivates the use of markets for micro Grids. The results of the paper<br />
have been conducted within the scope of the interdisciplinary research project “SESAM”.<br />
Achieving a sustainable level of energy production and consumption is one of the major challenges<br />
in modern society. This paper contributes to the objective of increasing energy efficiency<br />
by introducing a market mechanism that facilitates the efficient matching of energy<br />
(i.e. electricity and heat) demand and supply in Micro Energy Grids. More precisely a combinatorial<br />
double auction mechanism is proposed that performs the allocation and pricing of<br />
energy resources especially taking the specific requirements of energy producers and consumers<br />
into account. The potential role of decentralized micro energy grids and their coupling to<br />
the large scale power grid is outlined. Furthermore an emergency fail over procedure is introduced<br />
that keeps the micro energy grid stable even in cases where the auction mechanism<br />
fails. As the <strong>und</strong>erlying energy allocation problem itself is NP-hard, a fast heuristic for finding<br />
efficient supply and demand allocations is defined. Lastly, the applicability of this approach<br />
is shown through numerical experiments.<br />
7 Conclusion and Outlook on Future Research<br />
In this section the work at hand is recapitulated. Moreover, an outlook on future research in<br />
this domain is outlined.<br />
7.1 Concluding remarks<br />
The vision of a complete virtualization of Information and Communication Technology (ICT)<br />
infrastructures by the provision of ICT resources like computing power or storage over Grid<br />
infrastructures will make the development Open Grid Markets necessary. Over the Open Grid<br />
Market idle or unused resources – computational resources as well as hardware – can be supplied<br />
(e.g. as services) and client demand can be satisfied not only within organization but<br />
also among multiple administrative domains.<br />
In conventional IT environments, the decision “who shares with whom, at what time” is orchestrated<br />
via central server that uses scheduling algorithms in order to maximize resource<br />
utilization. Those central scheduling algorithms, however, have problems, when organizational<br />
bo<strong>und</strong>aries are crossed, and information about demand and supply are manipulated. In particular,<br />
if demand exceeds supply, the scheduling algorithms fail to allocate the resources efficiently.<br />
The further virtualization of ICT infrastructure and services requires technical solutions that<br />
allow for a reasonable – i.e. an economically efficient – allocation of resources. Economic<br />
mechanisms can be the basis for technological solutions that handle these problems by setting<br />
the right incentives to accurately reveal information about demand and supply. Market or<br />
pricing mechanisms foster information exchange and can therefore attain efficient allocations.<br />
Establishing pricing mechanisms raises many questions:<br />
• Can a price be calculated by only considering supply and demand, or will there be the<br />
need for a regulatory body?<br />
• How can the overall supply and demand volume be detected?<br />
• Will participants communicate their supply and demand wishes openly in bids, even if<br />
competitors have access to this information, e.g. to estimate production load?<br />
• How can the bid generation process be automated so as to adequately reflect the <strong>und</strong>erlying<br />
business model?<br />
In this work the design of an Open Grid Market is motivated. In addition, an architecture is<br />
proposed as a blueprint for Grid markets. To establish an Open Grid Market in practice, there<br />
are several obstacles that have to be overcome: The bidding process goes beyond the feasibili-<br />
27
ty of mere manual configuration, so there is a need for intelligent tools that reduce the complexity<br />
of Grid-based systems. In essence, those intelligent tools must support the automation<br />
of the bidding process, which is dependent on the resource supply situation and on business<br />
policies. Additionally, the Open Grid Market needs to be equipped with intelligent monitoring<br />
tools that audit the resources continuously, in order to correct unexpected events such as demand<br />
fluctuations, failure to share resources etc. Most of previous attempts to establish a vivid<br />
market process have hitherto failed, as the <strong>und</strong>erlying economic models were inadequate<br />
or incomplete – most prototypes lacked intelligent tools for automation.<br />
The set-up of an Open Grid Market has even more far-reaching ramifications on the ICT<br />
<strong>management</strong>. Essentially, the Open Grid Market cannot only be used to allocate idle but all<br />
available resources. In this case, ownership of resources does no longer play a central role in<br />
the resource allocation process, as the cheapest resources that assure the required QoS are<br />
allocated to the processes regardless from which organization they come. It is the price, reflecting<br />
the demand and supply situation, which determines which resources are accessed.<br />
The intelligent tools will help to make the Open Grid Market flexible and transparent enough<br />
such that the doubts concerning market-based resource allocation can be resolved. This paper<br />
tries to address all problems of an Open Grid Market that deal with economic models and<br />
algorithms. It intends to bridge the gap between conceptual strength and real world implementations<br />
of Grid platforms.<br />
7.2 Outlook<br />
Assuming that Grid markets will find their way into practice, more challenges need to be addressed.<br />
Today’s competitive markets require companies to flexibly adapt their products and<br />
services to meet rapidly changing business strategies and business models at low cost. Accommodating<br />
innovative services and products is mainly associated with changes in the business<br />
processes – and thus with IT systems embedding these processes – and in the dimensioning<br />
of the IT infrastructure to deliver the services in proper quality. Service orientation in<br />
software engineering grants companies the necessary flexibility by orchestrating services into<br />
different workflows, thus paving the way for new business processes. The shift towards service<br />
orientation also implies a transition from local optimization of the business processes to<br />
collaborative and (administratively) distributed business processes.<br />
Apparently, Service-Oriented Architectures are the best response to meet the flexibility needs<br />
imposed by competition. In this context, Grid technologies complement service orientation to<br />
manage the <strong>und</strong>erlying IT infrastructure. As will be demonstrated within the articles, Grid<br />
technologies are one way to get a grip on the growing complexity of IT infrastructures which<br />
is going along with innovative enterprise applications requiring different hardware, operation<br />
systems and application platforms. In addition, Grid technologies open up new possibilities in<br />
dimensioning IT infrastructures. While service orientation makes resource demand very difficult<br />
to predict, Grid technologies embedded in market-based business models facilitate ondemand<br />
assignment of resources via access to a pool of resources. Instead of attributing a<br />
fixed number of resources to services supporting specific business processes, Grid technologies<br />
dynamically assign resources depending on the actual demand. This is in particular desirable<br />
for SMEs, as it trims down the costs for IT infrastructure quite considerably. Accordingly,<br />
service orientation and Grid technologies in unison are likely to revolutionize modern enterprises<br />
in the future. It is the premise of this book that Open Grid Markets bolster the widespread<br />
adoption of Grids. There are, however, many open problems associated with Grids and<br />
Grid markets.<br />
Currently, Grid markets almost exclusively deal with application-agnostic deployment<br />
schemes. Trading CPU hours and memory is insufficient to support the deployment of complex<br />
systems such as Enterprise Planning Systems (ERP). If application-independent services<br />
28
are traded, the resource allocation problem will become even more complex due to the dependencies<br />
between services. For instance, a workflow of service requests may require several<br />
services in parallel or in sequential order.<br />
All approaches described within this book are devoted to number-crunching activities taking<br />
CPU as the scarce good. Taking into consideration that hardware and thus CPU gets way<br />
cheaper, it raises the question, whether this is the real problem of Grid computing. The <strong>management</strong><br />
of storage or more precisely the distributed data <strong>management</strong> (e.g. replications) raises<br />
many questions that could be used by referring to economic principles. In all of the presented<br />
models, the impact of bandwidth in combination with the topology was widely neglected.<br />
Assuming that bandwidth will become the critical factor there is a strong need to incorporate<br />
this limiting factor into the resource allocation decision.<br />
At the bottom line, the results of this book suggest several intriguing research avenues:<br />
• Analyze the properties of the proposed mechanisms for the respective application<br />
model classes.<br />
• Implement the proposed economic models and algorithms and conduct field studies in<br />
order to get real data.<br />
• Develop market mechanisms, where theory is mostly silent (e.g. task-oriented applications).<br />
• Develop sustainable business models for companies that provide market platforms for<br />
trading Grid services or resources.<br />
• Identify the size and the potential revenue of the single segments of the Grid market<br />
• Identify limits of the use of market mechanisms in Grid.<br />
29
Bibliography<br />
Acharya, N., C. Chokkaredd, et al. (2001). The open computation exchange & auctioning<br />
network (ocean), Department of Computer & Information Science & Engineering,<br />
University of Florida.<br />
Alkadi, I. and G. Alkadi (2006). "Grid Computing: The past, now, and future." Human Systems<br />
Management 25(3): 161-166.<br />
Andrzejak, A., M. Arlitt, et al. (2002). "Bo<strong>und</strong>ing the Resource Savings of Utility Computing<br />
Models." Working Paper HPL-2002-339, Hewlett-Packard Laboratories, Palo Alto,<br />
CA.<br />
AuYoung, A., B. N. Chun, et al. (2004). Resource Allocation in Federated Distributed Computing<br />
Infrastructures. Proceedings of the 1st Workshop on Operating System and<br />
Architectural Support for the On-demand IT InfraStructure.<br />
Barak, A., A. Shiloh, et al. (2005). An Organizational Grid of Federated MOSIX Clusters. CC<br />
Grid, Cardiff, UK.<br />
Bell, G. and J. Gray (2002). "What's Next in High-Performance Computing?" Communications<br />
of the ACM 45(2): 91-95.<br />
Berstis, V. (2002). "F<strong>und</strong>amentals of Grid Computing." IBM Redbooks Paper, Austin, TX.<br />
Butler, D. (2006). "Amazon puts network power online." Nature 444: 528.<br />
Buyya, R. (1999). High Performance Cluster Computing: Architectures and Systems, Prentice<br />
Hall.<br />
Buyya, R., D. Abramson, et al. (2000). "Nimrod/G: An architecture of a resource <strong>management</strong><br />
and scheduling system in a global computational grid." Proceedings of the 4th International<br />
Conference on High Performance Computing in Asia-Pacific Region: 283–289.<br />
Buyya, R., D. Abramson, et al. (2004). The Grid Economy. IEEE Grid Computing.<br />
Buyya, R., H. Stockinger, et al. (2001). Economic models for <strong>management</strong> of resources in<br />
peer-to-peer and grid computing. International Conference on Commercial Applications<br />
for High Performance Computing, Denver, CO.<br />
Carr, N. (2003). "IT doesn't matter." Harvard Business Review 81(5): 41-49.<br />
Carr, N. (2005). "The End of Corporate Computing." MIT Sloan Management Review 46(3):<br />
66-73.<br />
Cherkasova, L., D. Gupta, et al. (2007). "When Virtual is Harder than Real: Resource Allocation<br />
Challenges in Virtual Machine Based IT Environments." HPL-2007-25.<br />
De Fraja, G. and J. Sakovics (2001). "Walras Retrouve: Decentralized Trading Mechanisms<br />
and the Competitive Price." Journal of Political Economy 109(4): 842-863.<br />
Eymann, T., O. Ardaiz, et al. (2005). Catallaxy-based Grid Markets. First International Workshop<br />
on Smart Grid Technologies (SGT05), Utrecht, Netherlands.<br />
Eymann, T., M. Reinicke, et al. (2003). Exploring Decentralized Resource Allocation in Application<br />
Layer Networks. Agent Based Simulation.<br />
Eymann, T., M. Reinicke, et al. (2003). Self-Organizing Resource Allocation for Autonomic<br />
Networks. Workshop on Autonomic Computing Systems (ACS/DEXA 2003), Prague,<br />
Cz, IEEE Computer Society.<br />
Fellows, W., S. Wallage, et al. (2007). Grid Computing - The state of the market,<br />
The451Group.<br />
Foster, I. and C. Kesselman (1998). The Grid: Blueprint for a New Computing Infrastructure,<br />
Morgan Kaufmann.<br />
Garlan, D. and M. Shaw (1993). An Introduction to Software Architecture. Advances in<br />
Software Engineering and Knowledge Engineering. V. Ambriola and G.Tortora. New<br />
Jersey, World Scientific Publishing Company. I.<br />
Gomolski, B. (2003). "Gartner 2003 IT Spending and Staffing Survey Results." Gartner Research,<br />
Stamford, Connecticut.<br />
Hardin, G. (1968). "The Tragedy of the Commons." Science 162: 1243–1248.<br />
30
Irwin, D. E., L. E. Grit, et al. (2004). "Balancing Risk and Reward in a Market-based Task<br />
Service." Working Paper.<br />
Kersten, G. E., S. Strecker, et al. (2004). Protocols for Electronic Negotiation Systems: Theoretical<br />
Fo<strong>und</strong>ations and Design Issues. Proceedings of the 15th International Workshop<br />
on Database and Expert Systems Applications (DEXA), Zaragosa, Spain.<br />
Knight, W. (2006). "Unlocking the Grid." Engineering & Technology 1(3): 42- 45.<br />
Kopparapu, C. (2002). Load Balancing Servers, Firewalls, and Caches, John Wiley & Sons.<br />
Krishna, V. (2002). Auction Theory. San Diego, CA, Academic Press.<br />
Lai, K. (2005). "Markets are Dead, Long Live Markets." Working Paper.<br />
Lai, K., B. A. Huberman, et al. (2004). "Tycoon: a Distributed Market-based Resource Allocation<br />
System." Working Paper.<br />
Lai, K., L. Rasmusson, et al. (2004). "Tycoon: an Implementation of a Distributed, Marketbased<br />
Resource Allocation System." Working Paper.<br />
LSF. (2005). "LSF."<br />
Macías, M., G. Smith, et al. (2007). Enforcing Service Level Agreements using an Economically<br />
Enhanced Resource Manager. 1st Workshop on Economic Models and Algorithms<br />
for Grid Systems (EMAGS 2007) In conjunction with the 8th IEEE/ACM International<br />
Conference on Grid Computing (Grid 2007), Austin, Texas, USA.<br />
Marcus, E. and H. Stern (2000). Blueprints for High Availability: Designing Resilient Distributed<br />
Systems, John Wiley & Sons.<br />
Neumann, D. (2006). Self-Organizing ICT Resource Management - Policy-based Automated<br />
Bidding. eChallenges e-2006 Conference, Barcelona.<br />
Neumann, D. (2008). Engineering Grid Markets. Negotiation and Market Engineering. H.<br />
Gimpel, N. R. Jennings, G. Kersten, A. Ockenfels and C. Weinhardt, Springer.<br />
Neumann, D., C. Holtmann, et al. (2006). "Grid Economics." Wirtschaftsinformatik 48(3):<br />
206-209.<br />
Neumann, D., J. Stoesser, et al. (2008). "A Framework for Commercial Grids - Economic and<br />
Technical Challenges." Journal of Grid Computing: submitted.<br />
Nisan, N., S. London, et al. (1998). Globally distributed computation over the internet - the<br />
popcorn project. 18th International Conference on Distributed Computing Systems,<br />
Amsterdam, The Netherlands, IEEE Computer Society.<br />
Padala, P., C. Harrison, et al. (2003). Ocean: The open computation exchange and arbitration<br />
network, a market approach to meta computing. International Symposium on Parallel<br />
and Distributed Computing.<br />
Rappa, M. A. (2004). "The Utility Business Model and the Future of Computing Services."<br />
IBM System Journal 43(1): 32-42.<br />
Regev, O. and N. Nisan (1998). The POPCORN market - an online market for computational<br />
resources. First international conference on Information and computation economies,<br />
Charleston, South Carolina, ACM Press.<br />
Rosenschein, J. and G. Zlotkin (1994). Rules of Encounter: Designing Conventions for Automated<br />
Negotiation among Computers. Boston, MIT Press.<br />
Satterthwaite, M. and A. Shneyerov (2003). "Convergence of a Dynamic Matching and Bargaining<br />
Market with Two-sided Incomplete Information to Perfect Competition."<br />
Working Paper.<br />
Schnizler, B., D. Neumann, et al. (2006). "Trading Grid Services - A Multi-attribute Combinatorial<br />
Approach." European Journal of Operation Research: forthcoming.<br />
Shneidman, J., C. Ng, et al. (2005). Why Markets Could (But Don’t Currently) Solve Resource<br />
Allocation Problems in Systems. Proceedings of the 10th conference on Hot<br />
Topics in Operating Systems.<br />
31
Sullivan, D. G. and M. I. Seltzer (2000). "Isolation with Flexibility: a Resource Management<br />
Framework for Central Servers." Proceedings of the USENIX Annual Technical Conference:<br />
337–350.<br />
Sun. (2005). "GridEngine."<br />
Sutherland, I. E. (1968). "A futures market in computer time." Communications of the ACM<br />
11(6): 449-451.<br />
Waldspurger, C. A., T. Hogg, et al. (1992). "Spawn: A distributed computational economy."<br />
IEEE Transactions on Software Engineering 18(2): 103–117.<br />
Werthimer, D., J. Cobb, et al. (2001). "SETI@HOME-massively distributed computing for<br />
SETI." Computing in Science and Engineering 3(1): 78-83.<br />
Wolinsky, A. (1988). "Dynamic Markets with Competitive Bidding." Review of Economic<br />
Studies, 55(1): 71-84.<br />
Wolski, R., J. Brevik, et al. (2003). Grid resource allocation and control using computational<br />
economies. Grid Computing - Making The Global Infrastructure a Reality, John Wiley<br />
& Sons: chapter 32.<br />
Wolski, R., J. Brevik, et al. (2004). Grid resource allocation and control using computational<br />
economies. Grid Computing - Making The Global Infrastructure a Reality, John Wiley<br />
& Sons: chapter 32.<br />
Wolski, R., J. Plank, et al. (2001). "Analyzing market-based resource allocation strategies for<br />
the computational grid." International Journal of High Performance Computing Applications<br />
15(3): 258–281.<br />
Wolski, R., J. Plank, et al. (2001). G-commerce: Market formulations controlling resource<br />
allocation on the computational grid. Proceedings of the International Parallel and Distributed<br />
Processing Symposium (IPDPS).<br />
Yeo, C. S. and R. Buyya (2004). "A taxonomy of market-based resource <strong>management</strong> systems<br />
for utility-driven cluster computing." Technical Report, GRIDS-TR-Grid Computing<br />
and Distributed Systems Laboratory, University of Melbourne.<br />
32
Part B<br />
Articles<br />
33
Bridging the Adoption Gap – Developing a Roadmap<br />
for Trading in Grids<br />
Dirk Neumann, Jochen Stoesser, Christof Weinhardt<br />
<strong>Institut</strong>e of Information Systems and<br />
Management (<strong>IISM</strong>)<br />
Universität Karlsruhe (TH)<br />
{Neumann|Stoesser|Weinhardt}@iism.uni-karlsruhe.de<br />
Introduction<br />
Grid computing is increasingly gaining traction in a number of application areas. It describes a computing model<br />
that distributes processing across an administratively and locally dispersed infrastructure. By connecting many<br />
heterogeneous computing resources, virtual computer architectures are created, increasing the utilization of<br />
otherwise idle resources (Foster et al. 2001). By the use of Grid technology, it is possible to set up virtual<br />
supercomputers via the connection of normal servers that cost no more than US$3000 each. Adding and<br />
removing servers is simple, granting extreme flexibility in building up infrastructures. The best example of what<br />
Grid technology can achieve is illustrated by its prominent predecessor SETI@home: Over one million<br />
computers spread across 226 countries were connected to reach a processing power of 418.6 TFLOPS (Cirne et<br />
al. 2006). By comparison, the world’s fastest supercomputer, IBM’s BlueGene/L, has an estimated total<br />
processing power of between 280 and 267 TFLOPS 1 , whereas Google's search engine can muster between 126<br />
and 316 TFLOPS estimated total processing power. The business case for Grids is further <strong>und</strong>erlined by<br />
potential cost savings. It has been projected that Grids may decrease total IT costs by 30 % (Minoli 2004). Thus,<br />
it is not surprising that Insight Research projects an increase in worldwide Grid spending from US$714.9<br />
million in 2005 to approximately US$19.2 billion in 2010 (Anonymous 2006).<br />
1 http://www.top500.org/, accessed on 18 September 2007<br />
34
Currently, Grids are mainly employed within enterprises to connect internal divisions and business units. What is<br />
needed to extend Grid technologies beyond company borders is a set of mechanisms that can enable users to<br />
discover, negotiate, and pay for the use of Grid services on demand. According to The451Group, a leading Grid<br />
research institute, the application of resource trading and allocation models is a crucial success factor for<br />
establishing commercial Grids (Fellows et al. 2007). Recent Grid offerings by Sun Microsystems and<br />
Amazon.com represent a first step in this direction. Sun introduced network.com, which offers computing<br />
services for a fixed price of $1 per CPU hour 2 , while Amazon is currently launching comparable initiatives with<br />
its Amazon Elastic Compute Cloud (Amazon EC2) and the Amazon Simple Storage Service (Amazon S3).<br />
Despite these first approaches, electronic marketplaces for Grid resources have not yet taken off. Very few<br />
customers are using Sun’s network.com. Due to different legal frameworks, network.com was only offered to<br />
customers within the US. The adoption patterns in the US resemble those of the rest of the world – few<br />
customers have adopted Grid markets. 3 But what is the reason for this limited success of Grid markets? Almost<br />
every large computer hardware manufacturer like HP, Sun, or Intel has already worked on or at least considered<br />
the options for Grid markets, but still no Grid market has successfully been launched, creating a Grid adoption<br />
gap.<br />
This paper attempts to explain why Grid market initiatives have failed. The explanation mainly focuses on the<br />
object traded on Grid markets. The problem with current markets for Grids is that they are designed purely as<br />
markets for physical resources. For example, Amazon’s Elastic Compute Cloud only aims to sell CPU hours.<br />
This type of market is by design not relevant for enterprise customers who have deadlines for executing jobs and<br />
who have no idea of how many resources are required to meet this deadline. As a result of the analysis in this<br />
paper, it is concluded that a single Grid market for physical resources such as CPU and memory is insufficient to<br />
ensure successful take-up of Grid computing across organizational bo<strong>und</strong>aries. Instead, a set or catalogue of<br />
different marketplaces is needed to satisfy the diverse needs of different market segments. This paper begins the<br />
process of cataloguing the needed market mechanisms. Thus, this paper provides guidance for potential Grid<br />
2<br />
http://www.network.com/, accessed on 18 September 2007<br />
3 Quite recently, Sun has been facilitating access to customers in 25 countries. It remains to be seen if the<br />
removal of legal concerns will result in broader adoption.<br />
35
market operators (e.g. Telecom companies or hardware vendors such as Sun Microsystems) in the choice of the<br />
market mechanisms needed to increase the impact of Grid markets in commercial settings.<br />
The remainder of this paper is structured as follows. Section 2 discusses ways to define the trading object in Grid<br />
markets and thus forms the backgro<strong>und</strong> for the discussion of market mechanisms. Section 3 explores whether<br />
one marketplace alone is sufficient for meeting the needs of Grid providers and users. A two-tiered market<br />
structure is proposed as a viable solution for structuring commercial Grids. Section 4 discusses the use of<br />
different market mechanisms for this two-tiered market structure and shows which mechanisms are most<br />
adequate for which kind of application. Section 5 concludes the paper with a summary and points to future work.<br />
Backgro<strong>und</strong> Discussion<br />
Applications in a Grid environment can be deployed in two ways, either by directly accessing resources that are<br />
distributed over the network or by invoking a Grid service 4 which encapsulates the respective resources behind<br />
standardized interfaces. These alternative ways of deployment give rise to different requirements for potential<br />
markets.<br />
From a technical point of view, resources are simple to describe as there exists only a finite set of well-defined<br />
resources. A resource may be characterized by its operating system (e.g. Windows, Linux), number and type of<br />
CPUs (e.g. 4 * x86), memory (e.g. 128MB RAM), or any finite number of other attributes. The standardization<br />
of resources offers a simple way to describe them semantically. Glue Schema 5 , for instance, provides a<br />
standardized vocabulary for characterizing computing elements and their status. This in turn facilitates resource<br />
discovery, as matchmaking within the finite domain space is straightforward.<br />
Services on the other hand can be extremely difficult to describe. The service space is essentially infinite due to<br />
the myriad of variations in service design. For the description of complex application services (e.g. a virtual web<br />
4 Grid Services are stateful extensions of Web Services (http://www.globus.org/ogsa/, accessed on 18 September<br />
2007).<br />
5 See http://forge.gridforum.org/sf/projects/glue-wg, accessed on 18 September 2007<br />
36
server service “Apache on Linux”), domain-dependent ontologies may become necessary. In the case of<br />
“resource-near” services, henceforth called “raw application services” (for instance, computational services<br />
which use only CPU and memory), standardized languages such as the Job Submission Description Language<br />
(JSDL) 6<br />
exist. Nonetheless, the indefinite search space tremendously exacerbates service description and<br />
likewise service discovery.<br />
Both resources and services are provided on the basis of Quality-of-Service (QoS) assertions, i.e. essentially<br />
guarantees of access to specific resources or services at specific times and <strong>und</strong>er specific conditions. For<br />
resources, the QoS description is simple, as only standardized properties and the duration of resource access<br />
matter. For services, however, QoS is more difficult, as not only time aspects but also precision and accuracy of<br />
the services play a role. The definitions of precision, accuracy and further parameters depend on the individual<br />
service and cannot be standardized. This also has ramifications for monitoring. While monitoring resource<br />
access is relatively simple, the monitoring of complex application services becomes particularly demanding<br />
when services are intertwined.<br />
Figure 1 shows the different aggregation levels of services and resources with respect to the layers of the Grid<br />
Protocol Architecture (Joseph et al. 2004). On top of this stack, the actual Grid application is located, such as a<br />
demand forecasting tool for Supply Chain Management or computer-aided engineering tools for simulations, etc.<br />
While these applications can be run directly on physical resources, they may access several complex application<br />
services, e.g. services for integrating, aggregating and statistically analyzing vast amounts of data, to simplify<br />
development and deployment by shielding parts of the Grid’s complexity from the application. Complex<br />
application services are so diverse that they cannot reasonably be standardized. These complex application<br />
services might in turn access raw application services which provide standardized interfaces for accessing<br />
various data sources and or computational services. These raw application services are resource-near services<br />
(such as storage, memory or computation services). Raw application services also comprise applications and<br />
software libraries which can be standardized. These physical resources may be CPUs, memory, sensors, other<br />
hardware and software or even aggregated resources such as clusters (e.g. a Condor cluster) and designated<br />
computing nodes.<br />
6 See http://www.ogf.org/documents/GFD.56.pdf, accessed on 18 September 2007<br />
37
Figure 1: Grid Protocol Architecture and the different aggregation levels of services and resources<br />
When relying on direct access to physical resources, executables and external libraries need to be transferred<br />
across the Grid. Typically, state-of-the-art Grid middleware only supports limited resource <strong>management</strong><br />
functionality. In most cases, the middleware does not enforce policies concerning how many resources a job can<br />
consume. Only the local administrator can specify the degree to which resources can be shared. Trading physical<br />
resources is thus difficult to achieve by means of Grid middleware. Trading physical resources, however, is<br />
possible on the operating system level, which supports effective resource <strong>management</strong>. So called Grid Operating<br />
Systems, henceforth Grid OS (e.g. MOSIX 7 ), support resource <strong>management</strong> on the OS Kernel level and are<br />
potentially available for setting up markets (Stößer et al. 2007). With Grid OS, applications need not be altered<br />
to be run on the Grid, as is the case when Grid middleware is being used.<br />
From an economic perspective, Grid resource markets are promising for automation via an organized electronic<br />
market. There are standardized items for sale that potentially attract many buyers and sellers. However, complex<br />
application services have the disadvantage that demand is highly specialized and distributed across niche<br />
markets, such that only few potential buyers are interested in the same or related application services.<br />
One Market Fits All?<br />
In this section, we explore whether it is sufficient to build up and operate a single market for Grids. Based on the<br />
previous backgro<strong>und</strong> description, the answer to this question is straightforward: Designing one Grid market for<br />
7 See http://www.mosix.org, accessed on 18 September 2007<br />
38
all kinds of resources, from physical resources, such as processing power, memory and storage running on native<br />
platforms, to sophisticated virtual resources or application services that b<strong>und</strong>le and enrich such physical<br />
resources, seems inappropriate due to both technical and economical factors.<br />
a) Technical factors<br />
From the technical point of view, differences in the monitoring and deployment of services and resources<br />
make it very difficult to devise a generic system capable of supporting all kinds of resource and service<br />
trading. Furthermore, different deployment mechanisms impose different requirements on the market<br />
mechanism, as we will outline below.<br />
b) Economic factors<br />
From the economic point of view, market mechanisms need to achieve the following standard objectives in<br />
mechanism design (Stößer et al. 2007):<br />
• Allocative efficiency: Allocative efficiency is the overall goal of market mechanisms for Grid resource<br />
allocation. A mechanism is allocatively efficient if it maximizes the utility across all participating users<br />
(welfare or overall “happiness”).<br />
• Budget-balance: A mechanism is budget-balanced if it does not need to be subsidized by outside<br />
payments.<br />
• Computational tractability: The market mechanism needs to be computed in polynomial run time in<br />
the number of resource requests and offers.<br />
• Truthfulness: Truthfulness means that it is a (weakly) dominant strategy for users to reveal their true<br />
valuations to the mechanism.<br />
• Individual rationality: A mechanism is individually rational if users cannot suffer a loss in utility from<br />
participating in the mechanism, i.e. if it is individually rational to participate.<br />
As mentioned in the backgro<strong>und</strong> discussion, trading physical resources and trading application services<br />
impose completely different requirements on the market: while physical resources are more or less<br />
commodities for which auction mechanisms seem to work well, (complex) application services are<br />
inherently non-standardized, thus making auction-like mechanisms inapplicable.<br />
39
From this brief discussion, it can be seen that a one-size-fits-all market for Grids is infeasible from both the<br />
technical as well as from the economic perspective. Due to the heterogeneous properties of Grids, they can be<br />
divided into two different types of markets: resource-near markets for physical resources and raw application<br />
services on the one hand, and markets for complex application services on the other hand, spanning out what<br />
may be called a “two-tiered Grid market structure” 8 as depicted in Figure 2. These two classes of markets are<br />
analyzed below in terms of their requirements for market structure 9 .<br />
Market for Complex<br />
Application Services<br />
Application<br />
Complex<br />
Application<br />
Service<br />
Market for Raw<br />
Application Services<br />
Raw<br />
Application<br />
Service<br />
Application<br />
Complex<br />
Application<br />
Service<br />
Raw<br />
Application<br />
Service<br />
Application<br />
Application<br />
Complex<br />
Application<br />
Service<br />
Complex<br />
Application<br />
Service<br />
Physical<br />
Resource<br />
Physical<br />
Resource<br />
Grid Users<br />
Market for Physical Resources<br />
Integrators<br />
Resource Providers<br />
Tier 2 Tier 1<br />
Tier 1 Markets<br />
Figure 2: Two-tiered market structure<br />
In the resource-near market for physical resources and raw application services, low-level resources such as<br />
processing power, memory, and storage are traded. Demand in this market is generated by complex application<br />
services that need to be executed on these physical resources and raw application services. This setting poses<br />
8 We use the term “market structure” to denote the configuration of marketplaces.<br />
9 It should be noted that there could be n-intermediate markets. We consider only the extreme cases, as they<br />
exhibit different characteristics.<br />
40
special requirements for market mechanisms (Schnizler et al. forthcoming). However, as mentioned earlier, the<br />
requirements also depend on the way resources are deployed.<br />
Deployment as Physical Resource<br />
The technical requirements in markets for physical resources are the following:<br />
• Multi-attributes: Physical resources have quality attributes such as CPU speed, operating platform or<br />
bandwidth. Thus the mechanism needs to cope with multiple attributes simultaneously.<br />
• Bids on b<strong>und</strong>les: Generally, users require a combination of physical resources to execute an application<br />
(e.g. CPU and memory). If the mechanism does not account for bids on b<strong>und</strong>les, the user faces the risk<br />
of obtaining only one part of a b<strong>und</strong>le (the so called “exposure risk”). The market mechanism thus<br />
needs to support requests for b<strong>und</strong>les of resources.<br />
• Online mechanism: The allocation of the mechanism needs to be made instantaneously, as the market<br />
assumes the role of an operating system scheduler. The mechanism thus needs to be lightweight such<br />
that it requires little computation time. Online mechanisms are crucial in the case where information<br />
such as release time or request processing time is only gradually released to the scheduler.. The<br />
scheduler must be able to mitigate past decisions that prove unfortunate when new information enters<br />
the system. For example, facing a decrease in the performance of an application, the mechanism may be<br />
required to allocate additional physical resources in a timely manner.<br />
• Split-proofness: In some scenarios, one might want the mechanism to treat small and large requests in<br />
a fair manner. The mechanism may need to be split-proof in the sense that users cannot improve their<br />
requests’ priority by splitting them into multiple parts and submitting them <strong>und</strong>er different aliases.<br />
• Merge-proofness: The mechanism must be stable in the face of strategic users, who build coalitions to<br />
improve their priority. Thus the mechanism needs to assure that users do not have an advantage by<br />
merging their requests.<br />
Deployment as Raw Application Service<br />
The technical requirements in markets for raw application services are the following:<br />
41
• Multi-attributes: A virtual machine, for instance, may be characterized by the number of CPUs, share<br />
of memory, cache and bandwidth of the <strong>und</strong>erlying physical resource. A simple computational service<br />
may need to provide a certain speed and accuracy.<br />
• Bids on b<strong>und</strong>les: A user program may require multiple interdependent raw application services in<br />
parallel.<br />
• Time constraints: When raw application services are traded, the market mechanism needs to take time<br />
attributes into account. The requesters need to specify their demand, so that the market mechanism can<br />
efficiently schedule the requests according to the availability of resources and price. This situation<br />
differs from the trading of resources, where the market mechanism executes requests and the<br />
corresponding applications upon availability.<br />
• Co-allocation: Capacity-demanding Grid applications usually require the simultaneous allocation of<br />
several homogenous service instances from different providers. For example, a large-scale simulation<br />
may require several computation services to be completed at the same time. This is often referred to as<br />
“co-allocation”. A mechanism for raw application services has to enable and control co-allocations.<br />
• Coupling: For some applications, it may be necessary to couple multiple raw application services into a<br />
b<strong>und</strong>le in order to guarantee that the services are allocated from the same seller and will be executed on<br />
the same machine.<br />
• Resource isolation: Security and performance considerations lead to the requirement of resource<br />
isolation, i.e. that a specific raw application service can only be instantiated once at any given time.<br />
Tier 2 Markets<br />
Along the lines of the two-tiered market structure, complex application services can be decomposed into several<br />
raw application services that can in turn be translated into the physical resources required to execute these<br />
services. For instance, some complex application service might require a basic XML transformer service, which<br />
in turn needs processing power, memory, storage etc. Buyers in such a market request a complex application<br />
service; the provider of this complex application service, the service integrator, is responsible for obtaining the<br />
required raw application services and physical resources in turn, thus hiding parts of the Grid’s complexity from<br />
the buyer. Such a hierarchical masking of complexity seems to be an appropriate approach since service<br />
42
equesters typically have no insight into the resources the complex application service will consume (Eymann et<br />
al. 2006).<br />
The requirements of complex application services for market mechanisms are totally different than for resourcenear<br />
markets. Complex application services are rarely used by two different companies; hence creating<br />
competition via an auction mechanism does not make sense. Instead, the market for complex application services<br />
faces the difficulty of finding a counterpart that offers the exact capabilities needed to execute the application.<br />
As the following requirements suggest, the market mechanism is more “search-oriented”, in terms of the need<br />
for bilateral or multi-lateral negotiation protocols:<br />
• Multi-attributes (see above).<br />
• Workflow support: To support complex services, distributed resources such as computational devices,<br />
data, and applications need to be orchestrated while managing the application workflow within Grid<br />
environments. The market mechanism needs to account for this during design time and run time of the<br />
workflow.<br />
• Scalability: Scalability considers how the properties of a protocol change, as the size of a system (i.e.<br />
the participants in the Grid) increases.<br />
• Co-allocation: (see above).<br />
As pointed out in the introduction, resource-near Grid markets are not a viable solution for enterprises which<br />
typically have to run time-critical applications. However, applications in academia are usually less time<br />
dependent. As such, resource-near markets would be a viable business model for such settings; the users have to<br />
wait until the queued jobs are executed. But clearly, the issue of payment is controversial in academia. Even for<br />
the EGEE Grid 10 , billing and payment will soon become an issue as demand exceeds supply. It seems that<br />
resource-near markets will soon become an adequate model for academic Grids such as EGEE or D-Grid, the<br />
German Grid initiative.<br />
Grid markets that will be widely accessed by enterprises need to be of the form of markets for complex<br />
application services. Consider a manufacturer interested in executing a computer-aided engineering application<br />
10 Enabling Grids for eScience, http://www.eu-egee.org/, accessed on 18 September 2007<br />
43
and deploying it on a computationally intense platform. This complex application service showcases a very<br />
specific service which is likely to be demanded by a single requester only. To accommodate this complex<br />
application service, the service must be decomposed into its constituent raw application services and the required<br />
physical resources. Integrators – companies such as EDO2 11, GigaSpaces 12 , etc. that specialize in aggregating<br />
and disaggregating services into physical resources – are needed to facilitate this decomposition process. The<br />
specialization stems from experience, allowing the identification of service needs by comparing each service<br />
request with similar service requests in the past, where similarity is established in terms of algorithms, data<br />
structures and sizes, etc. Telecommunication companies and hardware producers seeking to virtualize IT<br />
infrastructures naturally have the interest and competency to become Grid services integrators. Table 1<br />
summarizes this discussion.<br />
Market level Tier 1-market (resource-near) Tier 2-market<br />
Deployment Physical Resource Raw Application Service Complex Application Service<br />
Description<br />
languages<br />
GLUE JSDL/RSL Domain-dependent ontologies<br />
Time limits No Yes Yes<br />
Application areas<br />
Small enterprises,<br />
Integrators and resource<br />
All enterprises<br />
academia<br />
providers<br />
Requirements<br />
Multi-attributes, Bids on<br />
Multi-attributes, Bids on<br />
Multi-attributes,<br />
b<strong>und</strong>les, Online<br />
b<strong>und</strong>les, Time attributes, Co-<br />
Workflow support,<br />
mechanism, Split-proof,<br />
Allocation, Coupling, Resource<br />
Scalability,<br />
Merge-proof<br />
Isolation<br />
Co-Allocation<br />
A Roadmap for Grids<br />
Table 1: Types of grid markets<br />
In previous sections, we argued that Grids do not require something like a single global market where all Grid<br />
requests and supplies are aggregated, but a more complex two-tiered market structure. Figure 2 above<br />
summarizes this (meta-)market structure. Applications demand the execution of several complex application<br />
services in Tier 2 markets. In resource-near markets (Tier 1), complex services request physical resources either<br />
11 http://www.cdo2.com/, accessed on 18 September 2007<br />
12 http://www.gigaspaces.com/, accessed on 18 September 2007<br />
44
plainly deployed or accessed via service interfaces. Integrators assume the responsibility for mediating between<br />
requesters unaware of their resource needs and the needed resources.<br />
Based on the preceding discussion, we set up a taxonomy below of known market mechanisms that support<br />
different types of Grid applications. This taxonomy is conceived as a roadmap for further Grid market<br />
developments to help bridge the Grid adoption gap.<br />
Market Mechanisms for Trading Physical Resources<br />
For Grids where physical resources (but not services) are traded, no time restrictions apply. Usually, the<br />
resources themselves are not traded, but rather shares of computing units (e.g. nodes) are. The idea is that<br />
bidding determines the share a user receives from the pool of available nodes. The more resources a user obtains,<br />
the faster the application is completed. The following mechanisms have been proposed for this setting:<br />
• Fair Share (Kay and Lauder 1988): In the Fair Share mechanism, all users get the same share of the<br />
respective resource. This share is dynamically adjusted as new users enter the system or existing users<br />
leave.<br />
• Proportional Share (Lai et al. 2004): In contrast to the Fair Share mechanism, with Proportional Share<br />
the shares can differ across users to reflect each user’s priority. This priority may be determined<br />
dynamically based on the users’ bids for resources: If there are n users in the system and user i has<br />
submitted a valuation of v i , then i will receive a share amounting to v i<br />
/ ∑ j =<br />
v<br />
1 j<br />
.<br />
n<br />
• Pay-as-bid (Sanghavi and Hajek 2005; Bodenbenner et al. 2007): The pay-as-bid mechanism operates<br />
in the same settings as Fair Share and Proportional Share but is specifically designed so as to induce<br />
users to truthful reporting of valuations. It has been shown that pay-as-bid improves on the prominent<br />
Proportional Share mechanism as regards efficiency and provider’s revenue (Bodenbenner et al. 2007).<br />
All three mechanisms, however, share a common drawback: they can only be used in scenarios where one<br />
resource provider serves several consumers, that is, there is no competition among providers. In cases where all<br />
resources are <strong>und</strong>er fully centralized control, this condition is unproblematic. But the idea in Grids is to cross<br />
administrative bo<strong>und</strong>aries. Consequently, there is a need for market mechanisms that support multiple resource<br />
providers.<br />
Market Mechanisms for Trading Application Services<br />
45
Dividing the service market into two parts – raw and complex application services –is too simplistic, as the<br />
timing of demand for services has not yet been considered. This timing is determined by the application itself<br />
and depends on the task the application is performing. We use the term “application model” as a characterization<br />
of the processing mode of the application. This encompasses in particular the workload of the application as well<br />
as the interaction model between applications and the Grid middleware virtualizing the execution platform.<br />
Depending on the application model, different requirements upon the market mechanisms emerge.<br />
• Batch applications (e.g. data mining) are characterized by a planned execution and expected<br />
termination time. Execution is serial and resource demands depend on the parameters, such as the size<br />
of input data.<br />
• Interactive applications (e.g. online data analysis) are applications that require services on demand,<br />
depending on the interactions with users. In contrast with batch applications, it is not possible to plan<br />
the execution and expected termination time of interactive applications far in advance, so unpredictable<br />
peaks of requests can occur within a short time.<br />
• Task-oriented applications are dynamically composed from (sub-)tasks to build more complex tasks.<br />
Service demand depends on the (work-) flow of requests from multiple users (e.g. transaction system of<br />
a bank constitutes).<br />
Most of the market-based approaches relate to batch applications. Batch applications are comparably easy for<br />
two main reasons. First, there is no need to consider a whole workflow with different resource demands on each<br />
echelon of the workflow. Second, the time to determine the allocation can be relatively long; immediacy is not<br />
essential. Thus, complex resource allocation computations can be performed without hampering the whole<br />
application due to latency times devoted to the calculation of the optimal allocation. Furthermore, most of the<br />
practical market-based Grid prototypes consider only one single resource type (e.g. CPU only) and thus make<br />
use of standard auctions (e.g. English auction). In applications other than pure number crunching, those auction<br />
types are inadequate as more than one object (e.g. memory) is required at the same time.<br />
Market Mechanisms for Raw Application Services<br />
As mentioned above, the market mechanisms for raw services depend on the application model. Mechanisms<br />
that we identify as being adequate for batch applications are:<br />
46
• Multi-attribute Combinatorial Auction (Bapna et al. 2006): In the model of Bapna et al., multiple<br />
requesters and providers can trade both computing power and memory for a sequence of time slots.<br />
First, an exact mechanism is introduced. By imposing fairness constraints and a common valuation<br />
across all resource providers, the search space in the <strong>und</strong>erlying combinatorial allocation problem is<br />
structured so as to establish one common, truthful price per time slot for all accepted requests.<br />
Additionally, this introduces a linear ordering across all jobs and time which reduces the complexity of<br />
the allocation problem, which however still remains NP-hard 13 . To mitigate this computational<br />
complexity, a fast, greedy heuristic is proposed at the expense of both truthfulness and efficiency.<br />
• Multi-attribute Exchange (Stößer et al. 2007): The model of Stößer et al. extends the model of Bapna<br />
et al. (2006) in that it accounts for strategic behavior not only on the demand side of the market but also<br />
among resource providers. Stoesser et al. design a truthful scheduling heuristic for Grid operating<br />
systems that achieves an approximately efficient allocation fast. A greedy heuristic is employed to solve<br />
the scheduling problem. It is complemented with an adequate pricing scheme which assures that<br />
reporting truthfully is a dominant strategy for resource requesters and payments to resource providers<br />
are approximately truthful.<br />
• MACE-mechanism (Schnizler et al. forthcoming): Schnizler et al. elaborate a Multi-Attribute<br />
Combinatorial Exchange (MACE). Users are allowed to request and offer arbitrary b<strong>und</strong>les of computer<br />
resources and can specify quality attributes of these resources. The scheduling problem in this<br />
combinatorial setting is NP-hard, and the pricing scheme of MACE yields approximately truthful prices<br />
on both sides of the market. Due to the NP-hardness of the problem, the mechanism is not applicable to<br />
large scale settings and interactive applications that require the immediate allocation of resources.<br />
• Combinatorial Scheduling Exchange (AuYoung et al. 2004): The Bellagio system also implements an<br />
exact combinatorial exchange for computing resources (AuYoung et al. 2004). Its pricing is based on<br />
the approximation of the truthful Vickrey-Clarke-Groves prices proposed by Parkes et al. (2001). As an<br />
exact mechanism it shares the computational drawbacks of MACE.<br />
• Augmented Proportional Share (Stoica et al. 1996): One major drawback of Proportional Share as<br />
proposed by Lai et al. (2004) is that users do not get any QoS guarantee. Generally, users will receive a<br />
13 Informally, for a complex / large problem instance, the allocation problem is not solvable with justifiable<br />
47
larger or smaller share than required and constantly need to monitor their resource needs and the actual<br />
allocation. To mitigate this deficiency, Stoica et al. propose an extension of the Proportional Share<br />
mechanism so that users can request and receive a fixed share of the resource. Thus, this approach<br />
inherits the benefits of the simplicity of Proportional Share and resource reservation if required.<br />
For interactive applications it is impossible to predict demand for raw application services. Thus, market<br />
mechanisms need to allocate these services continuously. This can be realized by frequent call mechanisms,<br />
where bids are collected for a very short time span and right away cleared. This requires that the mechanism is<br />
solvable in few milliseconds. The greedy approaches of the Multi-attribute Combinatorial Auction Heuristic<br />
(Bapna et al. 2006) and the Multi-attribute Exchange (Stößer et al. 2007) are capable of approximately solving<br />
the combinatorial allocation problem in the required timely manner. Alternatively, the mechanism could be a<br />
true online mechanism, which allows the real-time job submission to available resources (e.g. nodes or clusters).<br />
A third way is to introduce a derivative market to hedge against the risk of supernormal resource demand.<br />
Adequate market-based schedulers for interactive applications comprise:<br />
• Decentralized Local Greedy Mechanism (Heydenreich et al. 2006; Stößer et al. 2007): This<br />
mechanism is designed to provide a scalable and stable auctioning protocol. It operates in an online<br />
setting where jobs arrive over time. Its aim is to schedule these jobs so as to minimize the overall<br />
weighted completion times. This scheduling mechanism is complemented by a pricing rule which<br />
induces the users to truthful behavior.<br />
• Augmented Proportional Share (Stoica et al. 1996): The shares which are allocated to the users and<br />
the resulting prices can be adjusted very efficiently as new requests are submitted or requests are<br />
finished. Moreover, users can choose their risk by either fixing the price – and receiving a<br />
corresponding share which may or may not correspond to the true demand of their interactive<br />
application – or by fixing their share and having to pay the corresponding price, which may or may not<br />
match their valuation.<br />
• Derivative Markets (Kenyon and Cheliotis 2002; Rasmusson 2002): Derivative markets perform two<br />
basic functions, hedging and speculation, which become essential in the context of non-storable goods<br />
such as Grid resources. This generates early price signals which support capacity planning for both<br />
buyers and sellers. Rasmusson (2002) proposes the use of options to price network resources. Kenyon<br />
effort.<br />
48
and Cheliotis (2002) follow the same idea by proposing option contracts for network commodities such<br />
as bandwidth. They elaborate a pricing scheme for options which specifically takes into account aspects<br />
of the <strong>und</strong>erlying network topology that may affect prices, e.g. the existence of alternative paths.<br />
Options cannot only be used for hedging purposes but also for speculation (arbitrage) which contributes<br />
liquidity to the market.<br />
The requirements for market mechanisms that support task-oriented applications are very demanding, as all<br />
constituents of the workflow need to be allocated simultaneously, since otherwise the application cannot be fully<br />
executed and is of no value to the user. Currently, there are only bargaining protocols available that guide the<br />
users in their search for all components of the workflow, e.g.:<br />
• SNAP (Czajkowski et al. 2002): The Service Negotiation and Acquisition Protocol (SNAP) supports<br />
the reliable <strong>management</strong> of Service Level Agreements (SLAs). In particular, SNAP supports the<br />
process of negotiating simultaneously across multiple resource providers, a key requirement for<br />
complex and resource-demanding services.<br />
Market Mechanisms for Complex Application Services<br />
Trading complex application services is very demanding as there are not many providers and requesters.<br />
Currently, there is not much research available that aims to develop market mechanisms for trading complex<br />
application services. Hence, the market mechanisms for batch, interactive and task-oriented applications do not<br />
differ substantially. In particular, the MACE-mechanism (Schnizler et al. forthcoming), the Bargaining Protocol<br />
(Czajkowski et al. 2002), and the Decentralized Local Greedy Mechanism (Heydenreich et al. 2006; Stößer et al.<br />
2007) might be appropriate for trading such services. Furthermore, for all applications, the licensing model of<br />
software-as-a-service 14 is applicable. It refers to take-it-or-leave-it pricing, where the vendor sets the price and<br />
the users decide whether or not to purchase.<br />
Table 2 summarizes the existing market mechanisms for each application model.<br />
Application model Physical Resource Raw Application Service Complex Application Service<br />
14 Software-as-a-service refers more to model of software delivery where a service provider (e.g. SAP) offers to<br />
requesters applications that are specifically implemented for one-to-many hosting.<br />
49
Batch<br />
MACE<br />
Multi-attribute Combinatorial<br />
Auction<br />
Augmented Proportional Share<br />
MACE<br />
SNAP<br />
Software-as-a-service<br />
Interactive<br />
Fair Share<br />
Proportional Share<br />
Sanghavi-Hajek<br />
Decentralized Local Greedy<br />
Mechanism<br />
Multi-attribute Auction<br />
Augmented Proportional Share<br />
Derivative Markets<br />
Decentralized Local Greedy<br />
Mechanism<br />
SNAP<br />
Software-as-a-service<br />
Task-oriented<br />
SNAP<br />
SNAP<br />
Software-as-a-service<br />
Table 2: Market Mechanisms Canon for Grids<br />
Conclusion<br />
This paper argues that, despite its promising features, the technology of Grid computing has not yet been adopted<br />
by enterprises due to an absence of adequate business models. Grid markets may help to overcome this obvious<br />
“adoption gap”. This paper attempts to derive an economically so<strong>und</strong> set of market mechanisms based on a solid<br />
<strong>und</strong>erstanding of the technical possibilities.<br />
In the backgro<strong>und</strong> section, we analyzed the characteristics of the object traded in Grids. This trading object is<br />
closely associated with the deployment of software applications. The deployment directly on physical resources<br />
or via raw application services has major ramifications for the trading object and consequently for the<br />
requirements for market mechanisms. Physical resources are essentially commodities, whereas application<br />
services can be both standardized commodities (raw application services) and unique entities (complex<br />
application services).<br />
Based on this analysis, we derived a two-tiered market structure along the distinction between physical resources<br />
and application services, where each tier demands different market mechanisms. The first tier comprises the<br />
markets for physical resources (e.g. CPU, memory) and raw application services. The second tier comprises the<br />
markets for complex application services.<br />
50
We then presented and classified existing Grid market mechanisms according to this market structure. At the<br />
core of this paper, we argue that there is no single market that satisfies all purposes. Reflecting the distinct<br />
requirements of different application models (physical resource vs. raw application service vs. complex<br />
application service) and deployment modes (batch vs. interactive vs. task-oriented), a catalogue of co-existing<br />
market mechanisms is needed. Thus, in order to overcome the adoption gap, Grid market operators such as Sun<br />
Microsystems and Amazon must deploy multiple market mechanisms via a Grid market platform to satisfy the<br />
needs of their enterprise customers, or else integrators must step in to bridge these niche markets.<br />
In essence, this paper suggests several intriguing research avenues. Further research is needed to specify the<br />
properties of both the various application scenarios and the needed market mechanisms. Since existing market<br />
mechanisms represent rather theoretical constructs, they need to be deployed in real-world settings to verify the<br />
<strong>und</strong>erlying models and analytic results. However, existing mechanisms only represent a first step towards filling<br />
the identified market structure and satisfying the various requirements. This basic catalogue needs to be extended<br />
by means of improved and new mechanisms in order to further enhance efficiency and to account for the<br />
strategic dimension inherent in Grid markets. And, of course, the commercial adoption and adaption of Grid<br />
markets requires an analysis of the supply side as well as of the demand side, which was our focus in this paper.<br />
An in-depth analysis of the supply side would address questions such as the following: What are sustainable<br />
business models for companies that provide market platforms for trading Grid services or resources? And how<br />
large is each of the individual markets within the two-tiered market structure?<br />
Chargeable Grid services and sustainable business models are not the final or only answers to the Grid adoption<br />
problem. Other factors, such as trust in the security and reliability of Grid technology, are also relevant. But<br />
better technology alone will not ensure widespread adoption of the Grid: so<strong>und</strong> market mechanisms are also<br />
necessary.<br />
51
References List<br />
Anonymous (2006). Grid Computing: A Vertical Market Perspective 2006-2011. Boonton, NJ, USA, The Insight<br />
Research Corporation.<br />
AuYoung, A., Chun, B. N., Snoeren, A. C. and Vahdat, A. (2004) 'Resource Allocation in Federated Distributed<br />
Computing Infrastructures', Proceedings of the 1st Workshop on Operating System and Architectural<br />
Support for the On-demand IT InfraStructure.<br />
Bapna, R., Das, S., Garfinkel, R. and Stallaert, J. (2006) 'A market design for grid computing', INFORMS<br />
Journal of Computing forthcoming.<br />
Bodenbenner, P., Stößer, J. and Neumann, D. (2007) 'A Pay-as-Bid Mechanism for Pricing Utility Computing ',<br />
20th Bled eConference, Merging and Emerging Technologies, Processes, and <strong>Institut</strong>ions, Bled,<br />
Slovenia.<br />
Cirne, W., Brasileiro, F., Andrade, N., Costa, L. B., Andrade, A., Novaes, R. and Mowbray, M. (2006) 'Labs of<br />
the World, Unite!!!' Journal of Grid Computing 4(3): 225-246.<br />
Czajkowski, K., Foster, I., Kesselman, C., Sander, V. and Tuecke, S. (2002) 'SNAP: A Protocol for Negotiating<br />
Service Level Agreements and Coordinating Resource Management in Distributed Systems', 8th<br />
Workshop on Job Scheduling Strategies for Parallel Processing, Edinburgh, Scotland.<br />
Eymann, T., Neumann, D., Reinicke, M., Schnizler, B., Streitberger, W. and Veit, D. (2006) 'On the Design of a<br />
Two-Tiered Grid Market Structure ', Business Applications of P2P and Grid Computing, MKWI 2006.<br />
Fellows, W., Wallage, S. and Davis, J. (2007). Grid Computing - The state of the market, The451Group.<br />
Foster, I., Kesselman, C. and Tuecke, S. (2001) 'The Anatomy of the Grid - Enabling Scalable Virtual<br />
Organizations', Euro-Par 2001 Parallel Processing, Springer.<br />
Heydenreich, B., Müller, R. and Uetz., M. (2006) 'Decentralization and mechanism design for online machine<br />
scheduling.' in L. Arge and R. Freivalds (eds.) SWAT, Springer, 136-147.<br />
Kay, J. and Lauder, P. (1988) 'A Fair Share Scheduler', Communications of the ACM 31(1): 44-55.<br />
Kenyon, C. and Cheliotis, G. (2002) 'Forward Price Dynamics and Option Design for Network Commodities',<br />
Bachelier Finance Society 2nd World Conference, Crete.<br />
52
Lai, K., Rasmusson, L., Adar, E., Sorkin, S., Zhang, L. and Huberman, B. A. (2004) 'Tycoon: an Implementation<br />
of a Distributed, Market-based Resource Allocation System', Working Paper.<br />
Minoli, D. (2004) A Networking Approach to Grid Computing, New York, John Wiley & Sons, Inc.<br />
Rasmusson, L. (2002) Network capacity sharing with QoS as a financial derivative pricing problem: algorithms<br />
and network design, Ph.D. thesis, Royal <strong>Institut</strong>e of Technology, Stockholm.<br />
Sanghavi, S. and Hajek, B. (2005) 'A new mechanism for the free-rider problem', ACM SIGCOMM Workshop<br />
Peer-to-Peer Economics, Philadelphia, PA.<br />
Schnizler, B., Neumann, D., Veit, D. and Weinhardt, C. (forthcoming) 'Trading Grid Services – A Multiattribute<br />
Combinatorial Approach', European Journal of Operational Research.<br />
Stoica, I., Abdel-Wahab, H., Jeffay, K., Baruah, S., G., J. and Plaxton, G. C. (1996) 'A Proportional Share<br />
Resource Allocation Algorithm for Real-Time, Time-Shared Systems', IEEE Real-Time Systems<br />
Symposium.<br />
Stößer, J., Neumann, D. and Anandasivam, A. (2007) 'A Truthful Heuristic for Efficient Scheduling in Network-<br />
Centric Grid OS', European Conference on Information Systems (ECIS 2007), St. Gallen.<br />
Stößer, J., Roessle, C. and Neumann, D. (2007) 'Decentralized Online Resource Allocation for Dynamic Web<br />
Service Applications ', IEEE Joint Conference on E-Commerce Technology (CEC'07) and Enterprise<br />
Computing, E-Commerce and E-Services (EEE '07), Tokyo, Japan.<br />
53
A TRUTHFUL HEURISTIC FOR EFFICIENT SCHEDULING IN<br />
NETWORK-CENTRIC GRID OS<br />
Jochen Stoesser, <strong>Institut</strong>e of Information Systems and Management (<strong>IISM</strong>), Universität<br />
Karlsruhe (TH), Englerstr. 14, 76131 Karlsruhe, Germany, stoesser@iism.uni-karlsruhe.de<br />
Dirk Neumann, <strong>Institut</strong>e of Information Systems and Management (<strong>IISM</strong>), Universität<br />
Karlsruhe (TH), Englerstr. 14, 76131 Karlsruhe, Germany, neumann@iism.unikarlsruhe.de<br />
Arun Anandasivam, <strong>Institut</strong>e of Information Systems and Management (<strong>IISM</strong>), Universität<br />
Karlsruhe (TH), Englerstr. 14, 76131 Karlsruhe, Germany, anandasivam@iism.unikarlsruhe.de<br />
Abstract<br />
The Grid is a promising concept to solve the dilemma of increasingly complex and demanding<br />
applications being confronted with the need for a more efficient and flexible use of existing resources.<br />
Network-centric Grid Operating Systems (OS) aim at providing users and applications with<br />
transparent and seamless access to heterogeneous Grid resources across different administrative<br />
domains. Scheduling in these heterogeneous environments becomes the key issue since scarce<br />
resources must be distributed between strategic and self-interested users. Market-based algorithms<br />
are deemed to provide a good fit to the Grid environment by accounting for its distinct properties and<br />
the needs of the users. Current market mechanisms, however, leave room for inefficiency and are<br />
computationally intractable. The speed of heuristics becomes a desirable feature in interactive, largescale<br />
Grid OS settings. Heuristics achieve a fast, approximately efficient allocation of resources but<br />
generally fail in preserving truthfulness, i.e. users can benefit from cheating the mechanism. The<br />
contribution of this paper is the proposal of a greedy, market-based scheduling heuristic for networkcentric<br />
Grid OS which does achieve this distinct trade-off: it is designed so as to obtain an<br />
approximately efficient allocation schedule at low computational cost while accounting for strategic,<br />
self-interested users in a heterogeneous environment.<br />
Keywords: network-centric Grid OS, market-based scheduling, heuristic, truthful<br />
1052<br />
54
1 INTRODUCTION<br />
Grids allow the aggregation and sharing of a wide variety of geographically dispersed computer<br />
resources owned by different administrative units (Foster and Kesselman 2004). Grids are typically<br />
used for scientific applications that require massive amounts of computational power and storage such<br />
as protein folding, weather forecasts, or gene encoding. These applications potentially process and<br />
store terabytes of data and the sharing of resources is necessary to reduce the enormous processing<br />
time (Berlich et al. 2005; Cirne et al. 2006). The most famous predecessor of modern Grids is<br />
SETI@home which connected in its peak time over one million computers spread across 226 countries<br />
to achieve processing power of 418.6 TFLOPS (Cirne et al. 2006). For comparison, the world’s fastest<br />
supercomputer – IBM’s BlueGene/L system – has an estimated total processing power of 280<br />
TFLOPS, while Google's search engine system can muster between 126 and 316 TFLOPS. Grids offer<br />
enormous potential by aggregating idle resources that are geographically distributed. For businesses, it<br />
is projected that Grids may reduce total IT costs by 30 % (Minoli 2004).<br />
The use of market mechanisms has often been suggested in situations that establish a price for<br />
resources. The price being paid to resource owners that share idle resources reflects the scarcity of the<br />
provided resource. Since consuming resources is not for free, demand will be restricted and shifted to<br />
those time slots where the resources are really needed. The employment of market mechanisms in<br />
distributed computing is not new. The first attempt was reported at Harvard University where auctions<br />
were used to allocate computing time to users of the PDP-1 computer (Sutherland 1968). Other<br />
examples are Tycoon, Bellagio, and Nimrod/G (Buyya et al. 2000; Lai et al. 2004).<br />
From an economic point of view, the use of market mechanisms in Grid computing is promising, but<br />
most of the potential in state-of-the-art Grid middleware such as Globus, UNICORE, and gLite is<br />
being wasted. Current Grid middleware is based on the premise to share idle resources only and<br />
computational jobs are submitted to these idle resources where they are being processed in batch<br />
mode. It would be more efficient to allocate all available resources over markets. Ideally, resource<br />
allocation should be on-demand such that interactive processing of jobs is permitted as well (Gorlatch<br />
and Müller 2005).<br />
Recently, a new stream of research on network-centric Grid operating systems (OS) has emerged in<br />
Grid computing that especially addresses interactive applications. Network-centric Grid OS provide<br />
the user with seamless and transparent access to Grid resources such as memory and computing<br />
power. While current OS provide only limited support for Grid computing, network-centric Grid OS<br />
adjust their view on the system at run-time contingent upon the particular resources. By means of<br />
network-centric Grid OS, the Grid can be used in an interactive manner. This interactivity certainly<br />
raises problems concerning bandwidth and latency – network-centric-Grid OS are an active research<br />
topic (Padala and Wilson 2003; Gorlatch and Müller 2005). Nonetheless, there are currently some<br />
implementations available that come close to this notion of a network-centric Grid OS. MOSIX, for<br />
example, is realized as an OS virtualization layer that provides a single system image with the Linux<br />
run time environment to users and applications. As such, it allows applications to run on remote nodes<br />
as if they ran locally. Users can run their applications while MOSIX is automatically and transparently<br />
seeking resources and migrating processes among nodes in order to improve overall performance<br />
(Barak et al. 2005).<br />
The concept of network-centric Grid OS coupled with markets is deemed promising to revolutionize<br />
resource allocation in Grids. From their conception, network-centric Grid OS do not only allocate<br />
resources being idle but all available resources. The employment of markets for allocating and<br />
scheduling jobs to resources appears promising to achieve an efficient resource allocation in the<br />
economic sense – allocating those jobs to the resources that value it most.<br />
The problem of scheduling resources is, however, computationally demanding. To be harnessed for<br />
network-centric Grid OS, market mechanisms needs to be solvable quickly. Currently, there is no<br />
market mechanism available that is specifically tailored towards the needs of a network-centric Grid<br />
1053<br />
55
OS. This paper seeks to remedy this gap by designing a truthful scheduling heuristic for MOSIX that<br />
achieves an approximately efficient allocation fast. The contributions of this paper are threefold:<br />
Firstly, the paper designs a multi-attribute exchange that can be used for Grid OS. Secondly, a greedy<br />
heuristic is employed to solve the scheduling problem. Thirdly, an adequate pricing scheme is<br />
developed which assures that reporting truthfully is a dominant strategy for resource requesters and<br />
payments to resource providers are approximately truthful.<br />
The remainder of this paper is structured as follows. In Section 2, the market-based scheduling<br />
problem of network-centric Grid OS systems is introduced. Section 3 suggests (i) a heuristic to solve<br />
this problem and (ii) a pricing scheme that provides the market mechanism with desirable properties.<br />
Section 4 discusses related work in the light of the presented market mechanisms. Section 5 concludes<br />
the paper with a summary and points to future work.<br />
2 MARKET-BASED SCHEDULING IN GRID OS<br />
There are two classes of system users in network-centric Grid OS: Resource requesters who want to<br />
use the computing resources offered in the system for solving a computational problem and resource<br />
providers who want to sell idle resources. We introduce the term “job” to refer to a computational<br />
problem and we will call an atomic b<strong>und</strong>le of resources on which the job or parts of it can be<br />
computed a “node”, e.g. one server within a cluster. If a job gets allocated to a node, we will call this<br />
job (node) a “winning” job (node). We intend to design a direct, sealed-bid mechanism which<br />
allocates periodically: the mechanism collects resource requests and offers from the users for a period<br />
of time and then allocates jobs to nodes on the basis of these submitted requests and offers. Users do<br />
not get to know the requests and offers submitted by other users before the allocation is determined by<br />
the mechanism.<br />
2.1 Economic Requirements<br />
The mechanism is intended to perform job scheduling in a distributed computing environment with<br />
heterogeneous and selfish users. There are a number of economic requirements which a market-based<br />
scheduling mechanism should ideally satisfy in this environment (Schnizler et al. forthcoming):<br />
• Allocative efficiency: A mechanism is said to be allocatively efficient if it maximizes the<br />
utility across all participating users (welfare), i.e. the sum over the valuations of all winning<br />
resource requesters less the sum over the reservation prices of all winning resource providers.<br />
• Budget-balance: the mechanism does not need to be subsidized by outside payments. The<br />
payments coming from the resource requesters cover the payments made to the resource<br />
providers.<br />
• Computational tractability: the mechanism can be computed in polynomial run time in the<br />
size of its input, i.e. the number of resource requests and offers.<br />
• Truthfulness: it is a (weakly) dominant strategy for users to reveal their true valuations to the<br />
mechanism.<br />
• Individual rationality: users cannot suffer any loss in utility from participating in the<br />
mechanism, i.e. it is individually rational to participate.<br />
Budget-balance and individual rationality are hard constraints which the mechanism must suffice. If<br />
these two requirements are not met, the mechanism will not be sustainable. Truthfulness is a desirable<br />
feature since it tremendously simplifies the strategy space of the users; there is no need to reason about<br />
the strategies of other users. Due to the celebrated impossibility result of Myerson and Satterthwaite<br />
(1983), this inherently implies that allocative efficiency can only be approximated: There is no<br />
exchange which is at the same time budget-balanced, individually rational, truthful and allocatively<br />
efficient in equilibrium.<br />
1054<br />
56
There are three basic components to be designed: the bidding language which defines how requests<br />
and offers are specified, the allocation algorithm which decides which job is to be executed on which<br />
node at what time, and the pricing scheme which translates the resulting allocation schedule in<br />
corresponding monetary transfers. In this section, the bidding language and the winner determination<br />
problem will be formalized. The complexity of this scheduling problem forms the rationale for a<br />
market-based heuristic. The pricing scheme will only be specified for the heuristic due to the focus of<br />
this paper.<br />
2.2 Bidding Language<br />
There is on-going research on applying machine learning, regression and filtering techniques for<br />
predicting run times of future jobs (Ali et al. 2004). In many cases a certain resource requester will<br />
submit jobs to a distributed computing environment which are similar in terms of algorithms, data<br />
structures, job size etc. Thus the run time of a new job is likely to follow the run times of “similar”<br />
jobs in the past. Building on this research we will assume that the resources needed for the<br />
computation of a job are known to the requester a priori. This comprises the amount of computing<br />
power, memory, and the run time of the job. A resource requester wanting to compute a job j submits a<br />
request (b j , c j , m j , s j , e j ) to the market mechanism where b j ∈ R + denotes the requester’s willingness to<br />
pay per unit of computing power and time slot, c j ∈ℵand m j ∈ ℵ the minimum amount of computing<br />
power and memory, s j ∈ ℵ the time slot at which the job needs to be started, and e j ∈ ℵ the time slot<br />
until which the job needs to be run. A job can only be executed in its entireness (atomicity). It cannot<br />
be parallelized on multiple nodes but migrate across different nodes over time. Note that job migration<br />
is one of the main features which distinguish Grid OS in general and MOSIX in particular from<br />
traditional machine scheduling domains. A resource offer containing node n consists of a tuple (r n , c¯n,<br />
m¯ n, ε n , λ n ) where r n ∈ R + denotes the reservation price per unit of computing power and time slot, c¯n ∈<br />
ℵ and m¯ n ∈ ℵ the maximum amount of computing power and memory, and ε n ∈ ℵ and λ n ∈ ℵ the<br />
earliest and the latest time slot at which the resources are available. Contrary to atomic resource<br />
requests, resources offered by some node are divisible and moreover freely disposable.<br />
Example: The following resource requests and offers have been submitted to the system:<br />
Job j b j c j m j s j e j Node n r n c¯n m¯ n ε n λ n<br />
J1 11 60 50 2 3 N1 5 150 100 1 5<br />
J2 10 80 50 1 3 N2 7 100 120 2 6<br />
J3 9 90 100 2 5 N3 8 140 100 1 5<br />
J4 8 130 100 4 5<br />
J5 7 80 90 1 3<br />
J6 6 70 50 2 4<br />
J7 5 30 40 1 3<br />
Table 1. Sample resource requests and offers.<br />
Job J1 requests to be run in time slots 2 and 3 and requires at least 60 units of computing power and 50<br />
units of memory in each time slot. The resource requester for job J1 is willing to pay up to $11 per unit<br />
of computing power and time slot, i.e. $11 * 60 * 2 = $1,320 in total. The provider of node N1 offers<br />
at most 150 units of computing power and 100 units of memory in time slots 1 to 5 and requires a<br />
minimum payment of at least $5 per unit of computing power per time slot.<br />
To simplify notation, in the following we will assume that each request and each offer has been<br />
submitted by a separate user and we will use the terms request and job (offer and node)<br />
interchangeably. In contrast to Schnizler et al. (forthcoming), this bidding language does not support a<br />
combinatorial exchange where users can request and offer arbitrary sets of goods and possibly link<br />
multiple requests and offers by means of logical operators. In the scenario at hand, each request and<br />
offer specifies one and the same b<strong>und</strong>le of computing resources and memory and thus rather<br />
implements a multi-attribute exchange (Bichler et al. 1999) by allowing the specification of multi-unit<br />
b<strong>und</strong>les, e.g. 100 units of computing power and 50 units of memory. Note, however, that while the<br />
bidding language is not combinatorial, the scheduling problem of allocating jobs to nodes remains a<br />
combinatorial assignment problem as will be seen below.<br />
1055<br />
57
2.3 Exact Scheduling<br />
Exact mechanisms such as the well-known Vickrey-Clarke-Groves (VCG) mechanism are guaranteed<br />
to find an optimal solution to the scheduling problem. Let N be the set of nodes contained in resource<br />
offerings, J the set of jobs contained in resource requests, T S (n) := {t ∈ ℵ | ε n ≤ t ≤ λ n } the set of time<br />
slots in which node n ∈ N is available according to the respective resource offering, T b (j) := {t ∈ ℵ | s j<br />
≤ t ≤ e j } the set of time slots in which job j ∈ J requests to get executed, and N(j) := {n ∈ N | T b (j) ⊆<br />
T S (n), c j ≤ c¯n, m j ≤ m¯ n} the set of nodes which can potentially execute job j. We introduce the binary<br />
decision variable x jnt ∈ {0,1} with x jnt = 1 if job j will be executed on node n in time slot t and x jnt = 0<br />
otherwise. The scheduling problem can be formalized as the following mathematical integer<br />
programme:<br />
[SP]<br />
max V : = c x ( b −r<br />
)<br />
x jnt<br />
n∈N( j)<br />
b<br />
j∈J j t∈T ( j) n∈N( j)<br />
jnt j n<br />
b<br />
s.. t x ∈{0,1}, j∈J, t∈T ( j), n∈N( j) ( C1)<br />
jnt<br />
∑<br />
∑<br />
∑<br />
∑<br />
∑ ∑ ∑<br />
j∈J<br />
j∈J<br />
b<br />
x ≤1, j∈J, t∈T ( j) ( C2)<br />
jnt<br />
S<br />
x m ≤ m , n∈N, t∈T ( n) ( C3)<br />
jnt j n<br />
S<br />
x c ≤ c , n∈N, t∈T ( n) ( C4)<br />
jnt j n<br />
∑<br />
∑<br />
b<br />
x = ( e − s + 1) x , j∈J,<br />
t∈T<br />
( j) ( C5)<br />
b<br />
u∈T ( j) n∈N( j)<br />
jnu j j n∈N( j)<br />
jnt<br />
b<br />
≥ ∈ ∈ ∈<br />
j jnt n<br />
b x r , j J, n N( j), t T ( j) ( C6)<br />
SP assigns jobs to nodes as to maximize welfare V. The economic scheduling scenario is encoded in<br />
the constraints of this combinatorial assignment problem as follows: (C1) introduces x jnt as binary<br />
decision variable. Jobs can only be assigned to nodes which provide the necessary resources during the<br />
right time slots. (C2) ensures that a job is allocated to at most one node at a time. (C3) and (C4)<br />
specify that for any given time slot and node, the total resource consumption in terms of memory and<br />
computing power cannot exceed the resources provided by this node. (C5) defines atomicity, i.e. every<br />
job is either executed as a whole or it is not executed at all. (C6) ensures that for each allocation of a<br />
job to a node, the requester’s willingness to pay is equal to or greater than the provider’s reservation<br />
price. The optimal allocation schedule X* := (x*) j∈J,n∈N,t∈T b(j) yielding welfare V* can be derived<br />
directly from the solution to SP.<br />
Example: For the sample resource requests and offers listed in Table 1, building and solving SP yields<br />
the optimal allocation schedule X* shown in Figure 1. In time slot 1, only job J2 runs on node N1. The<br />
requester of job J2 is willing to pay up to $10 [per computing cycle] * 80 [computing cycles] = $800<br />
per time slot. The provider of node N1 requires a minimum payment of $5 per computing cycle and<br />
time slot and receives – depending on the pricing which will be elaborated below – at least $5 * 80 =<br />
$400. Thus the allocation of job J2 to node N1 in time slot 1 yields welfare of $800 - $400 = $400.<br />
Across all time slots, X* generates welfare of V* = $3,420.<br />
J1<br />
J2<br />
J3<br />
J4<br />
J5<br />
J6<br />
J7<br />
Exact<br />
scheduling<br />
Slot 1 Slot 2 Slot 3 Slot 4 Slot 5 Slot 6<br />
J2 J2<br />
J2<br />
J4 J4<br />
J1 J1<br />
Slot 1 Slot 2 Slot 3 Slot 4 Slot 5 Slot 6<br />
Node N1<br />
J3 J3 J3 J3 Node N2<br />
Slot 1 Slot 2 Slot 3 Slot 4 Slot 5 Slot 6<br />
Node N3<br />
Figure 1.<br />
Optimal Allocation Schedule.<br />
1056<br />
58
While this exact mechanism generates an optimal allocation schedule in terms of economic efficiency,<br />
this optimality only comes at the expense of computational efficiency: SP is an instance of a multiple<br />
knapsack problem (Ferrari 1994) and thus NP-hard. Computational costs of determining the allocation<br />
and the pricing may become a problem in dynamic network-centric Grid OS with a large number of<br />
resource requests and offers (Waldspurger et al. 1992). In these settings, a trade-off between efficiency<br />
and computational complexity may be desirable to support interactive applications.<br />
3 TRUTHFUL SCHEDULING HEURISTICS<br />
Heuristics aim at obtaining a good but generally suboptimal (in terms of welfare) allocation fast.<br />
Relaxing allocative efficiency not only impacts the outcome determination but also the pricing<br />
scheme: compromising a mechanism in terms of allocative efficiency generally implies compromising<br />
truthfulness as well (Mu’alem and Nisan 2002). Lehmann et al. (2002) and Mu’alem and Nisan<br />
(2002), however, elaborate necessary and sufficient conditions that do yield truthful, approximating<br />
mechanisms for a restricted class of users: known single-minded bidders. Informally, a bidder (user) is<br />
known single-minded if she only desires one specific set of goods and this set is known to the<br />
mechanism. Applying these conditions to the scenario above in order to design a truthful mechanism<br />
suggests itself: in a non-combinatorial multi-attribute exchange such as a market-based scheduler for<br />
Grid OS, resource requesters and providers are only allowed to request and offer one single b<strong>und</strong>le of<br />
resources corresponding to the b<strong>und</strong>le of attributes – in the scenario at hand this is computing power<br />
and memory.<br />
3.1 A Greedy Scheduling Heuristic<br />
Lehmann et al. (2002) and Mu’alem and Nisan (2002) propose greedy heuristics for constructing<br />
truthful heuristics for single-unit combinatorial auctions with one seller and multiple buyers. We build<br />
on this proposal and apply this greedy principle to construct a heuristic for the multi-attribute Grid<br />
exchange at hand with multiple sellers and multiple buyers. This heuristic consists of two basic<br />
phases:<br />
1. Sort the requests j ∈ J and offers n ∈ N in non-increasing and non-decreasing order with<br />
respect to some norm η;<br />
2. Sequentially run through the resulting rankings and allocate the requests with the highest<br />
ranking to the offers with the highest ranking (cf. pseudo code in Figure 2).<br />
(1) Sort jobs j ∈ J in non-increasing order and nodes n ∈ N in non-decreasing order of η.<br />
(2) Run sequentially through the job ranking, starting with the highest ranked job. For each job j:<br />
(3) Run sequentially through the node ranking, beginning with the highest ranked node, and<br />
check if j can be executed, i.e. whether conditions (b j ≥ r m ) ∧ (c j ≤ c¯m) ∧ (m j ≤ m¯ m) are<br />
satisfied for some m ∈ {n 1 ,...,n j } ⊆ N, i.e. the nodes with the highest ranking which can<br />
together accomodate job j, in time slots t ∈ T b (j).<br />
(4) If so, allocate j to {n 1 ,...,n j }; Update the residual capacities of m ∈ {n 1 ,...,n j }, i.e. subtract<br />
m j from m¯ m and c j from c¯m in time slots t ∈ T b (j).<br />
(5) Continue at (2).<br />
Figure 2. Heuristic for obtaining the initial allocation.<br />
The allocative efficiency of this greedy heuristic essentially hinges on norm η used in the sorting phase<br />
(Lehmann et al. 2002). This norm must lead to “efficient” rankings in the sense that the heuristic can<br />
create an approximately efficient allocation based on these rankings. A straightforward choice is to use<br />
b i and r i respectively as these set the valuation for a job (node) in relation to the amount of resources<br />
requested (offered). In conjunction with the sequential allocation rule in phase two, the resulting<br />
heuristic truly implements a greedy allocation algorithm: in each allocation step, it intends to<br />
maximize b j – r n from the objective function of SP above.<br />
1057<br />
59
Example: Assume η = b j , j ∈ J, and η = r n , n ∈ N.<br />
Job j b j c j m j s j e j Node n r n c¯n m¯ n ε n λ n<br />
J1 11 60 50 2 3 N1 5 150 100 1 5<br />
J2 10 80 50 1 3 N2 7 100 120 2 6<br />
J3 9 90 100 2 5 N3 8 140 100 1 5<br />
J4 8 130 100 4 5<br />
J5 7 80 90 1 3<br />
J6 6 70 50 2 4<br />
J7 5 30 40 1 3<br />
Figure 3. Example of the greedy heuristic.<br />
Jobs J1 and J2 can be fully executed on node N1. In time slots 2 and 3, job J3 does not fit on N1 but<br />
only on N2 which is next in the ranking. In time slots 4 and 5, J3 does fit on N1. Since J3 is allocated<br />
to N1 in time slots 4 and 5, J4 does not fit on N1 anymore but only on N3, in contrast to the optimal<br />
allocation X*. In total, X greedy yields welfare V greedy = $3,000 and an approximation ratio of V greedy / V* =<br />
$3,000 / $3,420 ≈ 88 %. The greedy heuristic generates the following allocation schedule X greedy :<br />
J1<br />
J2<br />
J3<br />
J4<br />
J5<br />
J6<br />
Heuristic<br />
scheduling<br />
Slot 1 Slot 2 Slot 3 Slot 4 Slot 5 Slot 6<br />
J2 J2<br />
J2<br />
J3 J3<br />
J1 J1<br />
Slot 1 Slot 2 Slot 3 Slot 4 Slot 5 Slot 6<br />
J3<br />
J3<br />
Slot 1 Slot 2 Slot 3 Slot 4 Slot 5 Slot 6<br />
J7 J4 J4<br />
Node N1<br />
Node N2<br />
Node N3<br />
Figure 4. Greedy allocation schedule.<br />
The question is: How can these allocations of requests to resources and vice versa be translated into<br />
corresponding monetary transfers (prices and payments) according to the desired design criteria<br />
introduced above? Mu’alem and Nisan (2002) and Lehmann et al. (2002) propose a pricing scheme<br />
which generates truthful prices in restricted one-sided, combinatorial auctions with one provider and<br />
multiple requesters. In the remainder, these theoretical results will be applied to the Grid OS context to<br />
generate truthful prices for resource requesters. In contrast to the setting of Mu’alem and Nisan (2002)<br />
and Lehmann et al. (2002), the Grid OS market consists of multiple requesters and multiple providers.<br />
An algorithm will hence be proposed which yields approximately truthful payments to resource<br />
providers.<br />
3.2 Critical Value-based Pricing of Resource Requests<br />
Following the spirit of the truthful Vickrey auction, the payment of job j corresponds to its critical<br />
value given sets J and N of jobs and nodes (Lehmann et al. 2002, Mu’alem and Nisan 2002). The<br />
critical value of φ j is the minimal valuation that j has to report in order to remain in the allocation<br />
schedule, keeping all other requests and offers unchanged. Note that it is not possible to simply take<br />
the valuation of the job with the highest ranking which does not get executed since this job may not be<br />
executable in general due to capacity constraints. Moreover, removing j from the winning allocation<br />
might change the allocation of other jobs within the winning allocation as well. To determine the<br />
critical value for each winning job j, we need to determine the allocation without j: we successively<br />
allocate all other jobs from the ranking and, after having allocated a job, check whether j can still be<br />
accommodated (cf. pseudo code in Figure 5).<br />
1058<br />
60
(1) Sort jobs k ∈ J, k ≠ j, in non-increasing order and nodes n ∈ N in non-decreasing order of η.<br />
(2) Is any job k ≠ j left in the ranking?<br />
If not, set φ j = max r<br />
m∈ { n1<br />
,..., nj<br />
) m<br />
where {n 1 ,...,n j } are the nodes with the highest rankings<br />
which can together accommodate j, and finish.<br />
If so, select job k with the highest ranking.<br />
(3) Can k be accommodated? If not, continue at (2). If so, allocate k to the nodes {n 1 ,...,n k } ⊆ N<br />
with the highest ranking that can accommodate k, and update the residual capacities of m ∈<br />
{n 1 ,...,n k }.<br />
(4) Can j still be accommodated on some nodes {n 1 ,...,n j } ⊆ N? If not, set φ j = b k and finish.<br />
(5) If so, set r: = maxm ∈{ n1<br />
,..., nj<br />
)<br />
rm. Either r < b k , then continue at (2); or r ≥ b k , then it is cheaper<br />
for j to push away k than to take the next available nodes {n 1 ,...,n j }; set φ j = b k and finish.<br />
Figure 5.<br />
Determining the critical value of job j and refining the initial allocation.<br />
The critical value must be computed once for each winning job j. However, compared to the<br />
computational intractability of the VCG mechanism, the computation now runs in polynomial time.<br />
Every winning job j has to pay p greedy,j = (e j – s j + 1)c j φ j whereas every rejected job pays nothing.<br />
Example: Applying the algorithm to the example above, we get p greedy,J1 = (e J1 – s J1 + 1)c J1 φ J1 =<br />
120·b J7 = 120·r N1 = $600, i.e. the “cheapest” option for job J1 is to push away J7 by bidding $5 (plus<br />
some ε) which happens to equal the minimum reservation price of node N1. In total, the mechanism<br />
collects a revenue R of $6,400 from the resource requesters.<br />
Job j J1 J2 J3 J4<br />
φ j 5 5 7 8<br />
p greedy,j 600 1,200 2,520 2,080<br />
Table 2. Payments of resource requesters.<br />
The greedy allocation scheme combined with critical-value based pricing creates a truthful scheduling<br />
heuristic with respect to resource requesters: It is a weakly dominant strategy for resource requesters<br />
to report their true valuation for a job (the required resources). The payment depends on the critical<br />
value which is independent of the reported valuation. Assume a risk-neutral requester j has reported<br />
her true valuation v j , i.e. b j = v j . Then there are four cases to be considered.<br />
State\Action Decrease b j Increase b j<br />
j was accepted Case 1 Case 2<br />
j was rejected Case 3 Case 4<br />
Table 3. Proof sketch.<br />
Suppose j has won. Reporting a lower valuation (Case 1) might have resulted in j not being accepted<br />
while it would not have reduced j’s payment. Reporting a higher valuation (Case 2) would not have<br />
generated more utility either since j has been accepted anyways.<br />
Now suppose j has been rejected (φ j > b j = v j ). Reporting a lower valuation (Case 3) would still have<br />
left j outside the winning allocation. Reporting a higher valuation (Case 4) might even have resulted in<br />
a loss: if j had been accepted, b j > φ j > v j .<br />
3.3 Budget-balanced, Approximately Truthful Payments to Resource Providers<br />
The concept of critical value-based pricing cannot be applied when determining the payments for<br />
divisible resource offers. The concept requires a binary decision in the sense that an offer is either<br />
fully executed or it is not executed at all. We allow divisibility of offers, i.e. not all offered resources<br />
need to be used but the offer can be executed partially.<br />
1059<br />
61
The algorithm for determining the critical values ensures that each winning job pays at least the<br />
reservation price of the node it is allocated to. If there is competition from another job for the same<br />
slot, the critical value will exceed the reservation price. The question is: How can revenue R be<br />
distributed to the resource providers so as to ensure budget-balance, individual rationality and to<br />
approximate truthfulness? The VCG mechanism is generally not budget-balanced (Schnizler et al.<br />
forthcoming) and, combined with the heuristic above, not truthful (Mu’alem and Nisan 2002). A<br />
straightforward approach is to adopt the algorithm of Parkes et al. (2002) to our greedy heuristic. The<br />
basic idea of their algorithm is to approximate the VCG mechanism’s truthfulness by computing<br />
discounts Δ parkes that minimize the distance to the VCG-discounts Δ vick while ensuring budget-balance.<br />
In our algorithm, we first compute Δ temp,n := V greedy – (V –n ) greedy for each winning node n. The<br />
<strong>und</strong>erlying assumption is that the greedy Δ temp,n (which may in total exceed the surplus<br />
R – ∑ n∈N<br />
v^n(X greedy ) from the request-side) approximate the truthful VCG-discounts Δ vick,n . We then<br />
solve the mathematical programme<br />
[BB greedy ]<br />
( Δ Δ<br />
temp greedy )<br />
∑ Δ = −<br />
n∈N<br />
greedy,<br />
n ∑<br />
min L ,<br />
Δ greedy<br />
greedy, n temp,<br />
n<br />
greedy<br />
( )<br />
s.. t R vˆ<br />
X ( C7)<br />
n∈N<br />
0 ≤Δ ≤Δ , n∈N ( C8)<br />
The aim of BB greedy is to compute discounts Δ greedy,n as to approximate Δ temp,n in turn while ensuring<br />
(strong) budget-balance by distributing the surplus R – ∑ n∈N<br />
v^n(X greedy )) to resource providers (C7).<br />
Parkes et al. (2002) suggest and examine various distance functions analytically and numerically and<br />
recommend the “threshold function” L 2 (Δ temp, Δ greedy ) := ∑ n∈N<br />
(Δ temp,n – Δ greedys,n ) 2 as it minimizes the<br />
“residual degree of manipulation freedom” (Parkes et al. 2002), that is the maximum amount of utility<br />
a user can gain from reporting untruthfully. Analogously to the algorithm of Parkes et al. (2002), the<br />
optimal Δ greedy,n can be computed without explicitly having to solve the non-linear programme BB greedy .<br />
Let ∞ = Δ temp,0 ≥ Δ temp,1 ≥ Δ temp,2 ≥ … ≥ Δ temp,|N| ≥ Δ temp,|N|+1 = 0 be the partial ordering over the<br />
temporary greedy discounts for the winning nodes. Then, for the threshold function L 2 , Parkes et al.<br />
(2002) show that there is an interval K and a unique point C t * within this interval such that<br />
∑<br />
greedy<br />
( R−∑<br />
vˆ<br />
n ( X ))<br />
K<br />
Δ −<br />
* i=<br />
1 temp,<br />
i<br />
n∈N<br />
C<br />
temp K + t temp K<br />
Δ ≤ = ≤ Δ ( C9)<br />
, 1 ,<br />
K<br />
and Δ greedy,n = max{0, Δ temp,n - C t * } solves BB greedy . Node n receives a positive discount if n’s temporary<br />
greedy discount is greater than the threshold C t * . Otherwise n does not get any discount. C t * can be<br />
computed by running sequentially through the partially ordered temporary discounts and checking if<br />
condition (C9) is satisfied.<br />
Example: In the example, R = $6,400 > $6,040 = ∑ n∈N<br />
v^n(X greedy ), i.e. there are $360 above the<br />
reservation prices which are to be distributed as to approximate truthful payments. Solving BB greedy<br />
with distance function L 2 leads to C t * = $1,680 and the greedy discounts listed in Table 4. The total<br />
surplus of $360 is allocated to node N1, N2 and N3 only receive payments amounting to their<br />
reservation prices.<br />
Node n v^n(X greedy ) (V –n ) greedy p temp,n Δ temp,n p greedy,n Δ greedy,n<br />
N1 -2,700 960 -4,740 2,040 -3,060 360<br />
N2 -1,260 2,820 -1,440 180 -1,260 0<br />
N3 -2,080 3,000 -2,080 0 -2,080 0<br />
Σ -6,040 -8,260 2,220 -6,400 360<br />
Table 4. Approximately truthful greedy payments.<br />
n<br />
1060<br />
62
3.4 Evaluation of the Heuristic<br />
The greedy heuristic is designed so as to obtain approximately efficient allocations fast. It runs in<br />
polynomial time in the size of resource requests and offers: In the first phase, requests and offers get<br />
sorted in O(|J|·log |J|) and O(|N|·log |N|). In the second phase, the greedy allocation scheme<br />
sequentially runs through the |J| sorted requests and for each job j tries to allocate j to one of the |N|<br />
sorted offers. Hence the allocation phase runs in O(|J|·|N|). Note that the feasibility of allocating a<br />
specific job to a specific node can be tested in O(1) since the number of attributes which need to be<br />
checked is constant. Inherently, this speed is only achieved at the expense of allocative efficiency<br />
compared to exact mechanisms. In a theoretical worst case analysis, the approximation of the efficient<br />
allocation can be made arbitrarily bad with respect to norm b j , r n suggested above by means of simple<br />
examples. The pricing scheme of the greedy mechanism consists of two parts: a truthful pricing<br />
scheme for resource requesters and an approximately truthful payment scheme for resource owners.<br />
BB greedy links these two sides as to ensure budget-balance. The resulting prices are individual rational;<br />
a user reporting her true valuation will not have to pay more than her reported valuation if her request<br />
gets accepted and she will not have pay anything if she does not obtain the resources, and accordingly<br />
for resource owners.<br />
4 RELATED WORK<br />
The study of market mechanisms for Grid and the implementation of running Grid market prototypes<br />
have received significant attention in the past. SPAWN (Waldspurger et al. 1992) implements a market<br />
for computing resources in which each workstation auctions off idle computing time to multiple<br />
applications by means of a Vickrey auction. All resources are allocated to at most one application at a<br />
time regardless of this application’s actual demand which yields low resource utilization. Chun and<br />
Culler (1999) realized a prototpyical market for computing time in which one resource provider sells<br />
computing time to multiple requesters. Resource requesters get allotted computing time proportional<br />
to their share in the total reported valuation across all requesters. The POPCORN market (Regev and<br />
Nisan 1998) is an online market for computing power which implements a Vickrey auction as well as<br />
two double auctions. All of these approaches share two major drawbacks: First, they allow the<br />
specification and trading of computing power only. But requesters require a b<strong>und</strong>le of resources such<br />
as computing power, memory, and bandwidth. On the one hand, the approaches at hand thus lead to<br />
inefficient allocations since requests with the same demand for computing power but different<br />
memory requirements, for instance, are treated the same. On the other hand, requesters are exposed to<br />
the risk of only being able to obtain one leg of the b<strong>und</strong>le of required resources without the other<br />
(“exposure risk”, Schnizler et al. forthcoming). A second limitation of these approaches is that they do<br />
not support reservations of resources in advance which are essential for Quality of Service assertions.<br />
Schnizler et al. (forthcoming) propose a comprehensive model that targets these deficiencies. They<br />
suggest the use of a multi-attribute combinatorial exchange (MACE). Users are allowed to request and<br />
offer arbitrary b<strong>und</strong>les of grid resources and can specify quality attributes on these resources. MACE<br />
implements an exact mechanism. The scheduling problem in this combinatorial setting is NP-hard, the<br />
pricing scheme of MACE yields approximately truthful prices. With truthful prices, strategic users do<br />
not have an incentive to report any valuation other than their true valuation. Due to the NP-hardness of<br />
the problem, the mechanism is adequate for batch applications – the use for interactive applications is<br />
rather limited.<br />
The work of Bapna et al. (forthcoming) is most relevant to the work presented in this paper. In their<br />
model, multiple requesters and providers can trade both computing power and memory for a sequence<br />
of time slots. First, Bapna et al. (forthcoming) introduce an exact mechanism. By introducing fairness<br />
constraints and imposing one common valuation across all resource providers, they structure the<br />
search space in the <strong>und</strong>erlying combinatorial allocation problem as to establish one common, truthful<br />
price per time slot for all accepted requests. Additionally, this introduces a linear ordering across all<br />
jobs and time which reduces the complexity of the allocation problem, which however still remains<br />
1061<br />
63
NP-hard. To mitigate this computational complexity, they thus propose a fast, greedy heuristic at the<br />
expense of both truthfulness and efficiency.<br />
Archer et al. (2003) also design a heuristic on the basis of Mu’alem and Nisan (2002) and Lehman et<br />
al. (2002) which, however, cannot be applied to the Grid OS context since multiple items of resources<br />
are traded, such as x computing cycles. The truthfulness of their mechanism, however, only holds for a<br />
small number of items.<br />
5 CONCLUSIONS AND FUTURE RESEARCH DIRECTIONS<br />
In Section 1, we argued that a network-centric Grid OS coupled with market-based scheduling can<br />
increase efficiency in Grid and cluster environments by adequately allocating all available resources.<br />
This distinguishes network-centric Grid OS from state-of-the-art Grid middleware such as Globus,<br />
gLite, and UNICORE which rely on batch processing of idle resources only. While there are highly<br />
sophisticated market mechanisms available for Grid middleware (e.g. MACE), there are no<br />
mechanisms available for network-centric Grid OS which rely on interactive application processing. In<br />
Section 2 we formalized the market mechanism as a multi-attribute exchange in which resource<br />
owners and consumers can publish their demand and supply. Exact market-based scheduling<br />
mechanisms share one deficiency: they are NP-hard. While this may not be a problem in a cluster<br />
setting with a small number of users, it may prove crucial in an interactive, large-scale Grid OS<br />
environment. We thus proposed a greedy heuristic in Section 3 which performs a fast scheduling while<br />
preserving truthfulness on the request-side and approximating truthfulness on the provisioning-side of<br />
the market. In Section 4, we presented related work on market-based resource allocation in Grids.<br />
In essence, our market mechanism suggests several intriguing research avenues. We intend to<br />
implement a prototype of the heuristic for dynamic scheduling in MOSIX, a state-of-the-art Grid OS<br />
presented in Section 1. Numerical analyses need to be performed to further analyze the heuristic’s<br />
properties with respect to the economic and technical requirements in Grid OS. The approximation of<br />
efficiency needs to be compared to the optimal allocation achieved by exact mechanisms. This ratio<br />
essentially depends on the norm used in the ranking phase of the heuristic. More sophisticated norms<br />
will be developed and analyzed. The run time of the presented scheduling mechanisms needs to be<br />
evaluated to determine the critical number of requests and offers at which an exact mechanism might<br />
become infeasible and to determine the speedup achieved by the heuristic. And finally, the<br />
approximation of truthfulness on the provisioning-side of the market needs to be analyzed. It is<br />
desirable to allow users to specify dependencies between resources, such as substitutability of<br />
computing power for memory. The model in this paper hence needs to be expanded to allow for such<br />
extended bidding logics.<br />
Acknowledgement<br />
This work has been partially f<strong>und</strong>ed by EU IST programme <strong>und</strong>er grant 034286 “SORMA – Self-<br />
Organizing ICT Resource Management”.<br />
References<br />
Ali, A., A. Anjum, J. Bunn, R. Cavanaugh, F. van Lingen, R. McClatchey, M. Atif Mehmood, H.<br />
Newman, C. Steenberg, M. Thomas, I. Willers (2004). Predicting the Resource Requirements of a<br />
Job Submission. Proceedings of Computing for High Energy Physics, Interlaken, Switzerland.<br />
Archer, A., C. Papadimitriou, K. Talwar and E. Tardos (2003). An Approximate Truthful Mechanism<br />
for Combinatorial Auctions with Single Parameter Agents. Proceedings of the 14 th annual ACM-<br />
SIAM symposium on Discrete algorithms, Baltimore, Maryland, pp. 205-214.<br />
Bapna, R., D. Sanjunkta, R. Garfinkel and J. Stallaert (forthcoming). A market design for grid<br />
computing. INFORMS Journal of Computing.<br />
1062<br />
64
Barak, A., A. Shiloh and L. Amar (2005). An Organizational Grid of Federated MOSIX Clusters.<br />
Proceedings of the 5 th IEEE International Symposium on Cluster Computing and the Grid<br />
(CCGrid), Cardiff, Wales.<br />
Berlich, R., M. Kunze and K. Schwarz (2005). Grid Computing in Europe: from research to<br />
deployment. Proceedings of the 2005 Australasian Workshop on Grid computing and e-research,<br />
Newcastle, Australia, 44, pp. 21-27.<br />
Bichler, M, M. Kaukal and A. Segev (1999). Multi-attribute auctions for electronic procurement.<br />
Proceedings of the 1 st IBM IAC Workshop on Internet Based Negotiation, Yorktown Heights, NY.<br />
Buyya, R., D. Abramson and J. Giddy (2000). Nimrod/G: An Architecture for a Resource<br />
Management and Scheduling System in a Global Computational Grid. Proceedings of the HPC<br />
ASIA’2000, the 4 th International Conference on High Performance Computing in Asia-Pacific<br />
Region, Beijing, China.<br />
Chun, B. and D.E. Culler (1999). Market-based proportional resource sharing for clusters. Millenium<br />
Project Research Report. Available at http://www.cs.berkeley.edu/~bnc/papers/market.pdf<br />
(November 16, 2006).<br />
Cirne, W., F. Brasileiro, N. Andrade, R. Santos, A. Andrade, R. Novaes and M. Mowbray (2006).<br />
Labs of the World, Unite! Journal of Grid Computing, 4 (3), pp. 225-246.<br />
Ferrari, C.E. (1994). On Combinatorial Optimization Problems Arising in Computer Systems Design.<br />
Ph.D. Thesis, Technische Universität Berlin.<br />
Foster, I., and C. Kesselman (2004). The Grid: Blueprint for a New Computing Infrastructure. Morgan<br />
Kaufmann, San Francisco, 2 nd edition.<br />
Gorlatch, S., and J. Müller (2005). From Interactive Applications Toward Network-Centric Operation<br />
Systems. Workshop on Network-Centric OS, Brussels.<br />
Lai, K., B.A. Huberman and L. Fine (2004). Tycoon: a Distributed Market-based Resource Allocation<br />
System. Technical Report, Hewlett Packard.<br />
Lehmann, D., L.I. O’Callaghan and Y. Shoham (2002). Truth Revelation in Approximately Efficient<br />
Combinatorial Auctions. Journal of the ACM, 49 (5), pp. 577-602.<br />
Minoli, D. (2004). A Networking Approach to Grid Computing. John Wiley & Sons, Hoboken, New<br />
Jersey.<br />
Mu’alem, A. and N. Nisan (2002). Truthful Approximation Mechanisms for Restricted Combinatorial<br />
Auctions. In AAAI (poster). Also presented at Dagstuhl workshop on Electronic Market Design.<br />
Myerson, R.B. and M.A. Satterthwaite (1983). Efficient mechanisms for bilateral trading. Journal of<br />
Economic Theory, 28, pp. 265-281.<br />
Padala, P. and J.N. Wilson (2003). GridOS: Operating System Services for Grid Architectures. LNCS<br />
2913/2003, pp. 353-362. Springer Verlag, Berlin.<br />
Parkes, D.C., J. Kalagnanam and M. Eso (2002). Achieving budget-balance with vickrey-based<br />
payment schemes in exchanges. IBM Research Report, draft, March 13, 2002.<br />
Regev, O. and N. Nisan (1998). The POPCORN market – an online market for computational<br />
resources. Proceedings of the 1 st international conference on Information and computation<br />
economies, New York, NY, pp. 148-157.<br />
Schnizler, B., D. Neumann, D. Veit and C. Weinhardt (forthcoming). Trading Grid Services – A<br />
Multi-attribute Combinatorial Approach. European Journal of Operational Research.<br />
Sutherland, I.E. (1968). A futures market in computer time. Communications of the ACM, 11 (6), pp.<br />
449-451.<br />
Waldspurger, C., T. Hogg, B.A. Huberman, J.O. Kephart and W.S. Stornetta (1992). Spawn: A<br />
Distributed Computational Economy. IEEE Transactions on Software Engineering, 18 (2), pp. 103-<br />
117.<br />
1063<br />
65
Economically Enhanced MOSIX for Market-based Scheduling in Grid OS<br />
Lior Amar ∗ , Jochen Stößer ∗∗ , Amnon Barak ∗ , Dirk Neumann ∗∗<br />
∗ <strong>Institut</strong>e of Computer Science<br />
The Hebrew University of Jerusalem<br />
Jerusalem, 91904 Israel<br />
{lior,amnon}@cs.huji.ac.il<br />
∗∗ <strong>Institut</strong>e of Information Systems and Management<br />
Universität Karlsruhe (TH)<br />
Englerstr. 14, 76131 Karlsruhe, Germany<br />
{stoesser,neumann}@iism.uni-karlsruhe.de<br />
Abstract<br />
Applying economic principles to grids is deemed promising<br />
to improve the overall value provided by such systems.<br />
By introducing market mechanisms, end users can influence<br />
the allocation of resources by reporting valuations for<br />
these resources. Current market-based schedulers, however,<br />
are static, assume the availability of complete information<br />
about jobs (in particular with respect to processing<br />
times), and do not make use of the flexibility offered by computing<br />
systems. In this paper, we present our progress in implementing<br />
a novel market mechanism for MOSIX, a stateof-the-art<br />
<strong>management</strong> system for computing clusters and<br />
organizational grids. The market mechanism is designed<br />
so as to be able to work in large-scale settings with selfish<br />
agents. Facing incomplete information about jobs’ characteristics,<br />
it dynamically allocates jobs to computing nodes<br />
by leveraging preemption and process migration, two distinct<br />
features offered by the MOSIX system.<br />
1. Introduction<br />
Grid computing denotes a computing paradigm where<br />
computer resources are shared across administrative bo<strong>und</strong>aries<br />
in order to efficiently serve compute-intense applications<br />
at low cost and to accommodate peak loads [6]. So<br />
called “Grid Operating Systems” (Grid-OS) are a revolutionary<br />
approach to grid computing [13]. Grid OS encapsulate<br />
the access to and <strong>management</strong> of grids on the operating<br />
system layer already. Applications can be left unchanged,<br />
leading to ease of use and hence increased acceptance of<br />
grid technologies. Moreover, instead of sharing only idle<br />
resources, grid OSs can be leveraged to efficiently share all<br />
resources.<br />
MOSIX 1 is a state-of-the-art grid OS [1] that makes an<br />
1 MOSIX R○ is a registered trademark of A. Barak and A. Shiloh.<br />
x86 based Linux cluster and an organizational grid perform<br />
like a single computer with multiple processors. A production<br />
organizational grid with 15 MOSIX clusters (∼ 650<br />
CPUs) in different departments is operational at Hebrew<br />
University (see http://www.MOSIX.org/webmon). The features<br />
of MOSIX allow better utilization of grid resources<br />
by users who need to run demanding applications but cannot<br />
afford such a large cluster.<br />
A key issue in inter-organizational grids is the decision<br />
which resource is allocated to whom. Current technical<br />
schedulers are designed to maximize the resource utilization<br />
or to balance the overall system load. In case of excess<br />
demand, such schedulers do not maximize the overall value<br />
of the system since users do not have means to report their<br />
valuations for resources.<br />
Market mechanisms are thus deemed promising to improve<br />
resource allocation in grids – and thus the systems’<br />
value – by explicitly involving users in the allocation process.<br />
Users report their valuations for resources to the system<br />
and are allocated resources and computational tasks<br />
based on these reports. Corresponding monetary transfers<br />
between resource requesters and providers serve so as to induce<br />
users to report truthfully about resource demand and<br />
supply, and to provide incentives to contribute resources to<br />
the system.<br />
Applying market mechanisms to resource allocation in<br />
computing systems is not a new idea [16, 3, 12]. However,<br />
this previous work is not tailored towards the use in<br />
grid OS for three reasons: (1) The Bellagio system [3] and<br />
the OCEAN system [12] are both located above the OS<br />
layer and interoperate with existing grid middleware such<br />
as Globus [5]. (2) From a market mechanism perspective,<br />
the employed matching algorithms are not suited for grid<br />
OS. The combinatorial allocation scheme of Bellagio and<br />
its computationally hard pricing scheme make this mechanism<br />
infeasible for settings with potentially large numbers<br />
of users and where immediacy of the resource allocation is<br />
crucial. The same holds for the negotiation protocol em-<br />
66
ployed in OCEAN which places significant communicational<br />
load on the system. (3) These existing market mechanisms<br />
assume complete information about resource requests<br />
and offers, in particular concerning processing times, and<br />
determine static allocations which do not dynamically adapt<br />
as the status of the system and its entities changes over time.<br />
The contribution of this paper is twofold:<br />
• We propose a market mechanism which is specifically<br />
tailored towards the needs and technical features<br />
of MOSIX so as to generate highly efficient resource<br />
allocations in settings with large numbers of selfish<br />
agents; and<br />
• We present a first implementation of the proposed<br />
market mechanism in MOSIX which serves as a proof<br />
of our concept.<br />
The paper is organized as follows. The next section<br />
presents an overview of MOSIX and its main relevant features.<br />
Section 3 presents the proposed market mechanism<br />
we use to enhance the resource <strong>management</strong> of MOSIX. In<br />
Section 4, we present the design and the pilot implementation<br />
of the MOSIX Economic Infrastructure (MEI). Section<br />
5 concludes the paper and points to future work.<br />
2. MOSIX Backgro<strong>und</strong><br />
MOSIX is a <strong>management</strong> system for clusters and organizational<br />
grids [1]. Its main feature is to make all the participating<br />
nodes perform like a single computer with multiple<br />
processors, almost like an SMP. In a MOSIX system, users<br />
can run applications by creating multiple processes, then let<br />
MOSIX seek resources and automatically migrate processes<br />
among nodes to improve the overall performance, without<br />
changing the run-time environment of migrated processes.<br />
The goal of MOSIX is to allow owners of nodes to share<br />
their computational resources from time to time, while still<br />
preserving the autonomy of the owners, allowing them to<br />
disconnect their nodes from the grid at any time, without<br />
sacrificing guest processes from other nodes.<br />
MOSIX is implemented as a software layer that provides<br />
applications with an unmodified Linux run-time environment.<br />
Therefore, there is no need to change or even link applications<br />
with any special library. Moreover, MOSIX supports<br />
most additional Linux features that are relevant to ordinary,<br />
non-threaded Linux applications, so that most Linux<br />
programs can run unchanged (and be migrated). To submit<br />
processes via MOSIX, users use the program mosrun. For<br />
example, the command “mosrun -m450 myprog myprogargs”,<br />
starts and later may migrate the program my-prog<br />
only to nodes with at least 450 MB of free memory.<br />
The relevant features of MOSIX to the work presented<br />
in this paper are the automatic resource discovery, the process<br />
migration and the freezing mechanisms. The following<br />
subsections provide some detailes about these features.<br />
Other features of MOSIX include a secure run-time environment<br />
(sandbox), that prevents guest processes from accessing<br />
local resources in hosting nodes; “live-queuing”<br />
that preserves the full generic Linux environment of queued<br />
jobs; gradual release of queued jobs, to prevent flooding of<br />
the grid or any cluster; checkpoint and recovery and support<br />
of batch jobs.<br />
2.1. Automatic Resource Discovery<br />
Resource discovery is performed by an on-line information<br />
dissemination algorithm [2], providing each node with<br />
the latest information about the availability and state of grid<br />
resources. The dissemination is based on a randomized gossip<br />
algorithm, in which each node regularly monitors the<br />
state of its resources, including the CPU speed, current load,<br />
free and used memory, etc. This information, along with<br />
similar information that has been recently received by that<br />
node is routinely sent to a randomly chosen node, where a<br />
higher probability is given to choosing target nodes in the<br />
local cluster. The outcome of this scheme is that each node<br />
maintains a local<br />
information-vector with information about all active<br />
nodes in the local cluster and the grid. Any client requiring<br />
information about cluster or grid nodes can simply access<br />
the local node’s information-vector and use the stored<br />
information. Information about newly available resources,<br />
e.g., nodes that have just joined the grid, is gradually disseminated<br />
across the active nodes, while information about<br />
nodes in disconnected clusters is quickly phased out.<br />
2.2. Process Migration and Scheduling<br />
MOSIX supports cluster and grid-wide (preemptive)<br />
process migration [4]. Process migration can be done either<br />
automatically or manually. The migration itself amounts<br />
to copying the memory image of the process and setting<br />
its run-time environment. In MOSIX, the node where the<br />
process was initially started is referred to as the process’s<br />
home-node. After a process is migrated out of its homenode<br />
all the system-calls of that process are forwarded to<br />
and processed in the home-node. The resulting effect is that<br />
there is no need to copy file or libraries to the remote nodes,<br />
and a migrated process can be controlled from its homenode<br />
as if it was running there.<br />
Automatic migration decisions are based on (run-time)<br />
process profiling and the latest information on availability<br />
of grid resources, as provided by the information dissemination<br />
algorithm. Process profiling is performed by continuously<br />
collecting information about its characteristics, e.g.,<br />
size, rates of system-calls, volume of IPC and I/O. This in-<br />
67
formation is then used by competitive on-line algorithms [7]<br />
to determine the best location for the process. These algorithms<br />
take into account the respective speed and current<br />
load of the nodes, the size of the migrated process vs. the<br />
free memory available in different nodes, and the characteristics<br />
of the processes. This way, when the profile of a<br />
process changes or when new resources become available,<br />
the algorithm automatically responds by considering reassignment<br />
of processes to better locations.<br />
2.3. Freezing Support<br />
In a dynamic grid environment the availability of resources<br />
may change over time, e.g., when clusters are connected<br />
or disconnected from the grid. In MOSIX, guest processes<br />
running on nodes that are about to be disconnected<br />
are migrated to other nodes or back to their home-node. In<br />
the last case, the available memory at the home-node may<br />
not be sufficient to receive all returning processes.<br />
To prevent the loss of running processes, MOSIX has a<br />
freezing mechanism that can take any running MOSIX process,<br />
suspend it, and store its memory image in a regular file<br />
(on any accessible filesystem). This mechanism can be activated<br />
automatically when a high load is detected, or manually<br />
upon specific request. The freezing mechanism ensures<br />
that a large number of processes can be handled without exhausting<br />
CPU and memory.<br />
Automatically frozen processes are reactivated in a circular<br />
fashion in order to allow some work to be done without<br />
overloading the owner’s nodes. Later, when more resources<br />
become available, the load-balancing algorithm migrates<br />
running processes away, thus allowing reactivation of<br />
additional frozen processes.<br />
3. A Market Mechanism for MOSIX<br />
In order to enhance the economic value provided by the<br />
system and to provide incentives to contribute resources<br />
to the grid, we implemented a market mechanism in the<br />
MOSIX system. In this section, we will specify our first<br />
approach towards designing such a market mechanism for<br />
MOSIX which accounts for its requirements – in particular<br />
online scheduling without knowing processing times in<br />
advance – while at the same time leveraging its distinct features<br />
such as migration and freezing.<br />
3.1. Requirements for Market-based<br />
Scheduling<br />
Scheduling in MOSIX gives rise to a number of requirements<br />
towards the market mechanism (e.g. [14]). The most<br />
prominent requirements are:<br />
Immediacy: The timing of the allocation is crucial. The<br />
allocation scheme must be scalable in order to be able to allocate<br />
large numbers of resource requests and offers (thousands<br />
of jobs and CPUs)<br />
Allocative efficiency: There shall be no “waste” of resources;<br />
the system is supposed to make optimal use of its<br />
resources and thus maximize the system’s overall which is<br />
returned to its users.<br />
Truthfulness: Participants cannot benefit from cheating<br />
the mechanism, i.e. reporting any valuation for a resource<br />
other than their true valuation.<br />
A market mechanism consists of three elementary components:<br />
a bidding language which specifies what information<br />
is exchanged between the mechanism and its users<br />
(Subsection 3.2), an allocation scheme which determines<br />
the assignment of offered resources to requests (Subsection<br />
3.3), and a pricing scheme which determines corresponding<br />
monetary transfers between the market mechanism<br />
and its participating users (Subsection 3.4).<br />
3.2. The Model<br />
Let J be the set of resource requests. We assume that<br />
request (or “job”) j ∈ J is fully described by the tuple<br />
(v j , c j , m j , s j , l j ). Computing power is the central scarce<br />
resource in the system. Consequently, v j ∈ R + denotes<br />
j’s valuation (or “maximum willingness to pay”) per unit<br />
of computing power and time (e.g. per standard processor<br />
and hour of processing time), c j ∈ N and m j ∈ N are j’s<br />
minimum required amount of computing power (e.g. number<br />
of processors in case of a parallelizable job) and memory<br />
(e.g. in MB) respectively, s j ∈ R + denotes j’s release<br />
time (the point in time at which is reported to the mechanism),<br />
and l j ∈ R + is j’s processing time (or length, e.g. in<br />
hours). However, with processes being located on the operating<br />
system level, users typically do not know the length<br />
of their jobs before these jobs are actually completed. Consequently,<br />
we assume that resource requesters submit the<br />
tuple (v j , c j , m j ) to the mechanism at time s j .<br />
Moreover, let N be the set of resource offers. We assume<br />
that offer (or “node”) n ∈ N is fully described by the<br />
tuple (r n , c n , m n , ɛ n , λ n ) where r n ∈ R + denotes n’s valuation<br />
(or “reservation price”) per unit of computing power<br />
and time, c n ∈ N and m n ∈ N is the maximum available<br />
amount of computing power and memory respectively, and<br />
ɛ n ∈ R + and λ n ∈ R + specify the timeframe during which<br />
these resources are available. For now, we further assume<br />
that nodes can potentially execute multiple jobs in parallel<br />
but that one job can only be executed by one node at a time. 2<br />
Again, we assume that resource providers do not know how<br />
long they can contribute their resources, e.g. because they<br />
2 In future extensions of this basic model we plan to also support parallelism.<br />
68
do not know when these resources will be required by a local<br />
process, and thus submit the tuple (r n , c n , m n ) to the<br />
mechanism at time ɛ n .<br />
This setting is oftentimes referred to as being an online<br />
setting where information is gradually being released to the<br />
mechanism as opposed to the offline setting where all information<br />
is known to the mechanism at the time it makes its<br />
allocation and pricing decisions. Furthermore, we are facing<br />
a so called non-clairvoyant scheduling problem where<br />
processing lengths are unknown [10].<br />
Example: Assume at time t when the market mechanism<br />
is to make its allocation and pricing decisions (“clearing”),<br />
the sample resource requests and offers listed in Table 1<br />
have been collected. Job J1 requires one processor and 200<br />
Job j v j c j m j Node n r n c n m n<br />
J1 1.8 1 200 N1 1.4 1 240<br />
J2 2.2 2 100 N2 1.0 2 200<br />
J3 2.0 1 80<br />
J4 1.0 1 120<br />
Table 1. Sample resource requests and offers<br />
MB of memory. It is willing to pay up to $1.8 per processor<br />
(and thus in total) per time unit. Node N2 is offering two<br />
processors and 240 MB of memory while requiring at least<br />
$1.0 per processor (and thus $2.0 in total) per time unit.<br />
3.3. The Allocation Scheme<br />
The mechanism clears at discrete points in time t ∈ R + .<br />
At time t, the requests J(t) = {j ∈ J | s j ≤ t ≤ s j + l j }<br />
and the offers N(t) = {n ∈ N | ɛ n ≤ t ≤ ɛ n + λ n } have<br />
been collected by the market mechanism. As introduced<br />
in Subsection 3.1, we intend to design and implement an<br />
allocation scheme with the aim of maximizing allocative<br />
efficiency (i.e. welfare), where allocative efficiency is<br />
the sum over all users’ utility resulting from a specific<br />
market outcome (allocations and prices) and may also be<br />
interpreted as overall “happiness”. Let V (t) be the resulting<br />
welfare at time t and V = ∑ t∈R +<br />
V (t) the overall welfare<br />
generated by the mechanism. Since we are facing an online<br />
setting, in order to maximize V , we must maximize V (t)<br />
for all t. This scheduling problem can be formalized as the<br />
following integer program:<br />
max X(t) V (t) = ∑ j∈J(t) c ∑<br />
j n∈N(t) x jnt(v j − r n )<br />
s.t. x jnt ∈ {0, 1}, v j ≥ r n , j ∈ J(t), n ∈ N(t) (1)<br />
∑<br />
n∈N(t) x jnt ≤ 1, j ∈ J(t) (2)<br />
∑<br />
j∈J(t) x jntc j ≤ c n , n ∈ N(t) (3)<br />
∑<br />
j∈J(t) x jntm j ≤ m n , n ∈ N(t) (4)<br />
X(t) gives the (optimal) allocation at time t. Constraint<br />
(1) introduces the binary decision variable x jnt with<br />
x jnt = 1 if job j ∈ J(t) is allocated to node n ∈ N(t) at<br />
time t and x jnt = 0 else. A job can only be allocated to<br />
a node whose reservation price (its minimal required payment)<br />
does not exceed this job’s maximum willingness to<br />
pay. Furthermore, a job can only be allocated to one node at<br />
a time (Constraint (2)). Constraints (3) and (4) enforce that<br />
the resource requirements by all jobs allocated to one specific<br />
node at a time do not exceed the amount of resources<br />
available on this node.<br />
This program is an instance of a Generalized Assignment<br />
Problem and thus NP-hard [9]. But in settings with<br />
large numbers of resource requests and offers, we must still<br />
be able to determine allocations quickly. Heuristics are a<br />
promising approach towards tackling this problem. They<br />
are very speedy while at the same time being simple to <strong>und</strong>erstand,<br />
implement, and extend.<br />
We present the pseudocode for a greedy heuristic in Algorithm<br />
1. We successively construct a feasible alloca-<br />
Algorithm 1 Greedy heuristic<br />
Require: A set J(t) of resource requests and a set N(t) of<br />
resource offers.<br />
Ensure: A feasible allocation X(t) = (x jnt ) of requests<br />
j ∈ J(t) to offers n ∈ N(t).<br />
1: init ∀j ∈ J(t), n ∈ N(t) : x jnt = 0<br />
2: while J(t) ≠ ∅ do<br />
3: Select and remove j ∈ argmax k∈J(t) v k<br />
4: N ′ (t) = ∅<br />
5: while N(t) ≠ ∅ do<br />
6: Select and remove n ∈ argmin k∈N(t) r k<br />
7: if c j ≤ c n and m j ≤ m n and r n ≤ v j then<br />
8: x jnt = 1<br />
9: c n = c n − c j and m n = m n − m j<br />
10: N ′ (t) ← n and break<br />
11: else<br />
12: N ′ (t) ← n<br />
13: end if<br />
14: end while<br />
15: N(t) ← N(t) ∪ N ′ (t)<br />
16: end while<br />
17: return X(t)<br />
tion request-by-request by employing a single-pass priorityrule-based<br />
scheduling heuristic, where the priority of requests<br />
and offers is determined by their reported valuation.<br />
3 By running through the list J(t) of requests in non-<br />
3 Ties may be broken randomly or by considering the memory requirements,<br />
for instance.<br />
69
ascending order of v j and trying to allocate this request to<br />
the cheapest possible offer n ∈ N(t), we greedily maximize<br />
the term v j −r n in the objective function of the integer<br />
program introduced above.<br />
Example: Job J2 has the highest ranking and can only<br />
be allocated to node N2. Consequently, there is only capacity<br />
left on node N1, which is then assigned to job J3 which<br />
is considered next in the job ranking. This allocation generates<br />
welfare amounting to v J2 c J2 + v J3 c J3 − r N2 c N2 −<br />
r N1 c N1 = $2.2 × 2 + $2.0 − $1.0 × 2 − $1.4 = $3 per time<br />
unit.<br />
This heuristic generates a feasible and approximately<br />
efficient allocation at low computational cost. The lists<br />
J(t) and N(t) can be sorted in O(|J(t)| log |J(t)|) and<br />
O(|N(t)| log |N(t)|) respectively. The actual allocation<br />
phase then runs in O(|J(t)||N(t)|).<br />
In our example, the generated welfare happens to correspond<br />
to the optimal welfare produced by solving the<br />
scheduling problem exactly. However, in general the computational<br />
speed of the heuristic will only come at the expense<br />
of allocative efficiency [15]. This is exacerbated by<br />
the fact that we are facing a non-clairvoyant online setting<br />
in which earlier allocation decisions may prove unfortunate<br />
(in terms of allocative efficiency) when new requests and offers<br />
(and hence new information) arrive in the system. We<br />
thus make use of the unique features provided by MOSIX to<br />
improve upon such unfortunate decisions: If the allocation<br />
scheme determines that a specific job j ∈ J(t) is allocated<br />
to some resource n ∈ N(t) at time t (x jnt = 1), in the<br />
next run at time t ′ the allocation scheme may decide to preempt<br />
this currently running job in case a job with a higher<br />
willingness to pay enters the system. This preempted job<br />
may then either be migrated to another node in case the allocation<br />
scheme decides to keep this job in the allocation<br />
(x jmt ′ = 1, m ∈ N(t ′ ), m ≠ n), or it may suspend the<br />
job and send it back to its home-node in case it decides to<br />
not keep the job in the allocation ( ∑ n∈N(t ′ ) x jnt ′) = 0).<br />
The implementation of this logic will be explained in detail<br />
in Section 4.<br />
3.4. The Pricing Scheme<br />
In inter-organizational grid settings, we assume users to<br />
be selfish and rational, meaning both resource requesters<br />
and providers try to maximize their individual benefit from<br />
participating in the system. Resource requesters, for instance,<br />
may benefit from <strong>und</strong>erstating their valuation, thus<br />
potentially lowering their price. We assume that resource<br />
requesters can only misreport about their valuations (ṽ j ≠<br />
v j ) and their release time (˜s j ≥ s j ). Overstating resource<br />
requirements can only increase the risk of not being allocated<br />
or having to pay a higher price. Understating resource<br />
requirements can easily be detected and punished by the<br />
system, e.g. by killing the job. We further assume that resource<br />
providers do not have an incentive to misreport their<br />
available resources. Again, overstating can easily be detected<br />
and punished by the system whereas providers obviously<br />
do not have an incentive to <strong>und</strong>erstate, as this cannot<br />
increase their resulting allocation and thus their payment.<br />
However, they can misreport about their reservation prices<br />
(˜r n ≠ r n ).<br />
Consequently, the users’ actions will generally not work<br />
towards the social optimum and we need to complement the<br />
allocation scheme with an “adequate” pricing scheme. With<br />
truthful prices (in dominant strategies), users cannot benefit<br />
from misreporting about their valuations, independently<br />
of the other resource requests and offers in the system.<br />
Building on the concept of “critical-value-based” prices as<br />
elaborated in [8] and [11], we will first construct such a<br />
scheme for determining truthful prices of resource requests<br />
and will then complement this scheme so as to distribute<br />
the resulting payments to the supply side of the market.<br />
Critical-value-based Pricing of Resource Requests<br />
Losing requests j ∈ J(t) (i.e. requests which have not been<br />
accepted) are not required to pay anything, so we set the<br />
price of these requests at time t to p j (t) = 0. For accepted<br />
requests, we set p j (t) = φ j c j , where φ j (t) ∈ R + denotes<br />
j’s critical value at time t. This critical value equals the<br />
minimum valuation j would have had to report to the system<br />
in order to remain in the allocation X(t).<br />
The strongest plus of critical-value-based pricing is that<br />
it generates truthful prices of resource requests [8, 11, 14].<br />
Truthfulness is a powerful concept. With truthful prices, resource<br />
requesters do not need to reason about other users’<br />
bidding strategies, instead they can “simply” report their<br />
true valuation to the system. They cannot benefit from<br />
trying to cheat the mechanism. The drawback of criticalvalue-based<br />
pricing is the additional computational burden<br />
it adds to the mechanism. The allocation is not independent<br />
of a specific request, meaning the allocation of the other requests<br />
might change in case a winning request is removed<br />
from the allocation. Consequently, in order to determine the<br />
critical value φ j (t) of a specific request j at time t, we need<br />
to determine the allocation without j.<br />
Critical-value-based prices guarantee payments from<br />
resource requesters which at least cover the reservation<br />
prices of resource providers and potentially generate a surplus<br />
exceeding these reservation prices. Intuitively, we will<br />
want to distribute this surplus so as to also induce resource<br />
providers to truthful reporting of their valuations. This,<br />
however, is impossible, as analytically shown in [8] and<br />
[15], and we can thus only aim to produce approximately<br />
truthful payments.<br />
70
Proportional Payments to Resource Providers<br />
Proportional payments distribute the surplus generated by<br />
critical-value-based pricing to resource providers according<br />
to these providers’ contribution of computing power as<br />
compared to the overall contribution. First numerical experiments<br />
indicate that this approach exhibits desirable strategic<br />
properties while only adding marginal computational<br />
burden on the mechanism [15]. More formally, let<br />
S(t) = ∑<br />
p j (t) −<br />
∑ ∑<br />
j∈J(t)<br />
˜r x jntc j<br />
n<br />
c n<br />
j∈J(t)<br />
n∈N(t)<br />
be the surplus at time t, i.e. overall payments by resource<br />
requesters less overall reservation prices. Then resource<br />
provider n receives a payment of<br />
∑<br />
j∈J(t)<br />
p n (t) = ˜r x ∑<br />
jntc j<br />
j∈J(t)<br />
n +S(t)·<br />
x jntc j<br />
∑ ∑<br />
c n<br />
m∈N(t) j∈J(t) x jmtc j<br />
per time unit the current allocation is valid.<br />
Example: Assume that ties between J2 and J1 and between<br />
J3 and J1 are broken in favor of J2 and J3. Then<br />
the winning jobs J2 and J3 both need to outbid J1 in order<br />
to remain in the allocation (φ J2 = φ J3 = $1.8). Consequently,<br />
J2 has to pay 2 · $1.8 = $3.6, J3 has to pay<br />
$1.8, and J1 and J4 do not have to pay anything. Node<br />
N2 receives 2 · $1.0 + 2 3 · $2 ≈ $3.33 while N1 receives<br />
$1.4 + 1 3 · $2 ≈ $2.07.<br />
4. Economically Enhanced MOSIX<br />
In this section we describe the enhancement of MOSIX<br />
to support economy aware scheduling using the market<br />
mechanism presented in the previous section. The MOSIX<br />
features used were the automatic resource discovery, manual<br />
process migration and manual freezing (the automatic<br />
process migration and freezing were disabled). A layer<br />
called the “MOSIX Economy Infrastructure” (MEI) was<br />
built on top of the existing MOSIX system. The MEI is<br />
composed of 3 main components: the provider component,<br />
the client component and the market manager component.<br />
Figure 1 shows a schematic view of those components.<br />
4.1. The Provider Component<br />
In our implementation each provider runs a daemon<br />
called providerd which is responsible for managing the economic<br />
properties of that provider. The economic properties<br />
currently includes: the provider market status, its reservation<br />
price and its current payment, see Figure 1. The market<br />
status may be ON or OFF, where ON indicates the provider<br />
is currently participating in the market. If the market status<br />
is ON, the amount of time the provider is expected to stay<br />
Figure 1. MEI components<br />
ON is also reported. This amount of time is only used as<br />
a hint and the provider may join or leave the market at any<br />
given moment. For example, student farm nodes can join<br />
the market from 22:00 until 8:00 the morning after (during<br />
this time the farm is closed for students). The reservation<br />
price specifies the minimal price the provider is willing to<br />
accept in order to run processes, and the current payment<br />
indicates how much the provider is currently getting payed<br />
(if it is used by a job).<br />
Those new economic properties were added to the information<br />
system of MOSIX and are constantly circulated<br />
among the cluster nodes. This way the market manager (see<br />
below) can easily obtain information about all the available<br />
providers by querying one node in the cluster and obtaining<br />
its information vector. An example of the relevant information<br />
which each provider supply is presented in Figure 2.<br />
A command line tool called provider-ctl enables the<br />
owners to online modify the economic properties of the<br />
provider. For example an owner may set different reservation<br />
prices for different nodes, or can set different time<br />
windows for participating in the market for each node.<br />
4.2. The Client Component<br />
The program erun (see Figure 4) allow the user to specify<br />
economic parameters in addition to the MOSIX standard<br />
parameters. The economic parameters includes the<br />
71
Figure 2. Sample provider XML information<br />
job’s maximal price and its budget. The standard MOSIX<br />
(non-economic) parameters are forwarded to the mosrun<br />
program. For example, the command “erun -p40 -m180<br />
myprog myprog-args“ submits a job with a maximal price<br />
of 40 and a memory requirement of 180 MB.<br />
Once an erun program is launched, it connects to a daemon<br />
called assignd, on the local host, which is responsible<br />
for managing all the economy-aware processes in each<br />
node. The assignd keeps track of all the erun programs and<br />
reporting to the market manager about newborn erun processes<br />
and also about erun processes which finished to run.<br />
Figure 3 shows an example of a message the assignd send<br />
to the market manager when a new job is submitted by the<br />
user.<br />
Figure 3. Sample job XML information<br />
In the figure it can be seen that each job has a unique jobid<br />
which is used to identify the job in the cluster. The user<br />
may specify the maximal amount of memory the job may<br />
need. This property helps the system to avoid assigning<br />
the job to nodes with not enough free memory. The cpunum<br />
property specifies how many CPUs this job requires,<br />
enabling the user to submit parallel jobs which need more<br />
than one CPU to run. The max-pay property stands for the<br />
maximal price (valuation) the user is willing to pay for running<br />
the jobs on a standard CPU for one hour. And finally<br />
the max-budget property let the user assign a limited budget<br />
to this job. Once the job consumes this budget the job will<br />
be suspended until the budget is increased again.<br />
The assignd is also responsible for receiving instructions<br />
regarding the allocation of erun processes from the market<br />
manager. Once such instructions arrive, the assignd uses the<br />
<strong>und</strong>erlying MOSIX system to enforce the market decision.<br />
For example, if the central market instruct the assignd to<br />
move a given erun processes from one provider to another<br />
provider, the assignd uses the manual migration capabilities<br />
of MOSIX to send the erun to the new location. If on<br />
the other hand the assignd is ordered to suspend a process,<br />
then the process is frozen to the local disk using the MOSIX<br />
freezing mechanism.<br />
4.3. The Market Manager Component<br />
The market manager component is responsible for performing<br />
the economic scheduling decisions. It consists of a<br />
daemon called marketd and a market solver program (which<br />
we refer to as the solver). The marketd receives job requests<br />
from client nodes (via the assignd’s) and collects information<br />
about the available providers using the MOSIX automatic<br />
resource discovery. Once there are jobs in the system<br />
and also available providers, the marketd passes this<br />
information to the solver and wait for the solver to output a<br />
scheduling decision. The solver is implemented as shown in<br />
Section 3. Once the schedule is computed, the solver sends<br />
to the marketd a scheduling decision for each job in the system.<br />
The marketd then forwards the scheduling decision of<br />
each job to the assignd responsible for that job (which carry<br />
on the decision). There are two possibilities for running the<br />
solver, the first is periodically and the second is upon arrival<br />
of a new process. Currently, periodic approach is used.<br />
Beside changing the status of the job, the market mechanism<br />
also specifies the payment of the job (in case it in<br />
a running state), this payment is used to update the totalpayment<br />
property of the job. So for each job we always<br />
know how much money it spent until now. Figure 4 shows<br />
two examples of scheduling decisions the solver may output.<br />
The first is for job 12345 which schedules the job to run<br />
on provider 132.65.174.10 with price of 10. In this case, assuming<br />
the job is already running on another provider, the<br />
assingd responsible for the job will migrate the job to the<br />
new provider. The second schedule is for job 1233, but in<br />
this case the job is suspended. Here the assignd responsible<br />
for the job will use the freezing mechanism of MOSIX and<br />
will freeze (suspend) the job in it home-node.<br />
5. Conclusion & Future Work<br />
We have presented an enhancement to the MOSIX system<br />
which performs market-based scheduling instead of the<br />
existing load-balancing-based scheduling. The process migration<br />
and freezing capabilities allow our system to dy-<br />
72
Figure 4. Sample scheduling decisions<br />
namically allocate and reallocate jobs to any provider node,<br />
not only to idle nodes. The market mechanism we propose<br />
is specifically tailored towards the needs and technical features<br />
of grid OS so as to generate highly efficient resource<br />
allocations in large-scale settings. We presented a pilot implementation<br />
of the economic enhancement of MOSIX.<br />
Our work suggests several intriguing avenues for future<br />
research. First, we plan to perform real runs of the system<br />
with thousands of jobs and h<strong>und</strong>reds of providers. We<br />
will research how to keep the scheduling period as short as<br />
possible (several seconds) in order to keep the system responsive.<br />
An interesting issue is supporting short jobs with<br />
run times smaller than the clearing period of the market.<br />
To do that we need to assign the jobs as soon as they arrive.<br />
Here we plane to run a fast version of the market solver each<br />
time a new jobs arrives, while running the full version periodically.<br />
Another important goal is to introduce technical<br />
limitations to the market in order to take into account both<br />
performance and economic issues. For example, in the current<br />
market, the migration cost of a job is not yet taken into<br />
account. This can lead to a scenario where each new schedule<br />
causes a large number of concurrent migrations, which<br />
in turn can overload the network. We plan to enhance the<br />
market by taking the cost of migration into account as well<br />
as limiting the number of concurrent migrations. This will<br />
make the system more stable and usable in practice.<br />
Acknowledgements<br />
We wish to thank Amnon Shiloh for his contribution in<br />
the development of the erun and assignd programs, and for<br />
his valuable ideas. This research was supported in part by<br />
grants from the EU IST programme <strong>und</strong>er grant no. 034286<br />
SORMA – “Self-Organizing ICT Resource Management”.<br />
[3] A. AuYoung, B. Chun, A. Snoeren, and A. Vahdat. Resource<br />
allocation in federated distributed computing infrastructures.<br />
Proceedings of the 1st Workshop on Operating<br />
System and Architectural Support for the On-demand IT InfraStructure,<br />
2004.<br />
[4] A. Barak, A. Shiloh, and L. Amar. An organizational grid of<br />
federated mosix clusters. In 5-th IEEE International Symposium<br />
on Cluster Computing and the Grid (CCGrid05), May<br />
2005.<br />
[5] I. Foster and C. Kesselman. Globus: a Metacomputing Infrastructure<br />
Toolkit. International Journal of High Performance<br />
Computing Applications, 11(2):115, 1997.<br />
[6] I. Foster, C. Kesselman, and S. Tuecke. The Anatomy of<br />
the Grid: Enabling Scalable Virtual Organizations. International<br />
Journal of High Performance Computing Applications,<br />
15(3):200, 2001.<br />
[7] A. Keren and A. Barak. Opportunity cost algorithms for<br />
reduction of i/o and interprocess communication overhead in<br />
a computing cluster. IEEE Tran. Parallel and Dist. Systems,<br />
14(1):39–50, 2003.<br />
[8] D. Lehmann, L. O’Callaghan, and Y. Shoham. Truth revelation<br />
in approximately efficient combinatorial auctions. Journal<br />
of the ACM (JACM), 49(5):577–602, 2002.<br />
[9] S. Martello and P. Toth. Knapsack problems: algorithms<br />
and computer implementations. John Wiley & Sons, Inc.<br />
New York, NY, USA, 1990.<br />
[10] R. Motwani, S. Phillips, and E. Torng. Nonclairvoyant<br />
scheduling. Theoretical Computer Science, 130(1):17–47,<br />
1994.<br />
[11] A. Mualem and N. Nisan. Truthful approximation mechanisms<br />
for restricted combinatorial auctions. Proc. 18th<br />
National Conference on Artificial Intelligence (AAAI-02),<br />
2002.<br />
[12] P. Padala, C. Harrison, N. Pelfort, E. Jansen, M. Frank, and<br />
C. Chokkareddy. OCEAN: the open computation exchange<br />
and arbitration network, a market approach to meta computing.<br />
Parallel and Distributed Computing, 2003. Proceedings.<br />
Second International Symposium on, pages 185–192,<br />
2003.<br />
[13] P. Padala and J. Wilson. GridOS: Operating System Services<br />
for Grid Architectures. Proceedings of International<br />
Conference On HP computing, 2002.<br />
[14] J. Stößer, D. Neumann, and A. Anandasivam. A Truthful<br />
Heuristic for Efficient Scheduling in Network-Centric Grid<br />
OS. Proceedings of the European Conference on Information<br />
Systems (ECIS), 2007, to appear.<br />
[15] J. Stößer, D. Neumann, and C. Weinhardt. Market-based<br />
Pricing in Grids: On Strategic Manipulation and Computational<br />
Cost. Submitted for publication.<br />
[16] I. Sutherland. A futures market in computer time. Communications<br />
of the ACM, 11(6):449–451, 1968.<br />
References<br />
[1] Mosix. http://www.MOSIX.org, 2007.<br />
[2] L. Amar, A. Barak, Z. Drezner, and I. Peer. Gossip algorithms<br />
for maintaining a distributed bulletin board with guaranteed<br />
age properties. Submitted for publication.<br />
73
GreedEx – A Scalable Clearing Mechanism for Utility Computing<br />
Authors:<br />
Jochen Stoesser, Dirk Neumann ∗<br />
<strong>Institut</strong>e of Information Systems and Management (<strong>IISM</strong>)<br />
Universität Karlsruhe (TH)<br />
{stoesser,neumann}@iism.uni-karlsruhe.de<br />
Englerstr. 14, 76131 Karlsruhe, Germany<br />
Corresponding author:<br />
Jochen Stoesser<br />
<strong>Institut</strong>e of Information Systems and Management (<strong>IISM</strong>)<br />
Universität Karlsruhe (TH)<br />
Englerstr. 14, 76131 Karlsruhe, Germany<br />
Email stoesser@iism.uni-karlsruhe.de<br />
Phone +49–721–6088372<br />
Fax +49–721–6088399<br />
∗ The authors have been partially f<strong>und</strong>ed by the EU IST programme <strong>und</strong>er grant 034286 “SORMA –<br />
Self-Organizing ICT Resource Management”.<br />
74
Abstract<br />
Scheduling becomes key in dynamic and heterogeneous utility computing settings.<br />
Market-based scheduling offers to increase efficiency of the resource allocation<br />
and provides incentives to offer computer resources and services. Current<br />
market mechanisms, however, are inefficient and computationally intractable in<br />
large-scale settings.<br />
The contribution of this paper is the proposal as well as analytical and numerical<br />
evaluation of GreedEx, an exchange for clearing utility computing markets,<br />
based on a greedy heuristic, that does achieve a distinct trade-off: GreedEx<br />
obtains fast and near-optimal resource allocations while generating prices that<br />
are truthful on the demand-side and approximately truthful on the supply-side.<br />
Keywords: Market-based Scheduling, Scalable Heuristic, Truthful<br />
75
In current business domains, information technology (IT) departments are forced to cut<br />
down on IT expenses but are at the same time required to provide the business side with<br />
systems that can flexibly be adjusted to new business requirements and processes. This<br />
is reflected in the vision of utility computing where computer resources can be accessed<br />
dynamically in analogy to electricity and water: “Utility computing is the on-demand delivery<br />
of infrastructure, applications, and business processes in a security-rich, shared, scalable, and<br />
standards-based computer environment over the Internet for a fee. Customers will tap into<br />
IT resources – and pay for them – as easily as they now get their electricity or water”<br />
[Rappa, 1]. According to Rappa, utilities are characterized by necessity, reliability, ease of<br />
use, fluctuating utilization patterns, and economies of scale [Rappa, 1].<br />
Computer resources match this profile very well. Rather than being a strategic asset, IT<br />
has become a basic necessity for almost all businesses [Carr, 2]. If IT systems fail, this can<br />
result in unsatisfied customers and severe contractual penalties. Moreover, IT systems must<br />
be easy to use for customers and employees and must adapt and connect to heterogeneous<br />
systems. However, especially small- and medium-size companies will not constantly require<br />
massive amounts of computer resources. Instead, these resources will be required in the<br />
design phase of a new product only or to create daily/monthly/yearly reports, leading to<br />
fairly dynamic and unpredictable (from the provider’s point of view) demand. The validity<br />
of the business model “utility computing” mainly stems from its immense economies of scale.<br />
Instead of having each company and research center to maintain its own costly computing<br />
and data centers, utility computing providers benefit from economies of scale and offer these<br />
resources on-demand. Setting up computing and data centers for the first customer takes<br />
tremendous fixed costs, but serving an incremental customer or request with these existing<br />
resources only requires (comparably) minor efforts. Sun’s One-Dollar-Per-CPU-Hour and<br />
Amazon’s Elastic Compute Cloud are prominent examples for the industry take-up of utility<br />
computing offerings [Sun, 3; Amazon, 4].<br />
Rappa suggests to base pricing in utility computing on metering usage (also coined “paywhat-you-use”<br />
or “pay-as-you-go”), as is the case with classic utilities such as water, telephone<br />
and Internet access [Rappa, 1]. This approach has also been adopted by Sun ($1/CPU-hour)<br />
and Amazon (e.g. $0.20 per GB of data transfer). However, even in this metered model,<br />
prices are temporarily static and do not fully reflect the dynamicity of demand and supply.<br />
Moreover, setting appropriate prices is a complex task for utility computing providers.<br />
The use of market mechanisms has often been suggested in situations in which allocations<br />
of and prices for scarce resources need to be established. The prices being paid reflect the<br />
resources’ scarcity. Consequently, dynamic market prices support on the one hand resource<br />
requesters in coordinating and distributing their demand over time, thus balancing the system<br />
load. On the other hand, utility computing providers have an incentive to offer resources<br />
in return for the payment of the market price. In dynamic, large-scale utility computing settings,<br />
however, the allocation problem of the market is computationally demanding. Market<br />
mechanisms must be solvable quickly. Currently, there is no market mechanism which can<br />
fully satisfy the economic requirements of utility computing markets.<br />
The contribution of this paper is the proposal as well as analytical and numerical evaluation<br />
of GreedEx, an exchange for clearing utility computing markets, based on a greedy<br />
heuristic, that does achieve a distinct trade-off: it obtains fast and near-optimal resource<br />
allocations while generating prices that are truthful on the demand-side and approximately<br />
76
truthful on the supply-side. GreedEx may be used for two purposes:<br />
• Utility computing providers may choose to run a proprietary GreedEx-based market<br />
platform to allocate their scarce computing resources to resource requests in an efficient<br />
manner and to dynamically price these requests.<br />
• A GreedEx-based platform may be run as an intermediary market which aggregates<br />
demand as well as supply across multiple resource requesters and utility computing<br />
providers, thus increasing liquidity and efficiency.<br />
This paper is structured as follows. Section 1 presents related work on market-based<br />
scheduling. In Section 2, we elaborate the heuristic with its allocation and pricing scheme.<br />
The heuristic’s speedup and its efficiency are evaluated numerically in Section 3 before Section<br />
4 concludes the paper and points to promising future research directions.<br />
1 Related Work<br />
There are two main streams of research in market-based allocation of utility resources: mechanisms<br />
which consider the trading of one type of resource only and mechanisms which account<br />
for dependencies between multiple utility resources.<br />
Spawn implements a market for computing resources in which each workstation auctions<br />
off idle computing time to multiple applications by means of a Vickrey auction [Waldspurger<br />
et al., 5]. All resources are allocated to at most one application at a time regardless of<br />
this application’s actual demand which yields low resource utilization. Chun and Culler<br />
present a market for computing time in which one resource provider sells computing time<br />
to multiple requesters [Chun and Culler, 6]. Resource requesters get allotted computing<br />
time proportional to their share in the total reported valuation across all requesters. The<br />
Popcorn market is an online market for computing power which implements a Vickrey<br />
auction as well as two double auctions [Regev and Nisan, 7]. All of these approaches share<br />
two major drawbacks: First, they allow the specification and trading of computing power<br />
only. But requesters require a b<strong>und</strong>le of resources such as computing power, memory, and<br />
bandwidth. On the one hand, the approaches at hand thus lead to inefficient allocations since<br />
requests with the same demand for computing power but different memory requirements, for<br />
instance, are treated the same. On the other hand, requesters are exposed to the risk of only<br />
being able to obtain one leg of the b<strong>und</strong>le of required resources without the other (“exposure<br />
risk”). A second limitation of these approaches is that they do not support reservations of<br />
resources in advance which are essential for Quality of Service assertions [Foster, 8].<br />
Schnizler et al. [Schnizler et al., 9] elaborate a multi-attribute combinatorial exchange<br />
(MACE) which targets these deficiencies. Users are allowed to request and offer arbitrary<br />
b<strong>und</strong>les of computer resources and can specify quality attributes on these resources. MACE<br />
implements an exact mechanism. The scheduling problem in this combinatorial setting is<br />
NP-hard, the pricing scheme of MACE yields approximately truthful prices. With truthful<br />
prices, strategic users do not have an incentive to report any valuation other than their true<br />
valuation. Due to the NP-hardness of the problem, the mechanism is adequate for batch<br />
applications – the use for interactive applications is rather limited. The Bellagio system<br />
77
also implements an exact combinatorial exchange for computing resources [AuYoung et al.,<br />
10]. Its pricing is based on the approximation of the truthful Vickrey-Clarke-Groves prices<br />
proposed by Parkes et al. [Parkes et al., 11]. Being an exact mechanism it shares the<br />
computational drawbacks of MACE. The work of Bapna et al. [Bapna et al., 12] is most<br />
relevant to the work presented in this paper. Multiple requesters and providers can trade<br />
both computing power and memory for a sequence of time slots. First, an exact mechanism is<br />
introduced. By introducing fairness constraints and imposing one common valuation across<br />
all resource providers, the search space in the <strong>und</strong>erlying combinatorial allocation problem<br />
is structured so as to establish one common, truthful price per time slot for all accepted<br />
requests. Additionally, this introduces a linear ordering across all jobs and time which reduces<br />
the complexity of the allocation problem, which however still remains NP-hard. To mitigate<br />
this computational complexity, a fast, greedy heuristic is proposed at the expense of both<br />
truthfulness and efficiency.<br />
However, as shown in Lehmann et al. [Lehmann et al., 13] and Mu’alem and Nisan<br />
[Mu’alem and Nisan, 14], heuristics do not necessarily imply a loss of both truthfulness and<br />
efficiency. In their work, which is essential to this paper, Mu’alem and Nisan establish a<br />
set of necessary and sufficient properties which can be leveraged to indeed design a truthful<br />
heuristic.<br />
2 A Truthful Clearing Heuristic<br />
In this section we elaborate GreedEx, a market-based heuristic which obtains fast and<br />
near-optimal resource allocations while generating prices that are truthful on the demandside<br />
and approximately truthful on the supply-side of the utility computing market. To this<br />
end, we will first introduce the bidding language which defines the content of the messages<br />
being exchanged between the market mechanism and its participants and then we present<br />
the formal model of the <strong>und</strong>erlying allocation problem. Secondly, we will elaborate an allocation<br />
algorithm which achieves a fast approximation of the optimal allocation schedule.<br />
Thirdly, pricing schemes will be introduced which translate these resource allocations into<br />
corresponding monetary transfers.<br />
2.1 The Model<br />
There are two classes of participants in utility computing markets: requesters who would like<br />
to access computer resources such as processing power and memory, and utility computing<br />
providers who offer these resources in return for the market price. We introduce the term<br />
“job” to refer to a computational problem and we will call an atomic b<strong>und</strong>le of resources<br />
on which the job or parts of it can be computed a “node”, e.g. one server within a cluster.<br />
If a job gets allocated to a node, we will call this job and node a “winning” job and node<br />
respectively. The market mechanism is a centralized entity which clears periodically, meaning<br />
it collects resource requests and offers for a period of time before making its allocation and<br />
pricing decisions. Larger clearing periods generally allow for more efficient allocation decisions<br />
due to additional degrees of freedom when optimizing over a large number of requests and<br />
offers [Parkes et al., 11]. Note, however, that the clearing period can be made as small as<br />
78
a couple of seconds in order to support interactive applications [Stoesser et al., 15]. The<br />
market mechanism is a sealed-bid mechanism in the sense that both resource requesters and<br />
providers do not get to know the other users’ requests and offers before market clearing<br />
[Parkes et al., 11].<br />
A resource requester j who would like to submit a job to the utility computing system<br />
reports the job’s characteristics (v j , c j , m j , s j , e j ) to the market mechanism where v j ∈ R + 0<br />
denotes j ’s maximum willingness to pay (i.e. j ’s valuation) per unit of computing power<br />
and time slot, c j ∈ N and m j ∈ N the minimum required amount of computing power and<br />
memory respectively, and s j ∈ N and e j ∈ N specify the job’s estimated runtime. In the<br />
following we will use the terms “resource requester” and “job” interchangeably. There is ongoing<br />
research about using historical data and information about jobs’ size and application<br />
in order to predict resource requirements and runtimes [Degermark et al., 16; Smith et al.,<br />
17]. Based on this research we assume users to be able to report estimates about these<br />
characteristics to the system. We require the market mechanism to make atomic allocations<br />
in the sense that each job can only be executed if there are sufficient resources available in all<br />
requested time slots. Furthermore, jobs can potentially be migrated between several nodes<br />
over time but each job can only be executed on one node at a time.<br />
A utility computing provider n who would like to contribute a node to the utility computing<br />
system reports the node’s characteristics (r n , c n , m n , ɛ n , λ n ) to the market mechanism<br />
where r n ∈ R + 0 specifies this node’s reservation price per unit of computing power and time<br />
slot, c j ∈ N and m j ∈ N the maximum amount of computing power and memory available<br />
on this node, and ɛ n ∈ N and λ n ∈ N the time frame during which the node can be accessed.<br />
Given sufficient resources, we assume that each node is able to virtually execute multiple<br />
jobs in parallel, for instance by using virtualization middleware.<br />
Example: Assume the resource requests and offers listed in Table 1 have been submitted<br />
to the system. Job J1 requests to be run in time slots 1 to 7 and requires a minimum of 54<br />
Table 1: Sample resource requests and offers<br />
Job j v j c j m j s j e j Node n r n c n m n ɛ n λ n<br />
J1 12 54 126 1 7 N1 4 84 71 2 10<br />
J2 4 85 32 3 7 N2 7 100 101 1 9<br />
J3 12 43 43 1 8<br />
J4 16 35 43 2 7<br />
J5 11 47 37 3 7<br />
J6 6 31 19 2 7<br />
units of computing power and 126 units of memory in each time slot. J1 is willing to pay<br />
up to $12 per unit of computing power and time slot, that is $12 * 54 * 7 = $4,536 in total.<br />
Node N1 offers 84 units of computing power and 71 units of memory in time slots 2 to 10<br />
and requires a reservation price of $4 per unit of computing power and time slot.<br />
In real settings and with respect to designing usable systems, human users cannot be<br />
expected to frequently submit requests and offers manually. Instead, software agents may<br />
serve so as to hide the system’s complexity by automatically trading resources based on<br />
79
the current resource consumption of applications and configurable bidding rules which automatically<br />
derive corresponding valuations [MacKie-Mason and Wellman, 18; Neumann et al.<br />
19].<br />
2.2 Greedy Allocation Scheme<br />
Let J be the set of resource requests, N the set of resource offers, and T = {t ∈ N | s j ≤ t ≤<br />
e j , j ∈ J} ∪ {t ∈ N | ɛ n ≤ t ≤ λ n , n ∈ N} the set of time slots across all requests and offers,<br />
i.e. the allocation problem’s time horizon. Then the winner determination problem which<br />
solves the allocation problem exactly can be formalized as the following integer program:<br />
max V = ∑ ∑ ∑<br />
c j x jnt (v j − r n )<br />
X j∈J t∈T n∈N<br />
s.t. x<br />
∑ jnt ∈ {0, 1}, s j ≤ t ≤ e j , ɛ n ≤ t ≤ λ n , v j ≥ r n , j ∈ J, n ∈ N (C1)<br />
x jnt ≤ 1, s j ≤ t ≤ e j , j ∈ J<br />
(C2)<br />
n∈N ∑<br />
x jnt c j ≤ c n , ɛ n ≤ t ≤ λ n , n ∈ N<br />
(C3)<br />
j∈J<br />
∑<br />
x jnt m j ≤ m n , ɛ n ≤ t ≤ λ n , n ∈ N<br />
(C4)<br />
j∈J<br />
e j<br />
∑<br />
∑<br />
u=s j n∈N<br />
x jnu = (e j − s j + 1) ∑<br />
n∈N<br />
x jnt , s j ≤ t ≤ e j , j ∈ J<br />
The objective of this integer program is to maximize welfare V, the total difference between<br />
the requesters’ valuations and the providers’ reservation prices across all time slots. Constraint<br />
(C1) introduces the binary decision variable x and ensures that a job can only be<br />
allocated to a node which is accessible during the right time slots and whose reservation<br />
price does not exceed the job’s willingness to pay. Furthermore, a job can only be allocated<br />
to at most one node at a time (C2). Constraints (C3) and (C4) specify that the jobs allocated<br />
to one node at a time are not allowed to consume more resources than are available on<br />
this node. Constraint (C5) enforces atomicity, i.e. a job is either fully executed or it is not<br />
executed at all.<br />
Example: Figure 1 shows the schedule which results from optimally allocating the sample<br />
requests and offers above.<br />
The allocation problem at hand is an instance of a multi-dimensional Generalized Assignment<br />
Problem [Martello and Toth, 20] and as such clearly NP-hard. Heuristics have the<br />
desirable property of generating suboptimal allocations fast. We define the following greedy<br />
heuristic:<br />
(C5)<br />
1. Sort jobs j ∈ J in non-ascending order of their reported willingness to pay v j . Sort<br />
nodes n ∈ N in non-descending order of their reported reservation prices r n .<br />
2. Starting with job j with the highest ranking (i.e. the highest reported willingness<br />
to pay), allocate j to the nodes n 1 , . . . , n k with the highest ranking (i.e. the lowest<br />
reported reservation prices) which can together accommodate j.<br />
3. Repeat the allocation procedure of Step 2 with the next job in the ranking until there<br />
are no more jobs which can be allocated to the available nodes.<br />
80
Figure 1: Sample allocation schedule<br />
This heuristic truly implements a greedy allocation scheme: it tries to greedily maximize<br />
the term v j − r n in the objective function of the exact allocation problem above.<br />
Example: For the sample requests and offers at hand, the greedy heuristic is illustrated<br />
in Figure 2.<br />
Figure 2: The greedy heuristic applied to the sample requests and offers<br />
The winning jobs and nodes are highlighted. Job J4 can be allocated to node N1 since<br />
N1 offers sufficient resources over all required time slots and its reported reservation price is<br />
less than J1’s reported willingness to pay. In time slot 1, J3 can be allocated to node N1.<br />
However, in time slots 2 to 8, there is not sufficient residual capacity left due to the execution<br />
of J4. So J3 is subsequently allocated to the next available node N2. J1 cannot be executed<br />
at all due to its excessive memory requirements. The heuristic proceeds until the ranked<br />
list of jobs ends and finally happens to generate the same allocation schedule as the exact<br />
mechanism for the example at hand.<br />
2.3 Pricing Schemes<br />
The allocation algorithm of market mechanism intends to achieve some global or social aim, in<br />
this case maximize social welfare. In achieving this goal, it depends on the resource requesters<br />
and providers to report their true valuations and resource characteristics. These participants,<br />
however, are assumed to be rational and self-interested agents trying to maximize their<br />
81
individual benefit which may not per se be aligned with the social aim. Pricing schemes<br />
are introduced and closely linked to the allocation algorithm so as to induce participants to<br />
indeed report truthfully about their valuations and characteristics while maintaining other<br />
economic criteria [Stoesser et al., 15; Schnizler et al., 9]. In this section, we will first introduce<br />
GreedEx’s pricing scheme. The prices of resource requests are determined by critical-valuebased<br />
pricing while the resulting revenue is distributed to the utility computing providers<br />
by proportional pricing. Subsequently, we will briefly outline K-Pricing [Schnizler et al., 9]<br />
as an alternative pricing scheme which we will use in section 4 to numerically analyze and<br />
benchmark both GreedEx as well as its pricing scheme.<br />
2.3.1 Critical-value-based Pricing<br />
The rationale behind the pricing of resource requests is similar to the well-known Vickrey-<br />
Principle: the payment of each winning job amounts to the lowest (total) willingness to pay<br />
it would have needed to report in order to still remain in the allocation, keeping all other<br />
resource requests and offers fixed. This is also called the critical value φ j of job j [Lehmann<br />
et al., 13; Mu’alem and Nisan, 14]. This is illustrated best by looking at the example at<br />
hand.<br />
Example: The critical values and resulting payments for the winning jobs J3, J4, J5 and<br />
J6 are given in Table 2.<br />
Table 2: Critical-value-based prices of the sample resource requests<br />
Job j J3 J4 J5 J6<br />
v j 12 16 11 6<br />
φ j 7 7 7 4<br />
p cv,j $2,408 $1,470 $1,645 $744<br />
Job J3 would have needed to report a willingness to pay of at least $7 – the reservation<br />
price of node N2 – in order to still remain in the allocation schedule. Consequently, φ j = $7<br />
and the resulting critical-value-based price of J3 is p cv,j = (e j − s j + 1)c j φ j = 8 ∗ 43 ∗ $7 =<br />
$2, 408. The prices of J4, J5 and J6 are determined accordingly. Jobs J1 and J2 have not been<br />
allocated and consequently no payment is required. Overall, the pricing scheme generates<br />
revenue of $6,267.<br />
Truthfulness with respect to a job’s resource requirements is straightforward. If a job’s<br />
resource requirements are <strong>und</strong>erstated, the job will not be finished; it has to pay for the used<br />
resources but these are of no value. Overstating a job’s requirements either increases the<br />
job’s payment or the job is not scheduled at all. Regarding the reported willingness to pay,<br />
the main feature of the proposed pricing scheme is its ability to generate truthful prices of<br />
resource requests: It is a (weakly) dominant strategy for resource requesters to report their<br />
true valuations [Stoesser et al., 15]. We assume quasli-linear utility functions. For utility<br />
P<br />
j∈J x jntc j<br />
c n<br />
computing provider n ∈ N, u n (X) = p n (X) − ∑ t∈T r n where X denotes a specific<br />
allocation schedule and p n (X) is provider n’s resulting payment. For resource requester j ∈ J,<br />
u j (X) = v j c j (e j − s j + 1) − p j (X) if j is among the winning jobs and u j (X) = 0 else, where<br />
82
p j (X) denotes j ’s price <strong>und</strong>er allocation schedule X. Then, with truthful critical-value-based<br />
prices p cv,j , j ∈ J,<br />
∀ ṽ j ∈ R + 0<br />
: u j (X) = v j c j (e j − s j + 1) − p cv,j (X) ≥ ṽ j c j (e j − s j + 1) − p cv,j ( ˜X) = u j ( ˜X)<br />
Truthfulness is a strong and desirable feature as it tremendously simplifies the resource<br />
requesters’ strategy space. For the individual participant, there is no need to reason about<br />
the other participants’ strategies. Instead, she can “simply” report her true valuation. Note<br />
that we assume participants to know their true valuations. As suggested above, a policybased<br />
approach may assist users in deriving adequate valuations from the current resource<br />
consumption of applications [MacKie-Mason and Wellman, 18; Neumann et al. 19].<br />
The critical value of a job essentially hinges on the competition of other jobs for the<br />
same resources. If there is no competition, the job is only required to pay the reservation<br />
price. Otherwise, the job at least needs to outbid these competing jobs. Consequently,<br />
critical-value-based pricing implements a desirable dynamicity from the providers’ point of<br />
view. Providers do not need to constantly monitor the demand-side of the market to set<br />
appropriate prices. Instead, they can simply delegate this to GreedEx. If there is sufficient<br />
competition such that critical values are above reservation prices, critical-value-based pricing<br />
of resource requests potentially generates a surplus, i.e. overall payments exceeding the utility<br />
computing providers’ reservation prices.<br />
2.3.2 Proportional Payments to Utility Computing Providers<br />
Like the resource requesters, these providers are assumed to be selfish agents trying to report<br />
to the mechanism so as to maximize their individual benefit. Utility computing providers<br />
clearly do neither have an incentive to <strong>und</strong>erstate nor to overstate their availability of resources;<br />
<strong>und</strong>erstating may reduce the generated revenue while overstating can be monitored<br />
and punished. Unfortunately, the critical-value-based pricing scheme is not applicable to<br />
resource offers; it requires a binary decision in the sense that either all resources of an offer<br />
are allocated or none at all. However, in our model, we allow the partial use of nodes. In<br />
order to at least approximate truthful payments, we thus fall back to payments proportional<br />
to each provider’s contribution of processing power to the allocation schedule. Let<br />
∑<br />
S(X) = ∑ p cv,j (X) − ∑ x<br />
∑ jnt c j<br />
j∈J<br />
r n<br />
c n<br />
j∈J<br />
n∈N t∈T<br />
be this surplus for allocation schedule X. Then provider n will receive a proportional payment<br />
amounting to<br />
∑ ∑ ∑<br />
p prop,n (X) = ∑ c j<br />
x jnt c j<br />
j∈J<br />
t∈T j∈J<br />
r n + S · ∑ ∑ ∑<br />
c n x jmt c<br />
t∈T<br />
j<br />
t∈T j∈J m∈N<br />
i.e. n will receive a share of the surplus according to its share in the total contribution of<br />
processing power across all providers.<br />
Example: In the example at hand, nodes N1 and N2 have reservation prices of $1,756<br />
83
and $3,752 respectively for the allocation schedule. Thus there is a surplus of $6,267 - $1,756<br />
- $3,752 = $759. In total, 975 units of processing power are provided, 439 by N1 and 536 by<br />
N2. Consequently, N1 receives a total payment of $1,756 + 0.45 * $759 = $2,097.55 and N2<br />
receives $3,752 + 0.55 * $759 = $4,169.45.<br />
As stated above, there are further economic design criteria besides truthfulness. The two<br />
most prominent and desirable are budget-balance and individual rationality.<br />
A mechanism is said to be budget-balanced if its pricing scheme does not need to be<br />
subsidized by outside payments, i.e. the payments from<br />
∑<br />
the resource providers cover the<br />
payments made to utility computing providers: ∀X : p cv,j (X) − ∑ p prop,n (X) ≥ 0. The<br />
pricing scheme of GreedEx is specifically designed so as to obtain strongly budget-balanced<br />
payments. Any surplus from the demand-side is distributed to the supply-side of the market.<br />
After settling the market, there are neither payments left, nor does the market need to be<br />
subsidized with outside payments.<br />
Individual rationality is satisfied if no participant can suffer a loss from participating in<br />
the mechanism: ∀X : u j (X) ≥ 0, j ∈ J, and u n (X) ≥ 0, n ∈ N. GreedEx (respectively<br />
participation) is individually rational. No resource requester needs to pay anything in case<br />
her request is not accepted, nor does she have to pay more than her reported willingness to<br />
pay in case her request is served. Each utility computing provider is guaranteed to receive<br />
at least her reported reservation price for her contributed resources.<br />
Budget-balance and individual rationality are hard feasibility constraints and must be<br />
satisfied in order for the market to be sustainable.<br />
2.3.3 K-Pricing<br />
K-Pricing is introduced in [Schnizler et al., 9]. The basic idea is to distribute the welfare<br />
generated by the allocation algorithm between resource requesters and utility computing<br />
providers according to a factor k ∈ [0, 1]. For instance, assume an allocation of resources<br />
from a specific provider to a specific requester. The buyer values these resources at $10 while<br />
the provider has a reservation price of $5. Then the (local) welfare is $10 - $5 = $5 and k *<br />
$5 of the surplus is allotted to the requester – thus having to pay $10 - k * $5 – and (1 - k)<br />
* $5 is allotted to the provider – thus receiving $5 + (1 - k) * $5.<br />
Besides allowing fairness considerations, the main advantage of K-Pricing is that it can be<br />
determined in polynomial runtime. On the downside, however, it only yields approximately<br />
truthful prices and payments on both sides of the market. In contrast to critical-value-based<br />
pricing, K-Pricing can be applied to both GreedEx and the exact mechanism and will thus<br />
be used for benchmark purposes in the following numerical experiment.<br />
j∈J<br />
n∈N<br />
3 Numerical Experiment<br />
Heuristics inherently imply a trade-off between computational speed and allocative efficiency.<br />
A theoretical analysis of the heuristic only allows examining its general behavior. Thus,<br />
computational speed and efficiency can only be evaluated properly by means of numerical<br />
simulations. In this section, we will shortly introduce the simulation setting which we chose<br />
84
to benchmark GreedEx against the exact mechanism before we will subsequently present<br />
and discuss the results of this simulation.<br />
3.1 Data Generation<br />
With numerical analysis, there are two general possibilities to come up with data sets upon<br />
which the heuristic can be tested [Böttcher et al., 21; Feitelson, 22]. Practical cases, e.g.<br />
derived from real workload traces of distributed systems, are of high practical relevance.<br />
However, it is hard to obtain complete traces about all job and node characteristics and to<br />
generalize from one workload trace to others. More importantly, to the best of our knowledge,<br />
there is currently no trace available containing the reported valuations of users. We<br />
consequently chose the second option, which is the creation of artificial instances. Artificial<br />
data sets yield the benefit that they can be constructed so as to specifically investigate certain<br />
properties of the heuristic.<br />
With our heuristic, reflecting the bidding language introduced in Subsection 2.1, there is<br />
a set of parameters upon which data sets need to be generated:<br />
Parameter<br />
Table 3: Simulation settings<br />
Number of orders per side 10, 20, . . . , 170<br />
Domain<br />
Processing power and memory requested Lognormal LogN(4, 0.28)<br />
Processing power and memory offered Lognormal LogN(4.5, 0.16)<br />
Start and end time of requests Normal N(2, 0.5) and N(6, 0.5)<br />
Start and end time of offers Normal N(2, 0.5) and N(8, 0.5)<br />
Willingness to pay Uniform [0, 20]<br />
Reservation price Uniform [0, 10]<br />
We call a specific combination of such parameter values a “setting”. We chose to fix the<br />
number of requests and offers within one setting but to increase this number across settings<br />
from 10 orders per side to 20, 30 and so forth up to 170 requests and 170 offers. As will be<br />
seen in our results, we fo<strong>und</strong> that the exact mechanism cannot be run within acceptable time<br />
with more than 170 orders per side upwards. Consequently, we defined 17 settings in total.<br />
According to Feitelson [Feitelson, 22], the lognormal distribution generates a good fit to<br />
characteristics in real workloads, such as file size. It represents a normal distribution in log<br />
space, is modal and positively skewed. The distributions specified in Table 3 have been kept<br />
fixed across all 17 settings. The same applies to time constraints and valuations, for which<br />
we chose normal and uniform distributions respectively.<br />
For each setting, we generated 30 order books with the corresponding number of requests<br />
and offers, meaning we ran each setting 30 times in order to mitigate stochastic outliers. We<br />
then fed both the heuristic as well as the exact mechanism with identical order books. The<br />
simulations were run using jCase on an Intel Pentium Xeon computer with 3.2 GHz and 2<br />
GB memory. jCase is a Java-based simulation environment for multi-attribute combinatorial<br />
85
environments. The exact formulations of the allocation problem were solved using CPLEX<br />
9.1, a state-of-the art commercial solver for mathematical optimization problems. The results<br />
presented below represent the averages across all 30 runs of each setting.<br />
In order to be able to run the simulation within acceptable time, we decided to stop the<br />
CPLEX solver after 300 seconds (dark red line) and after 1800 seconds (blue line). CPLEX<br />
then returned the best solution fo<strong>und</strong> so far. Since the results for the large settings do not<br />
always match with the optimum for large settings, we call these runs “Anytime[300s]” and<br />
“Anytime[1800s]” in the spirit of Sandholm et al. [Sandholm et al., 23] and Schnizler et al.<br />
[Schnizler et al., 9]. We chose K-Pricing as pricing scheme for the exact mechanism. Recall<br />
that critical-value-based pricing cannot be applied to the exact mechanism while the heuristic<br />
can be combined with both pricing schemes.<br />
3.2 Data Analysis<br />
In this subsection we will present the results of the experiment with respect to computational<br />
tractability and allocative efficiency.<br />
3.2.1 Computational Tractability<br />
GreedEx is designed so as to perform fast allocations. It runs in polynomial time of the<br />
number of resource requests J and offers N : In the first phase, requests and nodes are sorted<br />
in O(|J|log|J|) and O(|N|log|N|) respectively. In the second phase, jobs are allocated to<br />
nodes in O(|J||N|). While the allocation algorithm can thus be run very fast, critical-valuebased<br />
pricing adds computational complexity to GreedEx. The allocation schedule is not<br />
independent of a specific job j in the sense that the allocation of other jobs might change if<br />
j is not being considered. Thus, for every winning job j, in order to determine j ’s critical<br />
value, the allocation without j needs to be determined. This is similar to the drawback of<br />
the Vickrey-Clarke-Groves mechanism [Parkes et al., 11; Schnizler et al., 9]: The allocation<br />
needs to be computed n+1 times if n is the number of winning jobs. However, in contrast to<br />
the Vickrey-Clarke-Groves mechanism, this allocation can now be computed in polynomial<br />
time.<br />
The average runtime µ CT and the coefficient of variation CV CT = σ CT /µ CT (where σ CT is<br />
the standard deviation) for each setting and combination of allocation algorithm and pricing<br />
scheme is listed in Table 5 in the Appendix.<br />
In Figure 3, a linear scale was used. Note that the runtime of Anytime[300s] and Anytime[1800s]<br />
grows quickly in the number of orders per side. Especially the runtime of Anytime[1800s]<br />
shows an exponential growth and takes almost 22.3 minutes on average already<br />
with 170 orders per side. The curves of the anytime versions begin to flatten for large order<br />
books due to the predefined time limit, i.e. Anytime[1800s] and Anytime[300s] will converge<br />
asymptotically to an (average) runtime of 1800s and 300s respectively. In contrast,<br />
GreedEx clears very fast. It takes less than 800 milliseconds for 170 orders per side.<br />
Figure 4 shows the same data in milliseconds on a logarithmic scale in order to illustrate<br />
the runtime behavior of GreedEx in more detail. Moreover, this graph shows the computational<br />
cost of the critical-value-based pricing scheme compared to K-Pricing. Even though<br />
the allocation needs to be determined multiple times in order to compute the critical values of<br />
86
Figure 3: Runtime behavior in seconds on linear scale<br />
the winning jobs, GreedEx with critical-value-based pricing still only takes 763 milliseconds<br />
on average to clear order books with 170 orders per side and thus exhibits a good-natured<br />
linear growth in runtime. The speed of GreedEx allocation algorithm is showcased by<br />
GreedEx with K-Pricing. While Anytime[1800s] with K-Pricing takes about 22.3 minutes<br />
on average to clear order books with 170 orders per side, GreedEx with K-Pricing only<br />
takes 39 milliseconds.<br />
3.2.2 Allocative Efficiency<br />
The typical approach when analyzing the allocative efficiency of a heuristic is to benchmark<br />
it against the optimal welfare generated by an exact mechanism. Heuristics generally only<br />
produce suboptimal allocations. Furthermore, for most heuristics the ratio of the generated<br />
suboptimal allocation and the optimal allocation can be made arbitrarily small for special<br />
cases. This also holds for GreedEx. Such special cases, however, are mostly only of theoretical<br />
value since they do not reflect realistic workloads. We would consequently like to<br />
compare the efficiency achieved by the exact mechanism with the efficiency generated by<br />
GreedEx by means of numerical experiments. Unfortunately, as the analysis above illustrates,<br />
the runtime of the exact mechanism is prohibitively high. Anytime[1800s], however, is<br />
an accurate proxy for the optimal solution when benchmarking GreedEx. Table 4 indicates<br />
how many times the CPLEX solver returns a suboptimal solution (i.e. it hits its predefined<br />
time limit) dependant on the order book size:<br />
The results of Anytime[1800s] with order books with less than 130 orders per side cor-<br />
87
Figure 4: Runtime behavior in milliseconds on logarithmic scale<br />
Table 4: Suboptimal solutions<br />
Number of orders per<br />
side<br />
Number of runs with<br />
suboptimal solutions<br />
10 . . . 120 130 140 150 160 170<br />
0 . . . 0 1 1 7 7 14<br />
88
espond to the optimum. For larger order books, Anytime[1800s] always finds a feasible<br />
solution. Even though it returns an increasing number of suboptimal solutions, it still finds<br />
16 optimal solutions for the largest order book size while the suboptimal solutions will be<br />
fairly close to the optimum.<br />
Comparing the welfare generated by the anytime versions of the exact mechanism with the<br />
welfare generated by GreedEx yields the results shown in Figure 5. The average welfare µ W V<br />
and the coefficient of variation CV W V = σ W V /µ W V (where σ W V is the standard deviation)<br />
for each setting and combination of allocation algorithm is listed in Table 6 in the Appendix.<br />
GreedEx achieves a surprisingly high efficiency ratio. While it shows some minor spikes<br />
with small order books, efficiency successively levels off at aro<strong>und</strong> 95%. We conjecture that<br />
the spikes are settled due to the additional degrees of freedom the heuristic obtains with<br />
larger order books when making its allocation decisions. The efficiency of Anytime[300s]<br />
drastically deteriorates with order books of 100-110 orders per side upwards. The explanation<br />
is straightforward looking at Figure 4: With these order book sizes, Anytime[300s] begins to<br />
hit its predefined time limit. Consequently, the returned solutions are increasingly distant<br />
to the optimum or even worse, Anytime[300s] does not even find feasible allocations within<br />
5 minutes.<br />
Figure 5: Approximation of efficiency compared to the anytime versions of the exact mechanism<br />
3.3 Implications for Utility Computing<br />
While the results of this experiment must not be generalized overhastily, they are promising<br />
and clearly strengthen the case for heuristics such as GreedEx and related approaches.<br />
In real utility computing settings, the markets will be much larger than the sizes used<br />
in this experiment. Thus the results illustrate that exact mechanisms and even anytime<br />
89
approximations based on exact mechanisms are not appropriate for real utility computing<br />
markets but can merely be used as a baseline and reference point. The mere computational<br />
infeasibility of exact mechanisms even becomes exacerbated in interactive settings where<br />
immediacy of the allocation is the key requirement [Stoesser et al., 15].<br />
Designing market mechanisms in this domain is a complex task. The trade-off between<br />
computational speed and allocative efficiency, which is already challenging in itself, is aggravated<br />
by the introduction of economic design criteria such as truthfulness, budget-balance,<br />
and individual rationality, and the interdependencies between these criteria.<br />
The surprisingly high allocative efficiency of GreedEx in this experiment illustrates that<br />
we are on the right track in this regard. A promising trade-off was achieved while sacrificing<br />
only a minor fraction of efficiency. Market mechanisms may thus be a feasible means to allocate<br />
utility computing resources in an efficient manner while allowing fully dynamic pricing<br />
and the aggregation of demand and supply across multiple resource requesters and utility<br />
computing providers.<br />
4 Conclusion and Outlook<br />
In the introduction to this paper, we motivated the use of utility computing as a promising<br />
concept to increase the flexibility of IT systems while at the same time cutting down on IT<br />
expenses. Utility computing markets may serve so as to aggregate demand and supply of<br />
computer resources and to ease the discovery of appropriate utility computing providers. In<br />
inter-organizational and dynamic settings with a large number of both resource requesters<br />
and providers, however, the clearing of such markets becomes computationally demanding.<br />
The current mechanisms presented in Section 1 are not fully able to cope with this setting. To<br />
this end, in Section 2 we elaborated GreedEx consisting of a greedy allocation algorithm<br />
and a pricing scheme. This market mechanism can be used to perform this clearing in a<br />
scalable manner while at the same time satisfying basic economic design criteria. Besides its<br />
computational speed, the main characteristic of GreedEx is its ability to account for the<br />
inter-organizational nature of utility computing by generating truthful prices for resource<br />
requests and approximately truthful payments to utility computing providers. In general,<br />
however, the computational speed of heuristic comes at the expense of efficiency. Consequently,<br />
in Section 3 we reported the results of a numerical simulation which illustrated<br />
GreedEx’s speed-up as well as its surprisingly high efficiency.<br />
These results strengthen our approach and point to several interesting avenues for future<br />
research. These can be distinguished into two categories: Firstly, we will investigate whether<br />
we can further improve GreedEx’s approximation of efficiency by means of alternative<br />
norms in the sorting phase, look-ahead extensions, and/or randomization. However, this<br />
is a tricky thing to do as the impact on the economic criteria needs to be considered as<br />
well. Secondly, truthful payments to utility computing providers can only be approximated.<br />
We intend to perform an in-depth analysis of alternative pricing schemes and their strategic<br />
impacts.<br />
90
References<br />
[1] Rappa, M.A. (2004). “The utility business model and the future of computing services,”<br />
IBM Systems Journal 43(1), 32–42.<br />
[2] Carr, N.G. (2003). “IT Doesn’t Matter,” Harvard Business Review 811(5), 41–53.<br />
[3] Amazon.com (2007). http://www.amazon.com, visited March 2, 2007.<br />
[4] Sun Microsystems (2007). http://www.sun.com/service/sungrid/overview.jsp, visited<br />
March 2, 2007.<br />
[5] Waldspurger, C., T. Hogg, B.A. Huberman, J.O. Kephart, and W.S. Stornetta (1992).<br />
“Spawn: A Distributed Computational Economy,” IEEE Transactions on Software<br />
Engineering 18(2), 103–117.<br />
[6] Chun, B., and D.E. Culler (1999). “Market-based proportional resource sharing for<br />
clusters,” Millenium Project Research Report.<br />
[7] Regev, O., and N. Nisan (1998). “The POPCORN Market – An Online Market for Computational<br />
Resources,” Proceedings of the 1 st International Conference on Information<br />
and Computation Economies, New York, USA, 148–157.<br />
[8] Foster, I. (2005). “Globus Toolkit Version 4: Software for Service-Oriented Systems,”<br />
Proceedings of the IFIP International Conference on Network and Parallel Computing,<br />
Beijing, China, 2–13.<br />
[9] Schnizler, B., D. Neumann, D. Veit, and C. Weinhardt (2006). “Trading Grid Services –<br />
A Multi-attribute Combinatorial Approach,” European Journal of Operational Research,<br />
forthcoming.<br />
[10] AuYoung, A., B.N. Chun, A.C. Snoeren, and A. Vahdat (2004). “Resource Allocation<br />
in Federated Distributed Computing Infrastructures,” Proceedings of the 1 st Workshop<br />
on Operating System and Architectural Support for the On-demand IT Infrastructure,<br />
Boston, MA, USA.<br />
[11] Parkes, D.C., J. Kalagnanam, and M. Eso (2001). “Achieving Budget-Balance with<br />
Vickrey-Based Payment Schemes in Combinatorial Exchanges,” IBM Research Report<br />
RC 22218 W0110-065.<br />
[12] Bapna, R., S. Das, R. Garfinkel, and J. Stallaert (2006). “A Market Design for Grid<br />
Computing,” INFORMS Journal of Computing, forthcoming.<br />
[13] Lehmann, D., L.I. O’Callaghan, and Y. Shoham (2002). “Truth Revelation in Approximately<br />
Efficient Combinatorial Auctions,” Journal of the ACM 49(5), 577–602.<br />
[14] Mu’alem, A., and N. Nisan (2002). “Truthful Approximation Mechanisms for Restricted<br />
Combinatorial Auctions,” Proceedings of the 18 th National Conference on Artificial<br />
Intelligence (AAAI-02), Edmonton, Alberta, Canada.<br />
91
[15] Stoesser, J., D. Neumann, and A. Anandasivam (2007). “A Truthful Heuristic for Efficient<br />
Scheduling in Network-Centric Grid OS,” Proceedings of the 15 th European Conference<br />
on Information Systems (ECIS), St. Gallen, Switzerland.<br />
[16] Degermark, M., T. Kohler, S. Pink, and O. Schelen (1997). “Advance Reservations<br />
for Predictive Service in the Internet,” ACM/Springer Verlag Journal on Multimedia<br />
Systems 5(3), 177–186.<br />
[17] Smith, W., I. Foster, and V. Taylor (1998). “Predicting Application Run Times Using<br />
Historical Information,” Proceedings of the IPPS/SPDP 1998 Workshop on Job<br />
Scheduling Strategies for Parallel Processing, Orlando, FL, USA.<br />
[18] MacKie-Mason, J.K., and M.P. Wellman (2006). Automated Markets and Trading<br />
Agents,” In: Tesfatsion, L., and K.L. Judd (eds.), Handbook of Computational Economics,<br />
vol. 2: Agent-Based Computational Economics, North-Holland.<br />
[19] Neumann, D., S. Lamparter, and B. Schnizler (2006). “Automated Bidding for Trading<br />
Grid Services,” Proceedings of the 14 th European Conference on Information Systems<br />
(ECIS), Gothenburg, Sweden.<br />
[20] Martello, S., and P. Toth (1990). “Knapsack problems,” John Wiley, Chichester, England.<br />
[21] Böttcher, J., A. Drexl, R. Kolisch, and F. Salewski (1999). “Project Scheduling <strong>und</strong>er<br />
Partially Renewable Resource Constraints,” Management Science 45(4), 543–559.<br />
[22] Feitelson, D.G. (2002). “Workload Modeling for Performance Evaluation,” Performance<br />
Evaluation of Complex Systems: Techniques and Tools, Lecture Notes in Computer<br />
Science 2459, Springer Verlag, 114–141.<br />
[23] Sandholm, T., S. Suri, A. Gilpin, and D. Levine (2001). “CABOB: A Fast Optimal<br />
Algorithm for Combinatorial Auctions,” Proceedings of the 17 th International Joint<br />
Conference on Artificial Intelligence, Seattle, WA, USA.<br />
92
Appendix – Numerical Results<br />
Table 5: Numerical results w.r.t. computational tractability (in milliseconds)<br />
Orders per side<br />
Anytime[1800s]<br />
Anytime[300s]<br />
GreedEx – GreedEx –<br />
Critical value K-Pricing<br />
µ CT CV CT µ CT CV CT µ CT CV CT µ CT CV CT<br />
10 42.40 1.02 52.77 0.88 5.77 1.49 4.23 2.34<br />
20 193.30 0.54 211.47 0.52 5.20 1.95 1.07 3.74<br />
30 960.43 1.09 1,254.73 1.39 7.23 1.07 3.60 1.81<br />
40 2,070.30 0.68 2,790.67 1.09 14.10 0.33 5.73 1.80<br />
50 6,000.70 0.90 6,498.37 1.11 27.20 0.42 9.37 1.11<br />
60 11,431.73 0.69 14,047.73 0.64 44.23 0.26 14.53 1.00<br />
70 28,573.00 1.19 36,452.57 1.45 59.33 0.20 13.50 1.22<br />
80 37,959.37 0.58 44,692.70 0.73 85.90 0.22 12.20 0.80<br />
90 57,150.47 0.49 56,592.63 0.68 131.87 0.20 12.47 0.50<br />
100 106,325.63 0.57 111,070.87 0.59 170.97 0.22 22.40 0.84<br />
110 141,735.97 1.02 112,234.80 0.49 216.17 0.14 21.30 0.69<br />
120 199,872.83 0.66 171,713.60 0.44 280.77 0.15 31.27 0.74<br />
130 503,131.70 0.95 229,594.20 0.34 340.70 0.14 25.60 0.37<br />
140 502,005.73 0.83 267,041.10 0.21 452.57 0.11 42.17 0.62<br />
150 991,292.77 0.58 292,886.40 0.10 538.60 0.14 33.77 0.36<br />
160 1,200,359.93 0.44 300,532.90 0.03 635.93 0.12 33.33 0.16<br />
170 1,334,458.33 0.40 302,720.20 0.00 762.90 0.14 39.07 0.34<br />
93
Table 6: Numerical results w.r.t. allocative efficiency<br />
Orders per side<br />
Anytime[1800s] Anytime[300s] GreedEx<br />
µ W V CV W V µ W V CV W V µ W V CV W V<br />
10 23,372.53 0.31 23,377.97 0.31 22,403.13 0.32<br />
20 46,669.43 0.23 46,637.00 0.23 43,209.23 0.23<br />
30 76,111.47 0.20 76,116.83 0.20 71,035.03 0.20<br />
40 102,620.77 0.14 102,572.77 0.15 94,277.03 0.16<br />
50 134,355.93 0.13 134,494.70 0.14 124,980.30 0.14<br />
60 161,050.57 0.09 161,076.40 0.09 151,352.00 0.10<br />
70 190,288.40 0.12 190,329.83 0.12 178,100.40 0.12<br />
80 210,268.83 0.13 210,307.70 0.13 198,849.70 0.13<br />
90 245,961.03 0.09 245,829.93 0.09 229,105.50 0.09<br />
100 268,946.90 0.09 268,940.60 0.09 251,325.97 0.09<br />
110 302,435.27 0.07 302,229.43 0.07 284,234.43 0.07<br />
120 332,844.77 0.10 331,924.17 0.10 314,384.40 0.10<br />
130 354,773.97 0.09 350,776.97 0.09 333,256.77 0.10<br />
140 394,100.43 0.09 389,306.87 0.09 371,391.67 0.10<br />
150 415,811.30 0.08 391,304.48 0.14 395,062.93 0.07<br />
160 443,498.97 0.09 376,813.14 0.23 420,429.00 0.09<br />
170 464,621.13 0.06 390,701.57 0.21 443,740.97 0.06<br />
Author Biographies<br />
Jochen Stoesser finished his studies in Information Engineering and Management at Universität<br />
Karlsruhe (TH) with a degree as Diplom-Informationswirt in 2006. During his studies<br />
he spent two terms at Carleton University in Ottawa, Canada, and one term at The University<br />
of New South Wales in Sydney, Australia. He joined the <strong>Institut</strong>e of Information<br />
Systems and Management at Universität Karlsruhe (TH) in September 2006 as Ph.D. student<br />
and research assistant. Jochen’s main interest is in the design of market mechanisms<br />
for an efficient allocation of computer resources.<br />
Dirk Neumann received a degree as Diplom-Ökonom (economics) from Justus-Liebig University<br />
in Giessen and a Master of Arts degree in Economics from University of Wisconsin,<br />
Milwaukee. He received his doctoral degree from Universität Karlsruhe (TH) in 2004. Since<br />
then, he has been working as an assistant professor at the <strong>Institut</strong>e of Information Systems<br />
and Management at Universität Karlsruhe (TH). Dirk’s research interests basically cover two<br />
areas. The first topic is concerned with market engineering. In essence, market engineering<br />
denotes the theoretically fo<strong>und</strong>ed design of electronic markets. The second topic is concerned<br />
with the <strong>management</strong> of self-organizing networks. In self-organizing networks it is extremely<br />
difficult to influence the entire network or cliques, as those networks adapt quickly to the<br />
new situation in unforeseen directions.<br />
94
Market-Based Pricing in Grids: On Strategic Manipulation and<br />
Computational Cost<br />
Jochen Stößer, Dirk Neumann, Christof Weinhardt<br />
<strong>Institut</strong>e of Information Systems and Management (<strong>IISM</strong>)<br />
Universität Karlsruhe (TH)<br />
Englerstr. 14, 76131 Karlsruhe, Germany<br />
lastname@iism.uni-karlsruhe.de<br />
Abstract<br />
In Grid computing, computer resources are shared across administrative bo<strong>und</strong>aries. In this<br />
dynamic and inter-organizational setting, scheduling becomes a key challenge. Market<br />
mechanisms are deemed promising to revolutionize resource allocation in Grids. Current market<br />
mechanisms, however, are not adequately able to account for dependencies between multiple<br />
Grid resources in large-scale settings with strategic users who try to benefit from manipulating<br />
the mechanism. In this paper, we design a market-based heuristic for clearing large-scale Grid<br />
settings. At the core of this paper, we present alternative pricing schemes which may<br />
complement this heuristic so as to induce resource requesters and providers to report their true<br />
valuations and resource characteristics to the system. We provide an in-depth analysis of these<br />
pricing schemes and outline basic design options of how the mechanism can be integrated into<br />
current Grid schedulers.<br />
Key words: Distributed Systems, Market-Based Scheduling, Heuristic, Pricing<br />
Introduction<br />
IT departments in business and academia are facing a challenging dilemma. On the<br />
one hand, they need to support increasingly complex applications which require massive<br />
amounts of computer resources, such as ERP systems, engineering applications, or<br />
demand and weather forecasts. On the other hand, IT departments are <strong>und</strong>er constant<br />
pressure to cut down on hard- and software expenses. In order to be able to<br />
95
accommodate peak loads, they must maintain large computing clusters which lie idle<br />
most of the time, thus incurring tremendous costs. Results of a meta-study show that<br />
corporate data centers are only using 10% to 35% of their available computing power<br />
and that 50% to 60% of companies’ data storage capacity is being wasted [Carr, 2005].<br />
Grid computing is a promising way out of this dilemma by allowing organizations to<br />
respond quickly to peaks in demand for computer resources. It denotes a computing<br />
paradigm where computer resources are shared across administrative bo<strong>und</strong>aries, e.g.<br />
between enterprises and/or scientific computing centers or among business units and<br />
departments within one enterprise [Foster et al., 2001]. A recent study conducted by The<br />
Insight Research Corporation projects a growth in worldwide Grid spending from US$<br />
1.84 Billion in 2006 to US$ 24.52 Billion in 2011 [Anonymous, 2006]. Current scientific<br />
Grids such as D-Grid (www.d-grid.org) and EGEE (www.eu-egee.org) or enterprise<br />
Grids employed at GlaxoSmithKline [Anonymous, 2007a] and Sal. Oppenheim<br />
[Anonymous, 2007b], however, suffer from two major drawbacks: Firstly, resources are<br />
shared free of charge and users with idle resources do not have an incentive to<br />
contribute these resources to the Grid. Within enterprises, for example, business units<br />
are oftentimes unwilling to share idle resources with other units as they do not receive<br />
monetary compensation or future access to Grid resources in return. Secondly, Grids<br />
are highly dynamic in nature. Over time, participating organizations may switch from<br />
being a resource requester to being a resource provider and vice versa, and new entities<br />
may enter the system or existing entities may leave.<br />
In this dynamic and inter-organizational setting, scheduling, i.e. the decision about<br />
which user can access what resource at what time and at what cost, becomes a key<br />
challenge. Current scheduling mechanisms are based on system-centric measures and<br />
96
pre-defined policies and user priorities. The scheduler of Sun Microsystems’s N1 Grid<br />
Engine, for example, uses a combination of a central queue and a technical scheduler,<br />
as illustrated in Figure 1 [Anonymous, 2005].<br />
Figure 1. Scheduler in Sun Microsystems’s N1 Grid Engine<br />
Requests are placed in the central queue according to a priority value which is<br />
computed by using a complex combination of policies. A sample policy mix may<br />
comprise manually pre-defined shares for specific user groups, departments or projects,<br />
a contribution for requests which are close to their deadline or which have been waiting<br />
in the system for a long time, and users can sort their own requests according to their<br />
importance. Following this queue, a technical scheduler assigns the requests to Grid<br />
resources. The objective of technical schedulers is to maximize resource utilization and<br />
balance the system load [Buyya et al., 2005]. Users cannot specify their valuation for<br />
Grid resources. In settings with excess demand for resources, the resulting allocations<br />
will thus typically not coincide with economically efficient allocations that try to<br />
accommodate those requests with the highest valuations. Furthermore, pre-defined<br />
policies and priorities which are manually administered fail in dynamic and large-scale<br />
Grid settings with users frequently entering and / or leaving the system. In summary,<br />
current technical scheduling mechanisms are not fully able to handle the interorganizational<br />
and dynamic nature of Grids.<br />
97
To this end, market mechanisms are deemed promising to revolutionize resource<br />
allocation in Grids for two reasons: They dynamically and efficiently match resource<br />
requests to resource offers based on the users’ valuations. In addition, they establish<br />
previously unknown prices where these prices reflect the scarcity of the respective<br />
resources, thus offering users an incentive to contribute their idle resources to the Grid<br />
in return for the market price. Current market mechanisms, however, are not adequately<br />
able to account for dependencies between multiple Grid resources (e.g. computing<br />
power and memory are needed simultaneously) in large-scale settings with strategic<br />
users who try to benefit from manipulating the mechanism.<br />
Our paper is novel as:<br />
• We design a scalable market-based heuristic for clearing large-scale Grid<br />
settings.<br />
• We present alternative pricing schemes which may complement this heuristic<br />
so as to induce resource requesters and providers to report their true<br />
valuations and resource characteristics to the system.<br />
• We provide an in-depth analysis of these pricing schemes with respect to the<br />
distribution of welfare among resource requesters and providers, the incentives<br />
of single users to try to manipulate the mechanism, and the computational<br />
impact of the pricing schemes on the mechanism.<br />
• We outline basic design options of how the mechanism can be integrated into<br />
current Grid schedulers.<br />
This paper is structured as follows. We present related work in the field of market<br />
mechanisms in Grids. We subsequently describe a scalable heuristic for clearing Grid<br />
markets, before we present and analyze existing and elaborate novel pricing and<br />
98
payment schemes which distribute the revenue collected on the demand side of the<br />
market to the provisioning side while accounting for both computational and strategic<br />
considerations. These dimensions are evaluated in more detail by means of a numerical<br />
experiment. We finally summarize the paper and highlight future research directions.<br />
Related Work<br />
A conceptual view on the integration of market mechanisms into Grid systems is<br />
provided by the G-commerce project [Wolski et al., 2001] and GridBus [Buyya et al.,<br />
2002]. These projects propose and evaluate basic auction mechanisms. But as Wellman<br />
et al. [Wellman et al., 2001] point out, scheduling problems often exhibit properties which<br />
are not captured by such basic auction protocols, e.g. the presence of multiple resource<br />
providers and time constraints. Instead, a suite of mechanisms is likely to emerge for a<br />
broader range of scheduling problems. Research in market-based allocation of Grid<br />
resources can be separated into two streams with respect to the trading object:<br />
mechanisms which consider the trading of one type of resource only and mechanisms<br />
which account for dependencies between multiple Grid resources.<br />
In Spawn, workstations auction off idle computing time by means of a Vickrey auction<br />
[Waldspurger et al., 1992]. In market-based proportional share, resource requesters get<br />
allotted computing time proportional to their share in the total reported valuation across<br />
all requesters [Chun and Culler, 1999]. The Popcorn market is an online market for<br />
computing power which implements a Vickrey auction as well as two double auctions<br />
[Regev and Nisan, 1998]. All of these approaches allow the specification and trading of<br />
computing power only. But requesters require a b<strong>und</strong>le of resources such as computing<br />
power, memory, and bandwidth. On the one hand, the approaches at hand thus lead to<br />
inefficient allocations since requests with the same demand for computing power but<br />
99
different memory requirements, for instance, are treated the same. On the other hand,<br />
requesters are exposed to the risk of only being able to obtain one portion of the b<strong>und</strong>le<br />
of required resources without the other (“exposure risk”).<br />
Schnizler et al. [Schnizler et al., forthcoming] elaborate a Multi-Attribute Combinatorial<br />
Exchange (MACE) which targets these deficiencies. Users are allowed to request and<br />
offer arbitrary b<strong>und</strong>les of computer resources and can specify quality attributes on these<br />
resources. The pricing scheme of MACE yields approximately truthful prices on both<br />
sides of the market. The scheduling problem in this combinatorial setting is NP-hard and<br />
the mechanism is thus not applicable to large-scale settings and interactive applications<br />
which require the immediate allocation of resources. The Bellagio system also<br />
implements a combinatorial exchange for computing resources [AuYoung et al., 2004].<br />
Being an exact mechanism it shares the computational drawbacks of MACE. The work<br />
of Bapna et al. [Bapna et al., forthcoming] is most relevant to the work presented in this<br />
paper. In their model, multiple requesters and providers can trade both computing power<br />
and memory for a sequence of time slots. First, an exact mechanism is introduced. By<br />
introducing fairness constraints and imposing one common valuation across all resource<br />
providers, a linear ordering across all jobs and time is introduced which reduces the<br />
complexity of the allocation problem, which however still remains NP-hard. To mitigate<br />
this computational complexity, a fast, greedy heuristic is proposed at the expense of<br />
both truthfulness and efficiency.<br />
In summary, the mechanisms proposed for Grids are not fully able to account for<br />
dependencies between multiple Grid resources in large-scale settings with strategic<br />
users. In the following section, we propose a scalable heuristic which allows the trading<br />
of both computing power and memory. We present pricing schemes which may<br />
100
complement this heuristic so as to induce strategic users to truthful behavior, before we<br />
briefly discuss how this mechanism can be employed as economic Grid scheduler.<br />
The Exchange<br />
We will introduce the bidding language first which determines how resource requests<br />
and offers are expressed. In order to enable true usability of market-based approaches,<br />
human users may be released from the burden of having to issue requests and offers<br />
manually. Instead, software agents may serve so as to hide the Grid’s complexity from<br />
human users by automatically trading resources based on the current resource<br />
consumption of applications and configurable bidding rules which automatically derive<br />
corresponding valuations [MacKie-Mason and Wellman, 2006, Neumann et al., 2006].<br />
The market mechanism is a centralized entity and clears periodically, i.e. it collects<br />
resource requests and offers for a period of time before making its allocation decision.<br />
The mechanism’s order book is closed in that market participants do not obtain<br />
information about other participants’ requests and offers.<br />
Expressing Requests and Offers<br />
In our model, computing power is the central scarce resource to be traded. Let N be<br />
the set of resource offers which the mechanism has collected over a period of time.<br />
When submitting a resource offer (“node”) n ∈ N, resource providers report the tuple (r n ,<br />
c¯n, m¯ n, ε n , λ n ) to the system. r n ∈ R + denotes the provider’s reservation price per unit of<br />
computing power and time in some common monetary unit (MU), c¯n ∈ ℵ and m¯ n ∈ ℵ<br />
specify the maximum available amount of computing power (e.g. number of CPUs or<br />
CPU cycles) and memory (e.g. in Megabytes) respectively, while ε n ∈ ℵ and λ n ∈ ℵ<br />
indicate the earliest and latest time slot during which these resources are available. We<br />
101
assume that nodes can virtually execute multiple requests in parallel, e.g. by viewing<br />
each node as a virtual machine which makes this node almost perfectly divisible<br />
[Barham et al., 2003].<br />
Let J be the set of resource requests. Resource requesters wanting to submit a<br />
computational task (“job”) j ∈ J report the tuple (b j , c j , m j , s j , e j ) to the system where b j ∈<br />
R + expresses the requester’s maximum willingness to pay per unit of computing power<br />
and time, c j ∈ ℵ and m j ∈ ℵ specify the minimum required amount of computing power<br />
and memory respectively, and s j ∈ ℵ and e j ∈ ℵ mark the jobs start and end time. A job j<br />
can only be executed in its entirety, meaning it is only allocated if there are sufficient<br />
resources available in each time slot t ∈ [s j , e j ]. A job can only be run on one node at a<br />
time but can migrate across nodes over time.<br />
Table 1: Sample Requests and Offers<br />
Job j b j c j m j s j e j Node n r n c¯n m¯ n ε n λ n<br />
J1 10 35 50 3 7 N1 8 123 98 1 8<br />
J2 14 35 27 1 7 N2 11 98 82 1 9<br />
J3 11 84 45 2 7<br />
J4 16 71 48 2 7<br />
J5 12 63 30 2 7<br />
J6 17 55 20 2 7<br />
Example: The sample requests and offers listed in Table 1 have been submitted to<br />
the system. Job J1 is requesting 35 units of computing power and 50 units of memory<br />
throughout timeslots 3 to 7. It is willing to pay up to 10 MU per unit of computing power<br />
and timeslot, i.e. b j c j (e j – s j +1) = 1,750 MU in total. Node N1 requires a payment of at<br />
least 8 MU per unit of computing power and timeslot while it is offering up to 123 units of<br />
computing power and 98 units of memory throughout timeslots 1 to 8.<br />
102
Allocating Requests to Offers<br />
Let T be the scheduling horizon for a problem instance, i.e. the set covering all time<br />
slots specified in resource requests and offers. We introduce the binary decision variable<br />
x with x jnt = 1 if job j is allocated to node n in time slot t and x jnt = 0 else. Then the winner<br />
determination problem which allocates resource requests to resource offers so as to<br />
maximize allocative efficiency V, that is welfare (or overall “happiness” of the market<br />
participants), can be formalized as a mathematical integer program:<br />
maxV<br />
: =<br />
X<br />
s.<br />
t.<br />
x<br />
jnt<br />
∑<br />
∈<br />
{ 0,1 },<br />
∑ x<br />
n∈N<br />
jnt<br />
∑ x<br />
j∈J<br />
jnt<br />
∑ x<br />
j∈J<br />
jnt<br />
e j<br />
∑ ∑<br />
≤ 1, s<br />
c<br />
c<br />
m<br />
u= s j n∈N<br />
j∈J<br />
j T∈t<br />
n∈N<br />
j<br />
s<br />
j<br />
∑ ∑<br />
j<br />
x<br />
( b − r )<br />
≤ t ≤ e , ε ≤ t ≤ λ , b<br />
≤ t ≤ e , j ∈ J<br />
≤ c n , ε ≤ t ≤ λ,<br />
n ∈ N<br />
x<br />
≤ m n , ε ≤ t ≤ λ,<br />
n ∈ N<br />
jnu<br />
j<br />
=<br />
n<br />
n<br />
j<br />
j<br />
n<br />
jnt<br />
j<br />
n<br />
n<br />
j<br />
≥ r , j ∈ J,<br />
n ∈ N<br />
n<br />
( C1)<br />
( C2)<br />
( C3)<br />
( C4)<br />
( e − s + 1) x , s t e , j J ( C5)<br />
j j ∑ ≤ ≤ ∈<br />
n∈N<br />
jnt j<br />
j<br />
Constraint (C1) introduces the binary decision variable x and ensures that a job can<br />
only be allocated to a node which is accessible during the right time slots and whose<br />
reservation price does not exceed the job’s willingness to pay. Furthermore, a job can<br />
only be allocated to at most one node at a time (C2). Constraints (C3) and (C4) specify<br />
that the jobs allocated to one node are not allowed to consume more resources than are<br />
available on this node. Constraint (C5) enforces atomicity, i.e. a job is either fully<br />
executed or it is not executed at all.<br />
This winner determination problem constitutes a special case of the NP-complete<br />
Generalized Assignment Problem (GAP) [Chekuri and Khanna, 2000]. Consequently,<br />
solving this combinatorial winner determination problem to optimality makes scheduling<br />
in this model computationally intractable in practice. Numerical experiments indicate that<br />
clearing order books with 170 resource requests and 170 resource offers already takes<br />
103
about 22 minutes on average with CPLEX, a state-of-the-art commercial optimization<br />
engine, with the solver not being able to find any feasible solution within 30 minutes in<br />
14 out of 30 simulation runs [self-citing]. In practice, especially with interactive<br />
applications which consist of rapid CPU bursts intermitted by I/O-requests, users<br />
typically require the resources in a timely manner and the overhead for determining<br />
allocations must be kept as low as possible. In interval allocation mode, the scheduler in<br />
Sun N1 Grid Engine [Anonymous, 2005], for instance, is typically executed every 2 to 15<br />
seconds, making the computation of optimal allocation schemes infeasible in practice.<br />
This computational hardness forms the rationale for an allocation scheme based on a<br />
greedy heuristic which, in its basic form, has been proposed by Lehmann et al.<br />
[Lehmann et al., 2002] and Mu’alem and Nisan [Mu’alem and Nisan, 2002] for clearing<br />
single-sided single-attribute combinatorial auctions. We extend this heuristic to the<br />
setting of double-sided multi-attribute auctions. The greedy allocation scheme works as<br />
follows:<br />
(1) Sort jobs j ∈ J in non-increasing order of their reported willingness to pay b j , and<br />
nodes n ∈ N in non-decreasing order of their reported reservation prices r n .<br />
(2) Run sequentially through the job ranking, starting with the highest ranked job. For<br />
each job j, try to allocate this job to the cheapest feasible node which has still idle<br />
capacity left.<br />
The basic idea behind this heuristic is that, by sorting the jobs and nodes with respect<br />
to their reported valuation in Step (1), we try to greedily maximize the difference b j – r n in<br />
the objective function of the exact formalization of the winner determination problem<br />
above for each job assignment.<br />
104
Example: Applying the greedy allocation scheme to the example generates the<br />
allocation schedule depicted in Table 2 with welfare of V greedy = 6,570 MU, while the<br />
optimum is 6,858 MU. In this example, job J6 is willing to pay up to 17 MU per unit of<br />
computing power and timeslot, which is more than any other job is willing to pay.<br />
Consequently, J6 is ranked first in the sorting phase, while J4 and J6 are ranked second<br />
and third respectively. After having allocated these three jobs, the remaining jobs either<br />
do not fit on nodes N1 and N2 at the same time or are not willing to pay these nodes<br />
reservation prices and are thus not allocated.<br />
Table 2: Sample Greedy Allocation Scheme<br />
t 1 2 3 4 5 6 7 8 9<br />
N1 J2 J2, J6 J2, J6 J2, J6 J2, J6 J2, J6 J2, J6<br />
N2 J4 J4 J4 J4 J4 J4<br />
This allocation scheme runs in polynomial time of the size of its input, i.e. the number<br />
of resource requests and offers. The sorting phase (Step (1)) runs in O(|J|log|J|) and<br />
O(|N|log|N|) while the allocation phase (Steps (2) to (4)) runs in O(|J||N|). This<br />
computational speed is demonstrated by numerical experiments. In contrast to the 22<br />
minutes of the exact integer program formalization, the heuristic clears order books with<br />
170 resource requests and 170 resource offers in about 40 milliseconds on average<br />
[self-citing].<br />
The computational speed of heuristics generally comes at the expense of efficiency.<br />
Let V* be the (optimal) efficiency generated by the exact mechanism and V greedy the<br />
efficiency generated by the greedy heuristic.<br />
Proposition 1: In the worst case, the performance ratio V greedy /V* can be made<br />
arbitrarily close to zero.<br />
See Appendix A for the proof. While this is clearly a negative result, even the<br />
standard GAP is known to be APX-hard to approximate and only a 2-approximation<br />
105
exists [Chekuri and Khanna, 2000]. Due to the multiple resources and time constraints,<br />
the structure of our allocation problem is tremendously more complex than the standard<br />
GAP, and these bo<strong>und</strong>s thus probably do not hold for our model. Furthermore, the<br />
approximation techniques used to obtain these bo<strong>und</strong>s do not allow for constructing<br />
truthful prices [Mu’alem and Nisan, 2002], a key feature of our mechanism as we will<br />
present below. Those special worst-cases, however, are merely of theoretical value and<br />
the heuristic’s efficiency needs to be evaluated in more realistic settings. Numerical<br />
experiments show that the greedy heuristic approximates the optimal efficiency<br />
generated by the exact integer program between 88-96% [self-citing].<br />
Besides its computational speed and approximate efficiency, the heuristic allocation<br />
scheme implements desirable strategic properties as it limits the potential gain of both<br />
resource requesters and providers to manipulate the mechanism by reporting resource<br />
characteristics, i.e. computing and memory requirements and time constraints, other<br />
than their true characteristics. Truthfulness with respect to a job’s resource requirements<br />
is straightforward. If a job’s resource requirements are <strong>und</strong>erstated, the job will not be<br />
finished in time; it has to pay for the used resources but these are of no value.<br />
Overstating a job’s requirements either increases the job’s payment or the job is not<br />
scheduled at all. Analogously, resource providers clearly cannot gain from <strong>und</strong>erstating<br />
the amount of available resources. Given sufficiently severe penalties, providers cannot<br />
gain from overstating their available resources either, since they may then not be able to<br />
execute allocated jobs.<br />
Moulin [Moulin, forthcoming] introduces two further strategic properties of<br />
mechanism’s which are relevant in this regard: A mechanism is said to be split-proof if<br />
users cannot gain from splitting their jobs (nodes) into several smaller jobs (nodes). A<br />
106
mechanism is said to be merge-proof if users cannot benefit from merging several jobs<br />
(nodes) to one bigger job (node).<br />
Proposition 2: The greedy heuristic is both split- and merge-proof.<br />
See Appendix A for a sketch of the proof. Consequently, the strategy space of selfish<br />
users is restricted to misstating the valuation for resources only, i.e. the maximum<br />
willingness to pay respectively the reservation price. In the following section, we will<br />
propose pricing schemes which further limit the mechanisms vulnerability in this respect.<br />
Pricing the Outcome<br />
Market mechanisms consist of two basic components: an allocation scheme and a<br />
pricing scheme. While the allocation component (i.e. the winner determination problem)<br />
generally accounts for the technical peculiarities of the Grid setting, the objective of the<br />
pricing component is to induce the selfish agents in the market environment to act<br />
towards the general objective, i.e. maximize efficiency. As Lai [Lai, 2005] points out, “the<br />
attention of system designers, especially those designing scalable systems, should also<br />
be balanced with equal concern given to strategic behavior”. To this end, in this section<br />
we present two pricing schemes which may be used in conjunction with the greedy<br />
heuristic and analytically derive both positive and negative results.<br />
Proportional CV-Pricing<br />
The concept of critical-value-based pricing (henceforth “CV-Pricing”) has been<br />
introduced and studied by Lehmann et al. [Lehmann et al., 2002] and Mu’alem and<br />
Nisan [Mu’alem and Nisan, 2002]. Let φ j R +<br />
be the minimal valuation per unit of<br />
computing power and time slot which the winning job j would have needed to report to<br />
the mechanism in order to still remain in the allocation. Then, with CV-Pricing, we set the<br />
107
price of job j to p j = φ j c j (e j – s j + 1) if j is in the allocation and p j = 0 else. In the remainder<br />
of this paper, we assume resource requesters to have quasi-linear utility functions.<br />
Proposition 3: The greedy heuristic with critical-value-based prices for resource<br />
requests makes truthtelling a (weakly) dominant strategy.<br />
Refer to Appendix A for the proof. CV-Pricing is a powerful tool as it tremendously<br />
simplifies the strategy space of resource requesters: With truthful prices in dominant<br />
strategies, there is no need to reason about the strategies of other users. The critical<br />
value of a job essentially hinges on the competition of other jobs for the same resources.<br />
If there is no competition, the job is only required to pay the reservation price. Otherwise,<br />
the job at least needs to outbid these competing jobs. If there is sufficient competition<br />
such that critical values are above reservation prices, CV-Pricing of resource requests<br />
generates a surplus, i.e. overall payments exceeding the providers’ reservation prices.<br />
The major drawback of CV-Pricing in our setting is the computational cost of<br />
determining the critical values. In order to calculate the critical value for a specific<br />
allocated request, it is not possible to simply take the valuation of the job with the<br />
highest ranking which does not get executed since this job may not be executable in<br />
general due to capacity constraints. Moreover, removing j from the winning allocation<br />
might change the allocation of other jobs within the winning allocation as well.<br />
Consequently, to determine the critical value for each winning job j, we need to<br />
determine the allocation without j: We successively allocate all other jobs from the<br />
ranking and, after having allocated a job, check whether j can still be accommodated.<br />
Note that, while the mechanism seems to suffer from the same computational problems<br />
than the prominent VCG mechanism, we can now compute the critical values in<br />
polynomial time due to our greedy allocation scheme.<br />
108
The subsequent question is, how the surplus generated on the demand-side of the<br />
market (i.e. revenue exceeding reservation prices) can be distributed to resource<br />
providers so as to also produce truthful payments. Unfortunately, CV-Pricing is not<br />
applicable to the pricing of resource offers. It requires a binary decision in the sense that<br />
a job is only allocated as a whole, or not at all. In our model, however, resource offers<br />
are divisible. Worse, the following proposition holds:<br />
Proposition 4: There is no payment scheme which can complement the greedy<br />
heuristic so as to generate truthful payments to resource providers.<br />
See Appendix A for a proof sketch. This is clearly a strong negative result as it<br />
extends across any payment scheme for the greedy heuristic.<br />
As a consequence of Proposition 4, we cannot design a payment scheme which<br />
precludes any strategic behavior of resource providers but we can only try to limit these<br />
strategic possibilities. A first approach may be to distribute any surplus according to the<br />
providers’ contribution of computing power, the key resource in our model. Let<br />
S : =<br />
j∈J<br />
jnt j<br />
∑ p −<br />
r<br />
j∈J<br />
j ∈ ∈<br />
n<br />
n<br />
∑ ∑ ∑<br />
n N t T<br />
c<br />
be this surplus. The first term captures the total revenue collected by CV-Pricing,<br />
whereas the second term captures the providers’ reservation prices for the allocation<br />
schedule at hand. Then, with proportional payments, provider n receives payments<br />
amounting to<br />
∑t<br />
∈T<br />
∑<br />
x<br />
c<br />
x<br />
c<br />
∑t∈T<br />
∑ j∈J<br />
∑t<br />
T ∑j<br />
J∑<br />
j∈J<br />
jnt j<br />
jnt j<br />
r ⋅<br />
+ S ⋅<br />
n<br />
,<br />
c n<br />
x<br />
∈ ∈ m∈N<br />
jmt<br />
c<br />
j<br />
where the first summand captures n’s reservation price of the allocation schedule and<br />
the second summand captures n’s proportional payments. The rationale for proportional<br />
x<br />
c<br />
109
payments is that the share of the surplus which is allotted to n does not directly depend<br />
on n’s reported reservation price. Instead, it only depends on the final allocation<br />
schedule, which can only be influenced by n to a rather limited extent depending on the<br />
competition on both sides of the market. The prices and payments generated by<br />
Proportional CV-Pricing for the sample requests and offers are listed in Table 3.<br />
Table 3: Sample Prices and Payments<br />
Proportional CV-Pricing K-Pricing (k = 0.5)<br />
Job j p j Node n p n Job j p j Node n p n<br />
J2 2,940 N1 6,165.88 J2 2,695 N1 6,820<br />
J4 5,112 N2 5,846.12 J4 5,751 N2 5,751<br />
J6 3,960 J6 4,125<br />
J1, J3, J5 0 J1, J3, J5 0<br />
Σ 12,012 Σ 12,012 Σ 12,571 Σ 12,571<br />
K-Pricing<br />
K-Pricing is an alternative pricing scheme to Proportional CV-Pricing. It was<br />
introduced in Schnizler et al. [Schnizler et al., forthcoming] for double-sided<br />
combinatorial auctions. The basic idea is to distribute the welfare generated by the<br />
allocation algorithm between resource requesters and resource providers according to a<br />
factor k [0,1]. For instance, assume an allocation of resources from a specific provider<br />
to a specific requester. The requester values these resources at 10 MU while the<br />
provider has a reservation price of 5 MU. Then the (local) welfare of this transaction is<br />
10 MU – 5 MU = 5 MU and k * 5 MU of the surplus is allotted to the requester – thus<br />
having to pay 10 MU – k * 5 MU – and (1 – k) * 5 MU is allotted to the provider – thus<br />
receiving 5 MU + (1 – k) * 5 MU.<br />
K-Pricing has two main advantages. The distribution of welfare among resource<br />
requesters and providers can be flexibly pre-defined by setting the factor k accordingly,<br />
thus allowing for both fairness and revenue considerations. Prices can be determined in<br />
110
polynomial runtime. On its downside, however, it only yields approximately truthful prices<br />
and payments on both sides of the market.<br />
Example: The prices and payments generated by Proportional CV-Pricing and K-<br />
Pricing with k = 0.5 are listed in Table 3. By construction, both pricing schemes generate<br />
budget-balanced prices and payments. In this example, K-Pricing produces higher<br />
revenue (12,571 MU) than Proportional CV-Pricing (12,012 MU).<br />
Evaluation<br />
The analytic evaluation in the previous section allowed us to derive some general<br />
insights into the strategic and computational properties of the presented pricing<br />
schemes. These general properties, however, only give rather limited guidance to the<br />
market operator who has to choose an “adequate” pricing scheme for the strategic<br />
setting at hand. In this section, we thus employ a numerical simulation in order to obtain<br />
more detailed insights into the properties of the presented pricing schemes, in particular<br />
with respect to:<br />
• The distribution of welfare among resource requesters and providers;<br />
• The incentives of selfish users to misreport to the mechanism about their<br />
valuations;<br />
• The computational influence on the mechanism as a whole.<br />
Data Generation<br />
Based on our model and the corresponding bidding language, there is a set of<br />
parameters upon which problem instances (henceforth “order books”) need to be<br />
generated (cf. Table 4). The resource requirements of jobs were randomly drawn from a<br />
lognormal distribution with mean 4 and variance .28 (in log space), whereas the<br />
111
esource characteristics of nodes were drawn from a lognormal distribution with mean<br />
4.5 and variance .16. The lognormal distribution is recommended by Feitelson<br />
[Feitelson, 2002] for the creation of parameters such as file size. Representing a normal<br />
distribution in log space, it is modal and positively skewed. The start times and end<br />
times have been drawn from normal distributions while the maximum willingness to pay<br />
and reservation prices were drawn from uniform distributions.<br />
Data Analysis<br />
Table 4: Simulation Settings<br />
Parameter Resource Requesters Resource Providers<br />
Computing power Lognormal (4, .28) Lognormal (4.5, .16)<br />
Memory Lognormal (4, .28) Lognormal (4.5, .16)<br />
Start time Normal (2, .5) Normal (2, .5)<br />
End time Normal (6, .5) Normal (8, .5)<br />
Valuation Uniform [10, 20] Uniform [7.5, 12.5]<br />
The simulations were run on an Intel Pentium Xeon computer with 3.2 GHz and 2 GB<br />
memory. We simulated four pricing schemes: K-Pricing with k = .7 (i.e. 70% of welfare is<br />
allocated to the demand side), k = .5, k = .3, and Proportional CV-Pricing. We created<br />
200 order books in total based on the distributions listed above. We then fed each<br />
combination of the heuristic and these pricing schemes with the same set of order<br />
books, i.e. we performed 200 runs per setting. The large number of runs is due to the<br />
number of parameters and their interdependencies (e.g. recall that the total valuation of<br />
a job (node) equals its reported valuation times its required computing power times its<br />
time span) which may cause heavy statistical noise.<br />
Welfare<br />
In most market settings, the distribution of welfare (i.e. the sum over the true<br />
valuations of all winning resource requesters less the sum over the true reservation<br />
prices of all winning resource providers) among resource requesters and providers will<br />
112
e the primary design consideration: Which side of the market receives what fraction of<br />
the welfare that is generated by the allocation scheme? Table 5 lists the results of our<br />
simulation with respect to this question. In a simulation with 20 requesters and 20<br />
providers, the heuristic allocation scheme generates welfare amounting to 35,735.33 MU<br />
on average across 200 runs. For K-Pricing, the answer to the above question is simple,<br />
as the demand side receives the fraction k of the welfare whereas the supply side<br />
receives 1 – k. The answer is more complex for Proportional CV-Pricing. In this<br />
simulation, Proportional CV-Pricing assigns about 71% of welfare to the demand side.<br />
Resource providers need to abandon a comparably large fraction of welfare to the<br />
requesters in order to induce these to truthful reporting. The difference b j – φ j between<br />
the reported valuation of request j and its critical value may thus be interpreted as a<br />
discount. Recall from above that the critical values of jobs heavily depend on the<br />
competition in the market and that the providers’ surplus will thus be higher in more<br />
competitive settings.<br />
Table 5: Distribution of Welfare<br />
Pricing Scheme Requesters’ surplus Providers’ surplus<br />
K-Pricing (k = 0.3) 10,720.6 (30%) 25,014.73 (70%)<br />
K-Pricing (k = 0.5) 17,867.66 (50%) 17,867.66 (50%)<br />
K-Pricing (k = 0.7) 25,014.73 (70%) 10,720.6 (30%)<br />
Proportional CV-Pricing 25,353.20 (70.95%) 10,382.13 (29.05%)<br />
The results listed in Table 5 assume truthful reporting of valuations by both resource<br />
requesters and providers. Consequently, we want to examine the strategic properties of<br />
the presented pricing schemes in more detail.<br />
Manipulating requesters<br />
In a first setting, we analyze to what extent a single requester can benefit from<br />
reporting a valuation other than her true valuation. Let v j be the true valuation for request<br />
113
j. Then we let j report b j = α * v j and vary α from .7 to .8 up to 1.3, that is b j is a<br />
percentage bid of the true valuation. For j, there are intuitively two possible strategies of<br />
how to deviate from truthful bidding:<br />
α < 1, i.e. j <strong>und</strong>erstates its true valuation. With K-Pricing, there are two somewhat<br />
opposite implications of this strategy compared to truthful bidding: On the one hand (the<br />
adverse effect), j risks getting a lower ranking and may thus not be allocated at all, or it<br />
may be allocated to a node n with a higher reservation price and may thus obtain a<br />
smaller fraction of the resulting welfare spread b j – r n . On the other hand (the beneficial<br />
effect), j may remain in the allocation and may instead even be able to obtain a higher<br />
fraction of the welfare spread, e.g. if it is still allocated to the same node as with truthful<br />
bidding. Which effect finally prevails depends on the individual setting.<br />
α > 1, i.e. j overstates its true valuation. Equivalently, j may improve its ranking but<br />
risk getting a lower fraction of the welfare spread. On the other hand, it may instead<br />
even obtain a larger fraction.<br />
Table 6 lists the average utility gain across all 200 runs for a single requester from<br />
misstating compared to truthful reporting in each of the four pricing schemes for the<br />
request side of the market.<br />
Table 6: Average utility gain from misstating for a single<br />
requester<br />
α (b j = α* v j )<br />
K-Pricing K-Pricing K-Pricing Proportional<br />
(k = 0.3) (k = 0.5) (k = 0.7) CV-Pricing<br />
.7 149.40 -146.96 -443.32 -405.20<br />
.8 327.56 108.54 -110.48 -93.69<br />
.9 177.86 52.41 -73.04 -32.48<br />
1.1 -306.31 -191.12 -75.92 -28.71<br />
1.2 -632.36 -397.81 -163.25 -53.81<br />
1.3 -949.66 -598.58 -247.49 -53.81<br />
In the simulation we fo<strong>und</strong> that with α = .9, the manipulating requester dropped out of<br />
the allocation in 16.5% of the runs, with α = .8 in 25% of the runs, and with α = .7 the<br />
114
equester dropped out of the allocation already in 47% of the runs. This component of<br />
the adverse effect stems from the design of the ranking in the allocation heuristic and<br />
thus holds for all pricing schemes.<br />
Since we fed each pricing scheme with identical sets of 200 order books, we applied<br />
one-tailed matched pairs t-tests to determine the statistical significance of these<br />
deviations. The p-values are given in Table B1 in Appendix B. The test shows a<br />
statistically significant utility gain from <strong>und</strong>erstating (α < 1) for K-Pricing with k = .3 and<br />
for α = .8 and α = .9 with k = .5. For all other settings, the test shows a statistically<br />
significant utility loss from misstating (i.e. both <strong>und</strong>er- and overstating). In essence, the<br />
larger the factor k – i.e. the larger the fraction of welfare which is allocated to resource<br />
requesters – the less can these requesters gain from misstating. We hypothesize that<br />
this effect can be explained with the two opposite effects pointed out above: On<br />
average, the larger the factor k, the lower the beneficial and the higher the adverse<br />
effect from <strong>und</strong>erstating the valuation for a specific job. On the other side, for overstating<br />
resource requesters, we can see that on average, the larger k, the higher the beneficial<br />
and the lower the adverse effect, whereas the latter still prevails.<br />
As stated in Proposition 3, with Proportional CV-Pricing, requesters cannot gain from<br />
misstating their valuations in any case, so the same holds for the averages across all<br />
runs.<br />
Manipulating Providers<br />
In a second – and from the setup symmetric – setting, we analyze to what extent a<br />
single provider can benefit from reporting a reservation price other than her true<br />
reservation price. The implications from <strong>und</strong>er- respectively overbidding are essentially<br />
analogous to those on the demand side:<br />
115
α < 1: As for resource requesters, there are two opposite effects of <strong>und</strong>erstating the<br />
reservation price. Recall that the payment to a resource provider consists of two<br />
components: its reservation price and its fraction of the surplus, i.e. total revenue less<br />
total reservation prices. On the one hand, by <strong>und</strong>erstating its reservation price, the node<br />
inherently lowers the first component. On the other hand, the node may achieve a higher<br />
ranking and may thus be allocated more demand and thus a higher fraction of the<br />
surplus.<br />
α > 1: Again, the implications of overstating the reservation price are essentially the<br />
reversion of the consequences of <strong>und</strong>erstating this valuation.<br />
Table 7 lists the average utility gain across all 200 runs for a single requester from<br />
misstating compared to truthful reporting in each of the four payment schemes. The p-<br />
values for the one-tailed matched pairs t-tests for this setting are listed in Table B2 in<br />
Appendix B.<br />
Table 7: Average utility gain from misstating for a single<br />
provider<br />
α (b j = α* v j )<br />
K-Pricing K-Pricing K-Pricing Proportional<br />
(k = .3) (k = .5) (k = .7) CV-Pricing<br />
.7 573.54 23.19 -527.16 -1026.51<br />
.8 425.28 58.92 -307.45 -598.16<br />
.9 -37.23 -125.91 -214.58 -260.26<br />
1.1 -427.84 -248.67 -69.49 26.03<br />
1.2 -744.00 -449.03 -154.06 15.18<br />
1.3 -1067.02 -698.20 -329.38 -187.86<br />
For K-Pricing with k = .3, the average utility gain from <strong>und</strong>erstating the reservation<br />
price by 20% (α = .8) and by 30% (α = .7) is statistically significant with the p-values<br />
being 9 * 10 –10 and 1 * 10 –12 respectively. While there is still a slight potential gain from<br />
<strong>und</strong>erstating with k = .5, this gain is not statistically significant with p-values of .103 for α<br />
= .8 and .325 for α = .7. For k = .7, there is no gain from both <strong>und</strong>er- and overstating.<br />
116
These results can again be explained by means of the two opposite implications of<br />
<strong>und</strong>erstating respectively overstating. As discussed above, if the requester n<br />
<strong>und</strong>erstates its reservation price r n , it directly lowers this component of its payment, but<br />
may be able to extract a larger fraction of the spread b j – r n . However, since the<br />
requester receives (1 – k) * (b j – r n ) of this spread, this beneficial effect of <strong>und</strong>erstating<br />
gradually diminishes as k converges to 1.<br />
Proportional CV-Pricing punishes the <strong>und</strong>erstatement of the reservation price more<br />
than K-Pricing. On its downside, it offers potential gains from overstating. But these<br />
gains only seem to support slight deviations. Looking at the p-values in Table B2, the<br />
gain from overstating is not statistically significant (.235 for α = 1.1 and .382 for α = 1.2).<br />
Scalability<br />
As reported above, numerical experiments show that the greedy heuristic generates<br />
highly efficient allocations at low computational cost. However, not only the allocation<br />
scheme, but also the pricing scheme places a computational burden on the mechanism<br />
as a whole. As presented above, with Proportional CV-Pricing, in order to determine the<br />
critical value of a winning job, the allocation without this job needs to be determined.<br />
In order to examine the impact of each pricing scheme on the mechanism’s<br />
computational complexity in more detail, we varied the number of requests and offers in<br />
the market from 20 orders per side, 40 orders per side, and so forth up to 200 orders per<br />
side. We again randomly generated 200 order books for each of these settings, and<br />
performed the heuristic with both K-Pricing and Proportional CV-Pricing. The average<br />
runtimes across all 200 runs are listed in Table 8. These simulation results confirm the<br />
computational superiority of the K-Pricing scheme. Complemented with K-Pricing, the<br />
heuristic only takes about 90 milliseconds to clear comparably large order books with<br />
117
200 requests and 200 offers. The computational cost of Proportional CV-Pricing is<br />
illustrated by the heuristic with Proportional CV-Pricing, taking 2,400 milliseconds on<br />
average for the largest order books.<br />
Table 8: Average runtime in milliseconds<br />
Orders per<br />
market side<br />
20 40 60 80 100 120 140 160 180 200<br />
K-Pricing 2 7 16 26 41 48 52 58 70 87<br />
Proportional<br />
CV-Pricing<br />
6 29 77 162 302 506 813 1,214 1,717 2,400<br />
If employed in interval scheduling mode, Sun Microsystems’s N1GE scheduler is<br />
typically executed every 15 seconds. We fo<strong>und</strong> that, if complemented with K-Pricing, the<br />
heuristic can clear up to 2,500 orders per side on average across 200 order books within<br />
this timeframe (12.7 seconds). If compared to the size of PlanetLab, the largest testbed<br />
for networking and distributed computing which currently comprises about 700 nodes<br />
[Peterson et al., 2006], these results <strong>und</strong>erline our mechanism’s scalability and<br />
applicability to practical scenarios.<br />
Implications for Market-based Resource Allocation<br />
From a welfare perspective, K-Pricing offers the nice feature that, given truthful<br />
information from all users, welfare can be distributed among resource requesters and<br />
providers according to the parameter k, which can act as adjusting crew. The welfare<br />
implications of Proportional CV-Pricing are more involved, as the distribution of welfare<br />
hinges on the competition on both sides of the market. In contrast to K-Pricing, there is<br />
no direct adjusting screw for the market operator. However, via the critical values of<br />
requests, Proportional CV-Pricing implements a desirable economic dynamicity in the<br />
sense that prices and payments adequately reflect resource scarcity. If resources are<br />
not scarce, critical values will be low, and resource providers may just receive their<br />
118
eservation prices. With increasing scarcity, critical values converge to the requesters’<br />
maximum willingness to pay, and resource providers extract a larger fraction of welfare.<br />
Considering strategic users, Proportional CV-Pricing clearly shows its strengths, as<br />
resource requesters cannot profit from misreporting and resource providers can only<br />
slightly benefit from overstating their reservation prices. With K-Pricing, the freedom in<br />
choosing k becomes somewhat restricted. On the one hand, if k – and thus requesters’<br />
surplus – is small, both resource requesters and providers have a significant incentive to<br />
<strong>und</strong>erstate their true valuations. On the other hand, if k is large, and hence providers’<br />
surplus low, this reduces the incentive for resource providers to contribute their idle<br />
resources to the Grid. With respect to computational considerations, K-Pricing clearly<br />
outperforms Proportional CV-Pricing.<br />
In large-scale settings, K-Pricing will generally be preferable to Proportional CV-<br />
Pricing due to its computational simplicity. Moreover, as Jackson [Jackson, 1992]<br />
shows, as market size increases, the potential gains from misreporting become smaller.<br />
The increasing competition assumes the pricing scheme’s role of inducing users to<br />
truthful behavior. In small-scale settings, the importance of computational considerations<br />
naturally diminishes and, in line with Jackson’s [Jackson, 1992] results for increasing<br />
market sizes, strategic considerations gain in importance as the market size decreases.<br />
Proportional CV-Pricing may thus prove superior to K-Pricing in these settings as it not<br />
only ensures truthful reporting on average, but in dominant strategies.<br />
Returning to our discussion of Grid schedulers in the introduction, there are generally<br />
two ways in which market mechanisms can be integrated into Grid schedulers.<br />
119
Figure 2a. Market mechanisms to determine<br />
priority values / shares<br />
Figure 2b. Market mechanisms to determine<br />
both priority values and allocations<br />
Firstly, market mechanisms may be used to determine the priority values respectively<br />
shares of requests in the Grid scheduler’s central queue (cf. Figure 2a). In conjunction,<br />
as with Sun’s N1 Grid Engine, a technical scheduler can then subsequently be used to<br />
allocate the requests in this queue to Grid resources. As proposed in [self-citing], the<br />
pay-as-bid mechanism of Sanghavi and Hajek [Sanghavi and Hajek, 2004] or marketbased<br />
proportional share [Chun and Culler, 1999] are two mechanisms which may be<br />
suitable for this design option, which is particularly qualified to be employed as a so<br />
called online mechanism in order to schedule interactive applications, meaning resource<br />
requests and resources are allocated as soon as they arrive in the system. Secondly,<br />
market mechanisms may assume the role of both establishing priorities and allocations,<br />
thus also displacing the technical scheduler (cf. Figure 2b). This class of mechanisms is<br />
typically executed periodically. Larger clearing periods allow for more efficient allocation<br />
decisions since there are likely to be more requests and offers over which the<br />
mechanism can try to optimize. On the other hand, in dynamic settings where users<br />
require fast allocation decisions, the clearing period can be made as small as a couple<br />
of seconds. The market mechanism presented in this paper falls into this category.<br />
120
Concluding Remarks and Future Work<br />
In the introduction, we discussed that technical schedulers based on system-centric<br />
metrics – the current approach to allocating Grid resources – do not achieve efficient<br />
allocations in the economic sense since the pre-defined and static policies and priorities<br />
cannot cope with the dynamic and large-scale nature of Grids, do not allow requesters to<br />
specify their valuations for resources, and do not provide immediate incentives to<br />
contribute to the Grid. Market mechanisms are a promising approach to Grid scheduling<br />
by explicitly addressing these deficiencies. We presented related work on market<br />
mechanisms for Grid resource allocation and argued that these current approaches do<br />
not fully consider dependencies between multiple Grid resources, scalability issues, and<br />
strategic users which may try to manipulate the mechanism in their own benefit. The<br />
contribution of this paper is the proposal of a mechanism which achieves a distinct<br />
trade-off between allocative efficiency, computational tractability, and incentive<br />
compatibility. To this end, we designed a greedy heuristic which achieves near-optimal<br />
resource allocations at low computational cost. We subsequently presented alternative<br />
pricing schemes which may complement this allocation scheme and derived some<br />
general results on their strategic and computational properties. In order to examine<br />
these properties in more detail, we employed a numerical experiment which we<br />
described in our evaluation section. We compared these results for the presented pricing<br />
schemes and derived some general recommendations and design options for integrating<br />
market mechanisms into Grid schedulers.<br />
In the future, we intend to perform further analyses of the presented pricing schemes,<br />
e.g. by means of experiments and agent-based simulations. Moreover, we plan to<br />
integrate a prototype of the market mechanism into MOSIX, a state-of-the-art Grid<br />
121
system [Barak et al., 2005]. The heuristic’s approximation of efficiency needs to be<br />
compared in detail to the efficiency produced by the exact formalization of the winner<br />
determination problem. Extensions of the heuristic allocation scheme, such as the use of<br />
more sophisticated norms in the sorting phase, as well as their impact on the<br />
mechanism’s strategic properties, may be promising areas for future research.<br />
Appendix A – Proofs<br />
Proof of Proposition 1: W.l.o.g., we consider valuations and computing power only.<br />
Suppose one resource offer with characteristics (r, c¯) = (0 MU, k), k ∈ ℵ, and two resource<br />
requests (b 1 , c 1 ) = (2 MU, 1) and (b 2 , c 2 ) = (1 MU, k). The heuristic generates welfare of 2 MU<br />
while the optimum is k MU. Clearly, V greedy /V* → 0 for k → ∞.<br />
Proof sketch of Proposition 2: The sorting in Step 1 of the heuristic depends on the<br />
reported valuations per unit of computing power. Consequently, both requesters and providers<br />
can neither simultaneously improve the rankings of two requests (offers) by merging them, nor<br />
can they simultaneously improve the rankings of two requests (offers) which result from splitting<br />
a request (offer).<br />
Proof of Proposition 3: We assume resource requesters to have quasi-linear utility<br />
functions. Let u j be the resulting overall utility for request j, v j j’s true valuation for a unit of<br />
computing power per time slot, and p j the resulting payment. Then u j = v j (e j – s j + 1)c j – p j if j is<br />
allocated and u j = 0 else. Assume j has reported truthfully, i.e. b j = v j .<br />
Suppose j has won, i.e. b j = v j > φ j and p j = φ j (e j – s j + 1)c j . Reporting a lower valuation b j < v j<br />
either leaves j in the winning allocation while it does not change φ j and consequently it does not<br />
change u j , or it leads to j being rejected and thus uj’ = 0 < (v j – φj)(ej – sj + 1)cj = uj. Reporting a<br />
higher valuation b j > v j leaves j being accepted while it does not change φj and consequently it<br />
does not change uj.<br />
122
Now suppose j has lost, i.e. b j = v j < φ j and p j = 0. Reporting a lower valuation b j < v j leaves j<br />
being rejected. Reporting a higher valuation b j > v j either leaves j being rejected (b j < φj), or it<br />
leads to j being accepted (b j > φj > v j ) and thus uj’ = (b j – φj)(ej – sj + 1)cj < 0 = uj.<br />
Proof sketch of Proposition 4: A key requirement for achieving truthfulness is designing a<br />
monotonic allocation scheme in the sense that providers cannot do better by overstating their<br />
reservation prices [Lehmann et al., 2002, Mu’alem and Nisan, 2002]. By means of a simple<br />
example we will show that providers can potentially increase their load by overstating their<br />
reservation prices.<br />
W.l.o.g., we consider valuations and computing power only. Assume two resource requests,<br />
(10 MU, 1) and (9 MU, 2), and two truthful resource offers, (1 MU, 2) and (2 MU, 2). Then the<br />
first offer will be allocated one unit of computing power. Now assume the first provider had<br />
reported (3 MU, 2) instead. Then she would have been allocated 2 units of computing power.<br />
Appendix B – p-values<br />
Table B1: p-values of one-tailed matched<br />
pairs t-tests for the utility gains from<br />
misstating for a single requester provider<br />
α<br />
K-Pricing K-Pricing K-Pricing Proportional<br />
(k = 0.3) (k = 0.5) (k = 0.7) CV-Pricing<br />
.7 .004 .007 3 * 10 –10 6 * 10 –15<br />
.8 9 * 10 –15 .003 .005 1 * 10 –7<br />
.9 1 * 10 –12 .036 .023 9 * 10 –5<br />
1.1 3 * 10 –53 2 * 10 –30 2 * 10 –5 0.001<br />
1.2 1 * 10 –57 3 * 10 –46 5 * 10 –14 4 * 10 –4<br />
Table B2: p-values of one-tailed matched<br />
pairs t-tests for the utility gains from<br />
misstating for a single provider<br />
α<br />
K-Pricing K-Pricing K-Pricing Proportional<br />
(k = 0.3) (k = 0.5) (k = 0.7) CV-Pricing<br />
.7 1 * 10 –12 .325 4 * 10 –39 5 * 10 –73<br />
.8 9 * 10 –10 .103 4 * 10 –22 3 * 10 –56<br />
.9 .065 3 * 10 –11 1 * 10 –37 6 * 10 –32<br />
1.1 1 * 10 –11 2 * 10 –8 .006 .235<br />
1.2 1 * 10 –20 7 * 10 –16 6 * 10 –6 .382<br />
1.3 2 * 10 –26 9 * 10 –23 2 * 10 –13 4 * 10 –4<br />
1.3 1 * 10 –58 2 * 10 –52 4 * 10 –22 4 * 10 –4 123<br />
Bibliography<br />
Anonymous. (2005) Sun N1GE 6 User’s Guide. Sun Mircosystems, Inc.<br />
Anonymous. (2006) Grid Computing: A Vertical Market Perspective 2006-2011. The Insight Research Corportation.<br />
Anonymous (2007a) "GlaxoSmithKline: Implementing Breakthrough Technology for Clinical Development Modeling<br />
and Simulation," United Devices, http://www.ud.com/company/resources/files/cs_gsk.pdf (May 3, 2007).<br />
Anonymous (2007b) "Sal. Oppenheim claims first-mover advantage with a grid solution from IBM and Platform," IBM<br />
Corporation,<br />
http://www.platform.com/NR/rdonlyres/D0860543-DC4C-4D43-935F-<br />
2A86387A224E/0/3597_SalOppenheim.pdf (May 3, 2007).<br />
AuYoung, A., B. N. Chun, A. C. Snoeren, and A. Vahdat (2004) “Resource Allocation in Federated Distributed<br />
Computing Infrastructures,” in Proceedings of the 1st Workshop on Operating System and Architectural Support for<br />
the On-demand IT Infrastructure. Boston, MA, USA.<br />
Bapna, R., D. Sanjunkta, R. Garfinkel, and J. Stallaert (forthcoming) "A Market Design for Grid Computing," INFORMS
Journal of Computing.<br />
Barak, A., A. Shiloh, and L. Amar (2005) “An Organizational Grid of Federated MOSIX Clusters,” in Proceedings of the<br />
5th IEEE International Symposium on Cluster Computing and the Grid (CCGrid). Cardiff, Wales.<br />
Barham, P., B. Dragovic, K. Fraser, S. Hand et al. (2003) “Xen and the art of virtualization,” in Proceedings of the 19th<br />
ACM symposium on Operating systems principles, pp. 164-177. Banff, Alberta, Canada.<br />
Buyya, R., D. Abramson, J. Giddy, and H. Stockinger (2002) "Economic models for resource <strong>management</strong> and<br />
scheduling in Grid computing," Concurrency and Computation: Practice and Experience (14) 13-15, pp. 1507-1542.<br />
Buyya, R., D. Abramson, and S. Venugopal (2005) "The grid economy," Proceedings of the IEEE (93) 3, pp. 698-714.<br />
Carr, N. (2005) "The End of Corporate Computing," Sloan Management Review (46) 3, pp. 67-73.<br />
Chekuri, C. and S. Khanna (2000) “A PTAS for the multiple knapsack problem,” in Proceedings of the 11th annual<br />
ACM-SIAM symposium on Discrete algorithms pp. 213-222. San Francisco, California, USA<br />
Chun, B. and D. E. Culler. (1999) Market-based proportional resource sharing for clusters. Millenium Project Research<br />
Report.<br />
Feitelson, D. G. (2002) “Workload Modeling for Performance Evaluation,” in, vol. 2459 Performance Evaluation of<br />
Complex Systems: Techniques and Tools, Lecture Notes in Computer Science: Springer Verlag, pp. 114-141.<br />
Foster, I., C. Kesselman, and S. Tuecke (2001) "The Anatomy of the Grid: Enabling Scalable Virtual Organizations,"<br />
International Journal of High Performance Computing Applications (15) 3, pp. 200-222.<br />
Jackson, M. O. (1992) "Incentive compatibility and competitive allocations," Economics Letters (40), pp. 299-302.<br />
Lai, K. (2005) "Markets are Dead, Long Live Markets," ACM SIGecom Exchanges (5) 4, pp. 1-10.<br />
Lehmann, D., L. I. O’Callaghan, and Y. Shoham (2002) "Truth Revelation in Approximately Efficient Combinatorial<br />
Auctions," Journal of the ACM (49) 5, pp. 577-602.<br />
MacKie-Mason, J. K. and M. P. Wellman (2006) “Automated Markets and Trading Agents,” in L. Tesfatsion and K. L.<br />
Judd (Eds.) Handbook of Computational Economics, vol. 2: Agent-Based Computational Economics: North-Holland.<br />
Moulin, H. (forthcoming) "Proportional scheduling, Split-proofness, and Mergeproofness," Games and Economic<br />
Behavior.<br />
Mu’alem, A. and N. Nisan (2002) “Truthful Approximation Mechanisms for Restricted Combinatorial Auctions,” in<br />
Proceedings of the 18th National Conference on Artificial Intelligence, pp. 379-384. Edmonton, Alberta, Canada.<br />
Neumann, D., S. Lamparter, and B. Schnizler (2006) “Automated Bidding for Trading Grid Services,” in Proceedings<br />
of the 14th European Conference on Information Systems (ECIS). Gothenburg, Sweden.<br />
Peterson, L., A. Bavier, M. E. Fiuczynski, and S. Muir (2006) “Experiences Building PlanetLab,” in Proceedings of the<br />
7th Symposium on Operating Systems Design and Implementation. Seattle, WA, USA.<br />
Regev, O. and N. Nisan (1998) “The POPCORN Market – An Online Market for Computational Resources,” in<br />
Proceedings of the 1st International Conference on Information and Computation Economies, pp. 148-157. New York,<br />
USA.<br />
Sanghavi, S. and B. Hajek (2004) “Optimal allocation of a divisible good to strategic buyers,” in Proceedings of the<br />
43rd IEEE Conference on Decision and Control. Atlantis, Paradise Island, Bahamas.<br />
Schnizler, B., D. Neumann, D. Veit, and C. Weinhardt (forthcoming) "Trading Grid Services – A Multi-attribute<br />
Combinatorial Approach," European Journal of Operational Research.<br />
Waldspurger, C., T. Hogg, B. A. Huberman, J. O. Kephart et al. (1992) "Spawn: A Distributed Computational<br />
Economy," IEEE Transactions on Software Engineering (18) 2, pp. 103-117.<br />
Wellman, M. P., W. E. Walsh, P. R. Wurman, and J. K. MacKie-Mason (2001) "Auction protocols for decentralized<br />
scheduling," Games and Economic Behavior (35) 1-2, pp. 271-303.<br />
Wolski, R., J. S. Plank, J. Brevik, and T. Bryan (2001) "Analyzing Market-Based Resource Allocation Strategies for the<br />
Computational Grid," International Journal of High Performance Computing Applications (15) 3, pp. 258-281.<br />
124
Trading Grid Services –<br />
A Multi-attribute Combinatorial Approach<br />
Björn Schnizler, Dirk Neumann, Daniel Veit, Christof Weinhardt<br />
{schnizler,neumann,veit,weinhardt}@iism.uni-karlsruhe.de<br />
<strong>Institut</strong>e of Information Systems and Management, University of Karlsruhe (TH),<br />
Germany<br />
Abstract<br />
The Grid is a promising technology for providing access to distributed high-end computational<br />
capabilities. Thus, computational tasks can be performed spontaneously<br />
by other resources in the Grid that are not <strong>und</strong>er the user’s control. However, one<br />
of the key problems in the Grid is deciding which jobs are to be allocated to which<br />
resources at what time. In this context, the use of market mechanisms for scheduling<br />
and allocating Grid resources is a promising approach toward solving these<br />
problems. This paper proposes an auction mechanism for allocating and scheduling<br />
computer resources such as processors or storage space which have multiple quality<br />
attributes. The mechanism is evaluated according to its economic and computational<br />
performance as well as its practical applicability by means of a simulation.<br />
Key words: Auctions and Bidding, Integer Programming, Combinatorial<br />
Exchange, Market Engineering, Grid<br />
1 Introduction<br />
The increasing interconnection between computers has created a vision for<br />
Grids. Within these Grids, computing resources such as processing power,<br />
storage space or applications are accessible to any participant. This accessibility<br />
has major ramifications for organizations since they can reduce costs by<br />
outsourcing nonessential elements of their IT infrastructure to various forms<br />
of service providers. Such emerging e-Utilities – providers offering on-demand<br />
access to computing resources – enable organizations to perform computational<br />
jobs spontaneously through other resources in the Grid that are not<br />
<strong>und</strong>er the control of the (temporary) user (Foster et al., 2002).<br />
Most Grid research has been devoted to the development of hard and software<br />
infrastructures so that access to resources is dependable, consistent, pervasive<br />
and inexpensive (Foster and Kesselman, 2004). The specification of open<br />
Preprint submitted to Elsevier Science 29 June 2006<br />
125
standards that defines the interactions between different computing resources<br />
across organizational entities hallmarks a significant milestone in this development.<br />
With the Open Grid Service Architecture (OGSA), which specifies<br />
f<strong>und</strong>amental middleware components for the Grid, the Grid Community has<br />
laid the fo<strong>und</strong>ation for future developments. OGSA defines computer and storage<br />
resources as well as networks, programs, databases and the like as services,<br />
i.e. network-enabled entities that provide some capability. Using resources as<br />
service paves the road for interoperability among heterogeneous computing<br />
and application environments (Foster et al., 2002). OGSA and one of its reference<br />
implementations, Globus Toolkit 1 , provide the technical infrastructure<br />
for accessing computational services over the Grid. Computers equipped with<br />
a Globus installation can easily access computational resources over the Grid,<br />
where access for the user is transparent. Resource owners can become providers<br />
by offering their services using standardized interfaces in a global directory.<br />
The global directory allocates the services to appropriate service consumers.<br />
With the implementation of Grid middleware, institutional arrangements are<br />
becoming increasingly important. In essence, current Grid middleware provides<br />
insufficient incentive to participate in the Grid. This lack of incentive<br />
stems from the fact that most Grid middleware service directories employ<br />
batch algorithms which determine the allocation based on either naive heuristics<br />
such as first-come first-serve or on idiosyncratic cost functions (Buyya<br />
et al., 2002). From an economic point of view, however, these algorithms are<br />
harmful, as they strive to maximize a system-wide performance objective such<br />
as throughput or mean response time rather than maximize aggregate value.<br />
These batch algorithms can not guarantee that the buyers who value some<br />
services will really receive them.<br />
Market-based approaches are considered to work well in Grid settings. By assigning<br />
a value (also called utility) to their service requests, users can reveal<br />
their relative urgency or costs to the service, which is subject to service usage<br />
constraints. If the market mechanism is properly defined, users may be<br />
provided incentives to express their true values for service requests and offers.<br />
This in turn marks the prerequisite for attaining an efficient allocation<br />
of services, which maximizes the sum of aggregate valuations (Schnizler et al.,<br />
2005).<br />
In recent times, the idea of incorporating market mechanisms into Grid technology<br />
has increasingly gained attention. Among others, the proposals include<br />
the establishment of Open Grid Markets, where either idle or all available resources<br />
can be traded (Shneidman et al., 2005; Lai, 2005). Despite this recent<br />
interest in market-based approaches, research regarding market mechanisms<br />
for Grid resources remains in its infancy. The canon of available market mech-<br />
1 See http://www.globus.org/ for details.<br />
2<br />
126
anisms only insufficiently copes with the requirements imposed by the Grid.<br />
This paper seeks to supply the need for a market mechanism that is Gridcompatible<br />
and can thus be used for resource <strong>management</strong> in federated environments<br />
such as an Open Grid Market.<br />
The remainder of this paper is structured as follows: the economic properties<br />
which the proposed market mechanism should satisfy are presented in<br />
section 2. In section 3, comparable efforts in designing mechanisms for Grids<br />
and related domains are discussed. Since none of these mechanisms is directly<br />
applicable, section 4 tailors a mechanism for the Grid. In section 5, the proposed<br />
mechanism is evaluated according to the properties laid out in section 2.<br />
Section 6 concludes with a brief summary and an outlook for future research.<br />
2 Desirable Properties and Requirements<br />
The objective of an adequate market mechanism for the Grid is the efficient<br />
and reliable provision of resources to satisfy demand. Economic approaches<br />
explicitly assume that users act strategically in a way that does not reveal<br />
their true valuations for resources aggregated in services. Incentive compatible<br />
mechanisms are accordingly advantageous, as they encourage users to report<br />
their true valuations. This allows the mechanism to maximize allocative<br />
efficiency (i.e. the sum of the individuals’ utilities).<br />
A critical step in designing markets is to <strong>und</strong>erstand the nature of the trading<br />
object. This paper considers services which respect resource functionalities<br />
(e.g. storage) and quality characteristics (e.g. size), dependencies and time<br />
attributes. Relying on services instead of computational resources removes<br />
many technical problems. For instance, the resource CPU may technically not<br />
be offered without an appropriate amount of hard disk space on the same<br />
computer, while a computation service offering CPU cycles already includes<br />
the complementary resources. In the remainder of the paper, it is abstracted<br />
from those technical details by treating resources and services as synonyms.<br />
The following presents the design objectives and the Grid specific requirements<br />
for the market mechanism.<br />
2.1 Design Objectives<br />
The theoretical basis for designing mechanisms has emerged from a branch<br />
of game theory called mechanism design (Milgrom, 2004). Within the scope<br />
of practical mechanism design, the primary design objective is to investigate<br />
a mechanism that has desirable properties. The following comprises common<br />
economic properties of a mechanism’s outcome:<br />
• Allocative efficiency:<br />
3<br />
127
An allocation is efficient if the sum of individual utilities is maximized. It<br />
is assumed that utility is transferable among all participants. A mechanism<br />
can only attain allocative efficiency if the market participants report their<br />
valuation truthfully. This requires incentive compatibility in equilibrium.<br />
• Incentive compatibility:<br />
A mechanism is incentive compatible, if every participant’s expected utility<br />
maximizing strategy in equilibrium with every other participant is to report<br />
its true preferences (Parkes et al., 2001).<br />
• Individual rationality:<br />
The constraint of individual rationality requires that the utility following<br />
participation in the mechanism must be greater or equal to the previous<br />
utility.<br />
• Budget balance:<br />
A mechanism is said to be budget balanced if the prices add up to zero for<br />
all participants (Jackson, 2002). In case the mechanism runs a deficit, it<br />
must be subsidized by an outside source and is therefore not feasible per-se.<br />
• Computational tractability:<br />
Computational tractability considers the complexity of computing a mechanism’s<br />
outcome. With an increasing number of participants, the allocation<br />
problem can become very demanding and may delimit the design of choice<br />
and transfer rules.<br />
It is clear that the objective allocative efficiency meets the general design<br />
goal that the mechanism designer wants to achieve, whereas the remaining<br />
categories are constraints upon the objective (Neumann, 2004).<br />
2.2 Domain-Specific Requirements<br />
In addition to those mechanism properties pertaining to the outcome, the<br />
mechanism must also account for the <strong>und</strong>erlying environment. The constraints<br />
of the market participants impose very rigid requirements upon the design:<br />
• Double-sided mechanism:<br />
A Grid middleware usually provides a global directory enabling multiple<br />
service owners to publish their services and multiple service requesters to<br />
discover them. Since a market mechanism replaces these directories, it has<br />
to allow many resource owners (henceforth sellers) and resource consumers<br />
(buyers) to trade simultaneously.<br />
• Language includes bids on attributes:<br />
Participants in the Grid usually have different requirements for the quality<br />
characteristics of Grid services and require these in different time spans. For<br />
example, a rendering job could require a storage service with at least 250 GB<br />
of free space for four hours in any slot between 9 a.m. and 4 p.m. The Grid<br />
community takes these requirements into account by defining service level<br />
agreement protocols, e.g. WS-Agreement (Ludwig et al., 2004). To facilitate<br />
4<br />
128
the adherence of these agreement protocols, a market mechanism fitting the<br />
Grid is required to support bids on multiple quality attributes of services<br />
as well as their time objectives.<br />
• Language includes combinatorial bids:<br />
Buyers usually demand a combination of different Grid services as a b<strong>und</strong>le<br />
in order to perform a task (Subramoniam et al., 2002). As such, Grid services<br />
are complementarities, meaning that participants have super-additive<br />
valuations for the services, since the sum of valuations for single services<br />
is less than the valuation for the whole b<strong>und</strong>le (v(A) + v(B) ≤ v(AB)).<br />
Suppose a buyer requires services for storage, computation and rendering.<br />
If any service, e.g. the storage service, is not allocated to him, the remaining<br />
b<strong>und</strong>le has no value for him. In order to avoid this exposure risk (i.e.<br />
receiving only a subset of the b<strong>und</strong>le), the mechanism must allow for bids<br />
on b<strong>und</strong>les.<br />
The buyer may want to submit more than one bid on a b<strong>und</strong>le as well<br />
as many that exclude each other. In this case, the resources for the b<strong>und</strong>les<br />
are substitutes, i.e. participants have sub-additive valuations for the services<br />
(v(A) + v(B) ≥ v(AB)). For instance, a buyer is willing to pay a high price<br />
for a service during the day and a low price if the service is executed at<br />
night. However, this service may only be computed once. To express this,<br />
the mechanism must support XOR 2 bids to express substitutes. For the<br />
sake of simplicity, a seller’s bid is restricted to a set of OR 3 bids. This<br />
simplification can be justified by the fact that Grid services are non-storable<br />
commodities, e.g. a computation service currently available cannot be stored<br />
for a later time.<br />
• Language includes co-allocation constraints:<br />
Capacity-demanding Grid applications usually require the simultaneous allocation<br />
of several homogenous service instances from different providers.<br />
For example, a large-scale simulation may require several computation services<br />
to be completed at one time. Research literature often refers to the<br />
simultaneous allocation of multiple homogenous services as co-allocation. A<br />
mechanism for the Grid has to enable co-allocations and provide functionality<br />
to control it. In this context, two cases must be considered: Firstly, it<br />
is desirable to limit the maximum number of service co-allocations, i.e. the<br />
maximum number of service divisions. Secondly, it may be logical to couple<br />
multiple services of a b<strong>und</strong>le in order to guarantee that these resources are<br />
allocated from the same seller and – more importantly – will be executed<br />
on the same machine.<br />
An adequate market mechanism for the Grid must satisfy these requirements<br />
stemming from the economic environment and ideally meet the design objectives.<br />
2 A XOR B (A ⊕ B) means either ∅, A, or B, but not AB<br />
3 A OR B (A ∨ B) means ∅, A, B, or AB<br />
5<br />
129
3 Related Work<br />
For instance, Waldspurger et al. (1992) and Regev and Nisan (2000) propose<br />
the application of Vickrey auctions for allocating homogenous computational<br />
resources in distributed systems. Vickrey auctions achieve truthful bidding as<br />
a dominant strategy and hence result in efficient allocations.<br />
Buyya et al. (2002) were among the first researchers to motivate the transfer<br />
of market-based systems from distributed systems to Grids. Nonetheless, they<br />
propose classical one-sided auction types which cannot account for combinatorial<br />
bids. Wolski et al. (2001) compare classical auctions with a bargaining<br />
market, coming to the conclusion that the bargaining market is superior to<br />
an auction-based market. This result is less surprising since the authors only<br />
consider classical auction formats where buyers cannot express bids on b<strong>und</strong>les.<br />
Eymann et al. (2003) introduce a decentralized bargaining system for<br />
resource allocation in Grids. In their simulation the bargaining systems work<br />
fairly well; however, bids on b<strong>und</strong>les are largely ignored.<br />
Subramoniam et al. (2002) account for combinatorial bids by providing a<br />
tâtonnement process for allocating and pricing Grid resources. Furthermore,<br />
Ng et al. (2005) propose repeated combinatorial auctions as a microeconomic<br />
resource allocator in distributed systems. Nonetheless, the resources are still<br />
considered to be standardized commodities. Standardization of the resources<br />
would either imply that the number of resources are limited compared to the<br />
number of all possible resources or that there are many mechanisms which are<br />
likely to suffer due to meager participation. Both implications result in rather<br />
inefficient allocations.<br />
Additionally, state-of-the-art mechanisms widely neglect time attributes for<br />
b<strong>und</strong>les and quality constraints for single resources. Hence, the use of these<br />
mechanisms in the Grid environment is considerably diminished. The introduction<br />
of time attributes redefines the Grid allocation problem as a scheduling<br />
problem. To account for time attributes, Wellman et al. (2001) model singlesided<br />
auction protocols for allocating and scheduling resources <strong>und</strong>er different<br />
time constraint considerations. However, the proposed approach is single-sided<br />
and favors monopolistic sellers or monopsonistic buyers in a way that allocates<br />
greater portions of the surplus. Installing competition on both sides is deemed<br />
superior, since no particular market side is systematically given an advantage.<br />
Demanding competition on both sides suggests the development of a combinatorial<br />
exchange. In research literature, Parkes et al. (2001) introduce the<br />
first combinatorial exchange as a single-shot sealed bid auction. As payment<br />
scheme, Vickrey discounts are approximated. The approach results in approximately<br />
efficient outcomes; however, it neither accounts for time nor for quality<br />
constraints and is thus not directly applicable to the Grid allocation problem.<br />
6<br />
130
Counteractively, Bapna et al. (2005) propose a family of combinatorial auctions<br />
for allocating Grid services. Although the mechanism accounts for quality<br />
and time attributes and enables the simultaneous trading of multiple buyers<br />
and sellers, there is no competition on the sellers’ side as all orders are<br />
aggregated to one virtual order. Moreover, the mechanism does not take coallocation<br />
constraints into account.<br />
In reviewing the related mechanisms according to the requirements presented<br />
in section 2, it is revealed that no market mechanism installs competition on<br />
both sides, includes combinatorial bids, allows for time constraints, manages<br />
quality constraints, or considers co-allocation restrictions. This paper intends<br />
to address these deficiencies by outlining the design of a multi-attribute combinatorial<br />
exchange for allocating and scheduling Grid services.<br />
4 MACE: A Multi-attribute Combinatorial Exchange<br />
The design of a market mechanism mainly affects two components: (i) the<br />
communication language which defines how bids can be formalized and (ii)<br />
the outcome determination by means of winner determination and pricing<br />
rules.<br />
The following auction format follows common assumptions of mechanism design:<br />
participants are assumed to be risk neutral, have linear utility functions,<br />
as well as independent private valuations and reservation prices. Hence, the<br />
sellers’ reservation prices can be linearly transformed to any partial execution<br />
of any b<strong>und</strong>le.<br />
Firstly, a bidding language will be introduced which fits the requirements<br />
specified in section 2. Furthermore, a winner determination model (allocation<br />
rule) and a family of pricing schemes will be proposed.<br />
4.1 Bidding Language<br />
Meeting the requirements specified in section 2 requires a formal bidding language<br />
that enables buyers and sellers to submit combinatorial bids, including<br />
multiple attributes and co-allocation constraints:<br />
Let N be a set of N buyers and M be a set of M sellers, where n ∈ N defines<br />
an arbitrary buyer and m ∈ M an arbitrary seller. There are G discrete resources<br />
G = {g 1 , . . . , g G } with g k ∈ G and a set of S b<strong>und</strong>les S = {S 1 , . . . , S S }<br />
with S i ∈ S and S i ⊆ G as a subset of resources. For instance, S i = {g k , g j }<br />
denotes that the b<strong>und</strong>le S i consists of two resources g k and g j , where g k could<br />
be a computation service and g j a storage service.<br />
A resource g k has a set of A k cardinal quality attributes A gk = {a gk ,1, . . . , a gk ,A k<br />
}<br />
7<br />
131
where a gk ,j ∈ A gk represents the j.th attribute of the resource g k . For instance,<br />
in the context of a Grid service, a quality attribute can be the size of a storage<br />
service.<br />
A buyer n can specify the minimal required quality characteristics for a b<strong>und</strong>le<br />
S i ∈ S with q n (S i , g k , a gk ,j) ≥ 0, where g k ∈ S i is a resource of the b<strong>und</strong>le S i<br />
and a gk ,j ∈ A gk is a corresponding attribute of the resource g k . As such, the<br />
minimal required size of a storage service can be denoted by q n (S i , g k , a gk ,j),<br />
e.g. 200GB. Accordingly, a seller m can specify the maximum offered quality<br />
characteristics with q m (S i , g k , a gk ,j) ≥ 0.<br />
For each resource g k ∈ S i , a buyer n can specify the maximum number of coallocations<br />
in each time slot with γ n (S i , g k ) ≥ 0. This means that a buyer n can<br />
limit the number of partial executions for each resource g k . Let γ n (S i , g k ) = D,<br />
if the resource g k has no divisibility restrictions, where D is a large enough<br />
constant 4 . The coupling of two resources in a b<strong>und</strong>le is represented by the<br />
binary variable ϕ n (S i , g k , g j ) where ϕ n (S i , g k , g j ) = 1 if resources g k and g j<br />
have to be allocated from the same b<strong>und</strong>le bid of a seller and ϕ n (S i , g k , g j ) = 0<br />
otherwise. It is assumed that all services offered in a b<strong>und</strong>le are located on<br />
the same machine.<br />
Resources in the form of b<strong>und</strong>les S i can be assigned to a set of maximal<br />
T discrete time slots T = {0, . . . , T − 1}, where t ∈ T specifies one single<br />
time slot. A buyer n can specify the minimum required number of time slots<br />
s n (S i ) ≥ 0 for a b<strong>und</strong>le S i . The earliest time slot for any allocatable b<strong>und</strong>le<br />
S i can be specified by e n (S i ) ≥ 0 for a buyer n and e m (S i ) ≥ 0 for a seller<br />
m; the latest possible allocatable time slot by l n (S i ) ≥ 0 for a buyer n and by<br />
l m (S i ) ≥ 0 for a seller m.<br />
A buyer n can express the valuation for a single slot for a b<strong>und</strong>le S i by<br />
v n (S i ) ≥ 0, i.e. the maximum price for which the buyer n is willing to trade.<br />
The reservation price for allocating a single slot of a b<strong>und</strong>le S i is denoted by<br />
r m (S i ) ≥ 0, which represents the minimum price for which the seller m is<br />
willing to trade.<br />
Subsequently, a buyer n can submit a set of XOR b<strong>und</strong>le bids as an order<br />
B n = {B n,1 (S 1 ) ⊕ . . . ⊕ B n,u (S j )},<br />
where u is the number of b<strong>und</strong>le bids in the order and B n,f (S i ) is defined as<br />
4 The constant D has to be greater than the total number of seller bids.<br />
8<br />
132
the tuple<br />
B n,f (S i ) =(v n (S i ), s n (S i ), e n (S i ), l n (S i ),<br />
q n (S i , g 1 , a g1 ,1), . . . , q n (S i , g G , a gG ,A j<br />
), γ n (S i , g 1 ), . . . , γ n (S i , g G ),<br />
ϕ n (S i , g 1 , g 2 ), ϕ n (S i , g 1 , g 3 ), . . . , ϕ n (S i , g 1 , g G ), . . . , ϕ n (S i , g G , g G−1 )).<br />
For example, the bid B n,1 (S 1 ) = {1, 4, 2, 10, (3000, 30), (2, 4), 0} with S 1 =<br />
{g 1 , g 2 }, g 1 = {Computation} with one attribute A g1 = {Speed}, and g 2 =<br />
{Storage} with one attribute A g2 = {Space} expresses that a buyer n wants<br />
to buy the b<strong>und</strong>le S 1 and has a valuation of v n (S 1 ) = 1 per slot for it. The<br />
requested computation service g 1 should at least be capable of providing 3000<br />
MIPS 5 , and the storage service g 2 should have at least 30 GB of available<br />
space. The computation service g 1 can split in two parts at the most, while<br />
storage service g 2 can run on four different machines simultaneously. Furthermore,<br />
the two services have no coupling requirements. The buyer requires four<br />
slots of this b<strong>und</strong>le which must be fulfilled within a time range of slots two<br />
and ten.<br />
The sellers’ orders are formalized in a way similar to the buyers’ orders. However,<br />
they do not include maximum divisibility and coupling properties and<br />
assume that the number of time slots is equal to the given time range. An<br />
order B m is defined as a concatenated set of OR b<strong>und</strong>le bids<br />
B m = {B m,1 (S 1 ) ∨ . . . ∨ B m,u (S j )},<br />
where u is the number of b<strong>und</strong>le bids. A single bid B m,f (S i ) for a b<strong>und</strong>le S i<br />
is defined as the tuple<br />
B m,f (S i ) = (r m (S i ), e m (S i ), l m (S i ), q m (S i , g 1 , a g1 ,1), . . . , q m (S i , g j , a gj ,A j<br />
)).<br />
In the following it is assumed that the bid elicitation has already taken place.<br />
4.2 Winner Determination<br />
Based upon this notation, the multi-attribute combinatorial winner determination<br />
problem with the objective of achieving allocative efficiency can be<br />
represented as a linear mixed integer program.<br />
For formalizing the model, the decision variables x n (S i ), z n,t (S i ), y m,n,t (S i ),<br />
and d m,n,t (S i ) have to be introduced. The binary variable x n (S i ) denotes<br />
whether b<strong>und</strong>le S i is allocated to buyer n (x n (S i ) = 1) or not (x n (S i ) = 0).<br />
Furthermore, the binary variable z n,t (S i ) is assigned to a buyer n and is associated<br />
in the same way as x n (S i ) with the allocation of S i in time slot t. For<br />
a seller m, the real-valued variable y m,n,t (S i ) with 0 ≤ y m,n,t (S i ) ≤ 1 indicates<br />
5 Million instructions per second (MIPS) is a measure for a computer’s processor<br />
speed.<br />
9<br />
133
the percentage contingent of b<strong>und</strong>le S i allocated to the buyer n in time slot<br />
t. For example, y m,n,t (S i ) = 0.5 denotes that 50 percent of the quality characteristics<br />
of b<strong>und</strong>le S i are allocated from seller m to buyer n in time slot t. In<br />
the previous example including the storage service with 30GB of free space,<br />
a partial allocation of 15 GB from seller m to buyer n in time slot t would<br />
lead to y m,n,t (Storage) = 0.5. The binary variable d m,n,t (S i ) is linked with<br />
y m,n,t (S i ) and denotes whether the seller m allocates b<strong>und</strong>le S i to buyer n in<br />
time slot t (d m,n,t (S i ) = 1) or not (d m,n,t (S i ) = 0).<br />
By means of these variables, the multi-attribute winner determination model<br />
can be formulated as follows:<br />
max ∑ ∑ ∑<br />
v n (S i )z n,t (S i ) − ∑ ∑ ∑ ∑<br />
r m (S i )y m,n,t (S i ) (1)<br />
n∈N S i ∈S t∈T<br />
m∈M n∈N S i ∈S t∈T<br />
s.t. ∑<br />
S i ∈S<br />
x n (S i ) ≤ 1, ∀n ∈ N (2)<br />
∑<br />
z n,t (S i ) − x n (S i )s n (S i ) = 0, ∀n ∈ N , ∀S i ∈ S (3)<br />
t∈T<br />
∑<br />
y m,n,t (S i ) ≤ 1, ∀m ∈ M, ∀S i ∈ S, ∀t ∈ T (4)<br />
n∈N<br />
The objective function (1) maximizes the surplus V ∗ which is defined as the<br />
difference between the sum of the buyer’s valuations v n (S i ) and the sum of the<br />
sellers’ reservation prices r m (S i ). Assuming bidders are truthful, the objective<br />
function reflects the goal of maximizing economic social welfare. The first<br />
constraint (2) guarantees that each buyer n can be allocated only to one<br />
b<strong>und</strong>le S i . This constraint is necessary to fulfill the XOR constraint of a buyer<br />
order. Constraint (3) ensures that for any allocated b<strong>und</strong>le S i , a buyer n<br />
receives exactly the required slots within the time set T . The constraint has<br />
thus to formulated as an equality in to order to be binding.<br />
For each time slot t, constraint (4) ensures that each seller cannot allocate more<br />
than the seller possesses. The constraints (2-4) consider the basic allocation<br />
functionality of the exchange. In designing an adequate mechanism for the<br />
Grid, quality characteristics and dependencies between resources must also to<br />
be addressed:<br />
∑<br />
z n,t (S i )q n (S i , g k , a gk ,j) − ∑<br />
S i ∋g k<br />
∑<br />
S i ∋g k m∈M<br />
y m,n,t (S i )q m (S i , g k , a gk ,j) ≤ 0,<br />
∀n ∈ N , ∀g k ∈ G, ∀a gk ,j ∈ A gk , ∀t ∈ T (5)<br />
10<br />
134
∑ ∑<br />
d m,n,t (S i ) − ∑<br />
γ n (S i , g k )z n,t (S i ) ≤ 0,<br />
S i ∋g k m∈M<br />
S i ∋g k<br />
∀n ∈ N , ∀g k ∈ G, ∀t ∈ T (6)<br />
∑<br />
ϕ n (S j , g k , g l )( ∑<br />
S j ∋g k ,g l<br />
∑<br />
ϕ n (S j , g k , g l )( ∑<br />
S j ∋g k ,g l<br />
∑<br />
S i ∋g k m∈M<br />
S i ∋g k<br />
d m,n,t (S i ) − ∑<br />
S i ∋g l<br />
d m,n,t (S i )) = 0,<br />
∀n ∈ N , ∀m ∈ M, ∀g k , g l ∈ G, ∀t ∈ T (7)<br />
d m,n,t (S i )+ ∑<br />
∑<br />
S i ∋g l m∈M<br />
d m,n,t (S i ) − 2z n,t (S j )) ≤ 0,<br />
∀n ∈ N , ∀g k , g l ∈ G, ∀t ∈ T (8)<br />
Constraint (5) guarantees that for any allocated b<strong>und</strong>le in an arbitrary time<br />
slot t, all required resources have to be fulfilled in the same slot in at least the<br />
demanded qualities. Constraint (6) ensures that a resource will be provided<br />
by at most γ n (S i , g k ) different suppliers. For simplicity, it is assumed that a<br />
good g k with restricted co-allocations is not part of further XOR concatenated<br />
bids of the buyer n.<br />
Constraints (7) and (8) account for the coupling of two resources. Constraint<br />
(7) ensures that two resources must be provided by the same seller, in case they<br />
should be coupled. This constraint alone does not suffice for coupling resources<br />
requirements, since it would be possible for two sellers to co-allocate a coupled<br />
computation service with 3000 MIPS and a storage service with 30 GB in<br />
different quality characteristics, e.g. to allocate a computation service with<br />
2998 MIPS and a storage service with 1 GB from one seller, and a computation<br />
service with 2 MIPS and a storage service with 29 GB from another. To<br />
exclude these <strong>und</strong>esirable allocations, constraint (8) imposes the restriction<br />
that coupled resources cannot be co-allocated. Simplifying the model, this<br />
also includes free-disposal resources. For instance, if the computation service<br />
with 3000 MIPS and the storage service with 30 GB are allocated from one<br />
particular seller as a b<strong>und</strong>le, another seller cannot allocate a b<strong>und</strong>le containing<br />
a rendering service and another storage service to the same buyer. However,<br />
the seller may allocate any b<strong>und</strong>le without a storage and computation service<br />
to the buyer, e.g. the rendering service alone. Furthermore, it is assumed that<br />
coupled resources are only part of one particular b<strong>und</strong>le bid B n,f (S i ) in case<br />
a buyer submits two XOR concatenated bids containing coupled resources.<br />
The time restrictions of the bids are given by:<br />
(e n (S i ) − t)z n,t (S i ) ≤ 0, ∀n ∈ N , ∀S i ∈ S, ∀t ∈ T (9)<br />
(t − l n (S i ))z n,t (S i ) ≤ 0, ∀n ∈ N , ∀S i ∈ S, ∀t ∈ T (10)<br />
11<br />
135
(e m (S i ) − t) ∑ y m,n,t (S i ) ≤ 0, ∀m ∈ M, ∀S i ∈ S, ∀t ∈ T (11)<br />
n∈N<br />
(t − l m (S i )) ∑ y m,n,t (S i ) ≤ 0, ∀m ∈ M, ∀S i ∈ S, ∀t ∈ T (12)<br />
n∈N<br />
Essentially, constraints (9)-(12) indicate that slots cannot be allocated before<br />
the earliest and after the latest time slot of either buyer (constraint (9), (10))<br />
or seller (constraint (11), (12)).<br />
Finally, the establishment of the relationship between the real valued decision<br />
variable y m,n,t (S i ) and the binary variable d m,n,t (S i ) needs to be addressed and<br />
the decision variables of the optimization problem have to be defined:<br />
y m,n,t (S i ) − d m,n,t (S i ) ≤ 0, ∀n ∈ N , ∀m ∈ M, ∀S i ∈ S, ∀t ∈ T (13)<br />
d m,n,t (S i ) − y m,n,t (S i ) < 1, ∀n ∈ N , ∀m ∈ M, ∀S i ∈ S, ∀t ∈ T (14)<br />
x n (S i ) ∈ {0, 1}, ∀n ∈ N , ∀S i ∈ S (15)<br />
z n,t (S i ) ∈ {0, 1}, ∀n ∈ N , ∀S i ∈ S, ∀t ∈ T (16)<br />
y m,n,t (S i ) ≥ 0, ∀n ∈ N , ∀m ∈ M, ∀S i ∈ S, ∀t ∈ T (17)<br />
d m,n,t (S i ) ∈ {0, 1}, ∀n ∈ N , ∀m ∈ M, ∀S i ∈ S, ∀t ∈ T (18)<br />
Constraints (13) and (14) incorporate an if-then-else constraint, i.e. if a<br />
seller m partially allocates a b<strong>und</strong>le S i to a single buyer n (y m,n,t (S i ) > 0), the<br />
binary variable d m,n,t (S i ) has to be d m,n,t (S i ) = 1 (constraint (13)); otherwise,<br />
it has to be d m,n,t (S i ) = 0 (constraint (14)). Finally, the constraints (15) - (18)<br />
specify the decision variables of the optimization problem.<br />
As multiple optimal solutions may exist, ties are broken in favour of maximizing<br />
the number of traded b<strong>und</strong>les and then at random.<br />
Example (Winner determination): Suppose there are two buyers n 1 , n 2 ,<br />
two sellers m 1 , m 2 and two services g 1 = {Computation} and g 2 = {Storage}<br />
with G = {g 1 , g 2 } each with single attributes, namely a g1 ,1 = {Speed} and<br />
a g2 ,1 = {Size}. The buyers and sellers can submit bids on the b<strong>und</strong>les S 1 =<br />
{g 1 }, S 2 = {g 2 }, and S 3 = {g 1 , g 2 }. The b<strong>und</strong>les can be allocated within a<br />
time range T = {t 0 , . . . , t 4 } of T = 5 slots. Each buyer submits a set of XOR<br />
bids (shown in table 1) and each seller a set of OR bids (see table 2).<br />
12<br />
136
N S i v n (S i ) q n (S i , g k , a gk ,j) e n (S i ) l n (S i ) s n (S i ) γ n (S i , g k ) ϕ n (S i , g i , g j )<br />
n 1 S 3 2 g 1 → 500, g 2 → 15 0 4 2 g 1 , g 2 → 1<br />
n 2<br />
S 1 3 g 1 → 400 1 4 3<br />
S 3 2 g 1 → 300, g 2 → 25 0 4 2 g 2 → 1<br />
Table 1. XOR orders of the buyers.<br />
M S i r m (S i ) q m (S i , g k , a gk ,j) e m (S i ) l m (S i )<br />
m 1 S 3 1 g 1 → 500; g 2 → 40 0 4<br />
m 2<br />
S 1 2 g 1 → 1000 0 3<br />
S 3 2 g 1 → 700; g 2 → 20 1 4<br />
Table 2. OR orders of the sellers.<br />
For instance, buyer n 2 submits two XOR concatenated bids to b<strong>und</strong>les S 1 and<br />
S 3 . The buyer has a valuation of v n2 (S 1 ) = 3 for the b<strong>und</strong>le S 1 which consists<br />
of the computation service g 1 . The service must have at least 400 GB of free<br />
space and can be allocated between the slots e n2 (S 1 ) = 1 and l n2 (S 1 ) = 4.<br />
The buyer requires s n2 (S 1 ) = 3 slots of the service and has no co-allocation<br />
restrictions.<br />
An optimal solution for the winner determination problem is an allocation<br />
of the b<strong>und</strong>les S 3 and S 1 to the buyers n 1 and n 2 with x n1 (S 3 ) = 1 and<br />
x n2 (S 1 ) = 1. Seller m 1 is part of the allocation with b<strong>und</strong>le S 3 and seller m 2<br />
with b<strong>und</strong>le S 1 . The maximized value V ∗ of the winner determination problem<br />
is V ∗ = 8.6. The corresponding schedule for this allocation is given in table 3.<br />
M S t 0 t 1 t 2 t 3<br />
m 1 S 3 n 1 : g 1 → 500, g 2 → 40 n 1 : g 1 → 500, g 2 → 40<br />
m 2 S 1 n 2 : g 1 → 400 n 2 : g 1 → 400 n 2 : g 1 → 400<br />
Table 3. Allocation schema.<br />
Buyer n 1 receives b<strong>und</strong>le S 3 = {g 1 , g 2 } from seller m 1 in time slots 0 and<br />
1. The b<strong>und</strong>le is allocated from one seller and as such the buyer’s requested<br />
coupling property is satisfied. Buyer n 2 receives requested computation service<br />
S 1 = {g 1 } in time slots 1, 2, and 3.<br />
Although n 1 does not require the entire allocated space of the storage service<br />
g 2 , a partial execution of the b<strong>und</strong>le is not possible due to the computation<br />
service g 1 requirements. B<strong>und</strong>les can only partially executed as a whole. If an<br />
isolated partial execution of single resources in a b<strong>und</strong>le would be possible,<br />
these single resources would also have to be valued. However, as resources<br />
may be complementarities or substitutes, valuation and reservation prices for<br />
a single resource of a b<strong>und</strong>le do not exist (Milgrom, 2004). As such, resources<br />
of a b<strong>und</strong>le cannot be partially executed.<br />
13<br />
137
The outcome of the winner determination problem is allocative efficient as long<br />
as buyers and sellers reveal their valuations truthfully. The incentive to set<br />
bids according to the valuation is induced by an adequate pricing mechanism<br />
introduced in the following subsection.<br />
4.3 Pricing<br />
The question how to determine payments made by participants to the exchange<br />
and vice versa after the mechanism has determined the winners is<br />
referred to as pricing problem. With respect to the objective of achieving an<br />
efficient allocation, a pricing scheme based on the well-known Vickrey-Clarke-<br />
Groves (VCG) mechanism would attain this objective (Vickrey, 1961; Clarke,<br />
1971; Groves, 1973). Moreover, VCG mechanisms are the only allocativeefficient<br />
and incentive compatible mechanisms (Green and Laffont, 1977).<br />
Hence, it is logical to turn to the VCG mechanism.<br />
The basic idea behind a VCG mechanism is to grant any participant a discount<br />
(called Vickrey discount) on that bid. The discount reflects the impact of that<br />
bid on the social welfare. A VCG mechanism is efficient, incentive-compatible,<br />
and individually rational for participants with quasi linear utility functions.<br />
However, Myerson and Satterthwaite (1983) proved that it is impossible to design<br />
an exchange which is incentive compatible, (interim) individually rational,<br />
and budget balanced that achieves efficiency in equilibrium. The theorem is<br />
comprehensive and also applies to the presented mechanism.<br />
Obviously, the VCG pricing schema cannot be applied since it runs a permanent<br />
deficit requiring outside subsidization. The VCG mechanism can, however,<br />
serve as a benchmark.<br />
VCG Pricing<br />
Let ¯N be the set of buyers and ¯M be the set of sellers who are part of<br />
the allocation (hence, x n (S i ) = 1 with n ∈ ¯N and y m,n,t (S i ) > 0 with m ∈<br />
¯M). The union of both sets is defined as W = ¯N ∪ ¯M, where w ∈ W is a<br />
participant that is part of the allocation. Let V ∗ be the maximized value of the<br />
winner determination problem and (V −w ) ∗ be the maximized value without the<br />
participant w. Thus, the Vickrey discount for a participant w can be calculated<br />
by ∆ V ICK,w = V ∗ − (V −w ) ∗ . With regards to the Vickrey discount ∆ V ICK,w ,<br />
the price p V ICK,n (S i ) for a b<strong>und</strong>le S i and a buyer n can be calculated by<br />
p V ICK,n (S i ) = v n (S i )s n (S i ) − ∆ V ICK,n , (19)<br />
and the price p V ICK,m (S i ) for a b<strong>und</strong>le S i and a seller m by<br />
p V ICK,m (S i ) = r m (S i ) ∑ n∈N<br />
∑<br />
t∈T<br />
y m,n,t (S i ) + ∆ V ICK,m<br />
. (20)<br />
α<br />
14<br />
138
N (V −n ) ∗ ¯v n (S i ) ∆ V ICK,n p V ICK,n (S i ) M (V −m ) ∗ ¯r m (S i ) ∆ V ICK,m p V ICK,m (S i )<br />
n 1 6.6 4 2 2 m 1 7.6 2 1 3<br />
n 2 2 9 6.6 2.4 m 2 4.75 2.4 3.85 6.25<br />
Table 4. Vickrey discounts and prices.<br />
As sellers submit OR concatenated bids, they can participate in the allocation<br />
with multiple b<strong>und</strong>les. Because the Vickrey discount refers to the seller’s overall<br />
impact, the discount has to be portioned among all of the seller’s successful<br />
bids. Thus the discount is divided by α, where α is the number of b<strong>und</strong>les with<br />
which a seller m is participating in the allocation.<br />
Applying the VCG pricing scheme to the example presented above (V ∗ = 8.6)<br />
results in the prices p V ICK,w (S i ) and the discounts ∆ V ICK,w shown in table 4<br />
with ¯v n (S i ) = v n (S i )s n (S i ) and ¯r m (S i ) = ∑ t∈T ,n∈N y m,n,t (S i )r m (S i ).<br />
As stated by the Myerson-Satterthwaite impossibility theorem, a VCG mechanism<br />
is efficient and individually rational but non budget balanced in an<br />
exchange. Aggregating the net payments of the example leads to a negative<br />
value with 2 + 2.4 − (3 + 6.25) = −4.85. In this case, the auctioneer has to<br />
subsidize the exchange. Naturally, such a situation cannot be sustained for a<br />
long period of time, making the VCG mechanism unfeasible.<br />
Relaxing the requirement of having an efficient allocation opens up the possibility<br />
for a second-best mechanism that is budget balanced. These ideas gave<br />
rise to the development of a k-pricing scheme which is presented in the next<br />
section.<br />
K-Pricing<br />
The <strong>und</strong>erlying idea of the k-pricing scheme is to determine prices for a buyer<br />
and a seller on the basis of the difference between their bids (Sattherthwaite<br />
and Williams, 1993). For instance, suppose that a buyer wants to purchase a<br />
computation service for $5 and a seller wants to sell a computation service for<br />
at least $4. The difference between these bids is π = $1, where π is the surplus<br />
of this transaction that can be distributed among the participants.<br />
For a single commodity exchange, the k-pricing scheme can be formalized as<br />
follows: let v n (S i ) = a be the valuation of a buyer n and r m (S i ) = b be the<br />
reservation price of the buyer’s counterpart m. It is assumed that a ≥ b, i.e. the<br />
buyer has a valuation for the commodity which is at least as high as the seller’s<br />
reservation price. Otherwise no trade would occur. The price for a buyer n and<br />
a seller m can be calculated by p(S i ) = ka+(1−k)b with 0 ≤ k ≤ 1. In regard<br />
to the above-mentioned bids for the computation service g 1 , the resulting price<br />
for both participants is p(g 1 ) = 0.5 ∗ 9 + (1 − 0.5) ∗ 2.4 = 5.7 for k = 0.5.<br />
15<br />
139
In the following, it is assumed that the number of buyers equals the number<br />
of sellers. Therefore, k is set to k = 0.5 since it favors neither the buyers’ side<br />
nor the sellers’ side.<br />
The k-pricing schema can also be applied to a multi-attribute combinatorial<br />
exchange: in each time slot t in which a b<strong>und</strong>le S i is allocated from one or<br />
more sellers, the surplus generated by this allocation is distributed among<br />
a buyer and the sellers. Suppose a buyer n receives a computation service<br />
S 1 = {g 1 } with 1000 MIPS in time slot 4 and values this slot with v n (S 1 ) = 5.<br />
The buyer obtains the computation service S 1 = {g 1 } by a co-allocation from<br />
seller m 1 (400 MIPS) with a reservation price of r m1 (S 1 ) = 1 and from seller<br />
m 2 (600 MIPS) with r m2 (S 1 ) = 2. The distributable surplus of this allocation<br />
is π n,4 (S 1 ) = 5 − (1 + 2) = 2. Buyer n gets k ∗ π n,4 (S 1 ) of this surplus, i.e.<br />
the price buyer n has to pay for this slot is p k,n,4 (S i ) = v(S 1 ) − kπ n,4 (S 1 ).<br />
Furthermore, the sellers have to divide the other part of this surplus, i.e.<br />
(1 − k)π n,4 (S 1 ). This will be done by considering each proportion a seller’s<br />
bid has on the surplus. In the example, this proportion o m,n,t (S i ) for seller m 1<br />
is o m1 ,n,4(S 1 ) = 1 and for seller m 3<br />
2 is o m2 ,n,4(S 1 ) = 2 . The price a seller m<br />
3<br />
receives for a single slot 4 is consequently calculated as<br />
p k,n,4 (S i ) = r m (S 1 ) + (1 − k)π n,4 (S 1 )o m,n,4 (S 1 ).<br />
Expanding this scheme to a set of time slots results in the following formalization:<br />
let π n,t (S i ) be the surplus for a b<strong>und</strong>le S i of a buyer n with all<br />
corresponding sellers for a time slot t:<br />
π n,t (S i ) = z n,t (S i )v n (S i ) − ∑<br />
m∈M<br />
y m,n,t (S i )r m (S i ) (21)<br />
For the entire job (i.e. all time slots), the price for a buyer n is calculated as<br />
p k,n (S i ) = x n (S i )v n (S i )s n (S i ) − k ∑ t∈T<br />
π n,t (S i ). (22)<br />
This means that the difference between the valuation for all slots (v n (S i )s n (S i ))<br />
of the b<strong>und</strong>le S i and the k-th proportion of the sum over all time slots of the<br />
corresponding surpluses is determined.<br />
The price of a seller m is calculated in a similar way: First of all, the proportion<br />
o m,n,t (S i ) of a seller m allocating a b<strong>und</strong>le S i to the buyer n in time slot t is<br />
given by<br />
o m,n,t (S i ) =<br />
y m,n,t (S i )r m (S i )<br />
∑m∈M y m,n,t (S i )r m (S i ) . (23)<br />
Having computed π n,t (S i ) and o m,n,t (S i ), the price a seller receives for a b<strong>und</strong>le<br />
16<br />
140
S i is calculated as:<br />
p k,m (S i ) = ∑ ∑<br />
y m,n,t (S i )r m (S i ) + (1 − k) ∑ ∑<br />
o m,n,t (S i )π n,t (S i ). (24)<br />
n∈N t∈T<br />
n∈N t∈T<br />
Applying this pricing scheme to the example presented above results in the<br />
prices given table 5 with<br />
¯v n (S i ) = v n (S i )s n (S i ) and ¯r m (S i ) =<br />
∑<br />
t∈T ,n∈N<br />
y m,n,t (S i )r m (S i ).<br />
In this case, the exchange does not have to subsidize the participants since it<br />
fulfills the budget balance property in a way that no payments towards the<br />
mechanism are necessary. Hence, the k-pricing schema qualifies as a candidate<br />
pricing schema for the Grid. Since the Myerson-Satterthwaite impossibility<br />
theorem is strict, the implications of the pricing schema on the allocation<br />
need to be investigated.<br />
N ¯v n (S i ) p k,n (S i ) M ¯r m (S i ) p k,n (S i )<br />
n 1 4 3 m 1 2 3<br />
n 2 9 5.7 m 2 2.4 5.7<br />
Table 5. Prices using k-Pricing with k = 0.5.<br />
5 Evaluation<br />
The presented auction formats are evaluated by means of a stochastic simulation.<br />
The winner determination problem is solved using the optimization engine<br />
CPLEX 9.1. 6 The evaluation assesses the impact of the auction schemas<br />
on the outcome requirements defined in section 2.<br />
5.1 Data Basis<br />
As a data basis, a random bid stream including Decay distributed b<strong>und</strong>les<br />
is generated. The decay function has been recommended by Sandholm (2002)<br />
because it creates hard instances of the allocation problem. At the beginning, a<br />
b<strong>und</strong>le consists of one random resource. Afterwards, new resources are added<br />
randomly with a probability of α = 0.75. This procedure is iterated until<br />
resources are no longer added or the b<strong>und</strong>le already includes the same resource.<br />
The effects that can be obtained by the Decay distribution will be amplified.<br />
Hence, the Decay function is used to create a benchmark for upper bo<strong>und</strong>s of<br />
the effects.<br />
6 CPLEX is commercial product and is currently the state of the art optimization<br />
engine.<br />
17<br />
141
As an order, buyers and sellers submit a set of bids on 10 different b<strong>und</strong>les,<br />
where a b<strong>und</strong>le is Decay-distributed from 5 possible resources. Each<br />
resource has two different attributes drawn from a uniform distribution in line<br />
with most of the quality characteristics of computer resources described by<br />
Kee et al. (2004). The time attributes are each uniformly distributed where<br />
the earliest time slot has a range of [1..3], the latest time slot is in [4..6],<br />
and the number of slots lies in [1..3]. Forty percent of the buyers’ bids have<br />
co-allocation restrictions. Sixty percent of the resources contained in these<br />
b<strong>und</strong>les have uniform distributed restrictions on the maximum number of coallocations.<br />
The number of resources which have to be allocated from the same<br />
machine (coupling) is also drawn from an uniform distribution.<br />
The corresponding valuations and reservation prices for a b<strong>und</strong>le are drawn<br />
from the same normal distribution, whereas the number of resources in a<br />
b<strong>und</strong>le and the quality characteristics affect the mean and variance of the<br />
distribution. This means the probability is greater that a buyer has a higher<br />
valuation for a storage service with 200 GB than for a storage service with 10<br />
GB of space. In any problem instance, new orders for buyers and sellers are<br />
randomly generated. Subsequently, demand and supply are matched against<br />
each other, determining the winning allocation and corresponding prices.<br />
In total, 120 different problem instances are generated, where a problem consists<br />
of 10 or 20 participants. The proportion of buyers and sellers is equally<br />
selected out of {10/10, 8/12, 12/8, 5/5, 1/4, 4/1}. In each instance, the buyers<br />
and sellers each submit one single bid.<br />
5.2 Truthful Bidders<br />
In the first treatment, it is assumed that bidders are truthful. The average<br />
results of the treatment regarding the budget balance and participant utilities<br />
are shown in table 6.<br />
Price Mechanism Budget Utility Buyers Utility Sellers Utility<br />
VCG −3.9425 2.6484 4.0928 6.7412<br />
k-price 0 1.3994 1.3994 2.7988<br />
Table 6. Average Utility and Budget using Decay distributed bids.<br />
As the Myerson-Satterthwaite theorem suggests, the budget using a VCG auction<br />
is always negative. The mechanism requires subsidization from outside the<br />
system. Note that these subsidies amount to half the total utility. Obviously,<br />
incentive compatibility is extremely expensive, as these subsidizations are required<br />
to motivate the buyers and sellers to truthfully report their valuations<br />
and reservation prices.<br />
By applying a k-price auction, the budget deficit can be converted into a mech-<br />
18<br />
142
anism, which is independent from outside subsidies. Regarding its formal construction,<br />
the k-price auction is budget balanced; however, the average total<br />
utility shrinks by the amount of the budget deficit (including a small approximation<br />
increment). The share between buyers’ and sellers’ surplus remains<br />
unchanged. In the k-pricing schema, however, the utilities of the buyers and<br />
sellers are significantly smaller than in the VCG auction and denote a measure<br />
for the incentive compatibility loss.<br />
As table 6 shows, the sellers’ utilities in a VCG auction are higher than those<br />
of the buyers. Regarding the data basis, sellers expect to offer more available<br />
time slots than buyers request. Buyers submit bids on a number of time slots<br />
s n (S i ) between a time range e n (S i ) and l n (S i ). The number of requested slots<br />
is less than the number of allocatable slots, i.e. s n (S i ) ≤ l n (S i ) − e n (S i ). A<br />
seller, however, offers a service for the entire time range. As the time ranges<br />
of the buyers and sellers are drawn from the same distribution, the number<br />
of tradeable slots a seller offers is greater than the buyer’s requested slots.<br />
The greater the number of tradeable slots, the higher the impact of the bid in<br />
the VCG mechanism. As such, the sellers’ average discounts are higher than<br />
those of the buyers; therefore the sellers have a higher average utility than the<br />
buyers.<br />
Because the simulation assumes truthful bidding, this rather naive analysis<br />
is only accurate for the VCG mechanism. If the pricing rule is changed – for<br />
example from the VCG mechanism to the k-price auctions – strategic bidding<br />
behavior may change since the k-price auction is not incentive compatible.<br />
5.3 Manipulating Bidders<br />
In a second treatment, it is assumed that buyers and sellers using the k-pricing<br />
schema misrepresent their true valuation or reservation price respectively. Only<br />
simple misrepresentations by β% are considered, where λ% of the buyers and<br />
sellers reduce or increase their reported values by β%. Instead of observing only<br />
symmetric Nash-equilibria as in the analysis by Parkes et al. (2001), where<br />
buyers and sellers either misrepresent their valuation or reservation price by 0<br />
or by β%, the ratio of misrepresenting buyers and sellers to the total number<br />
of participants is also varied. A ratio of β = 50%, for instance, denotes that<br />
50% of the buyers and sellers misrepresent their valuations and reservation<br />
prices by λ%, while the other 50% report truthfully.<br />
By exploring the joint strategy space (i.e. varying the share of misrepresentative<br />
and truthful participants as well as the percentage of misrepresentation)<br />
the average efficiency loss for the k-pricing schema can be analyzed: Let ¯n ∈ ¯N<br />
be the buyers and ¯m ∈ ¯M be the sellers who are part of the allocation in a<br />
truthful bidding scenario, i.e. λ = β = 0. Furthermore, let ˆn ∈ ˆN be the buyers<br />
and ˆm ∈ ˆM be the sellers of the allocation having manipulating bidders<br />
19<br />
143
(λ ≥ 0, β ≥ 0). Subsequently, the efficiency loss EL can be calculated as the<br />
difference between the true preferences of the participants who are part of the<br />
allocation in both the truthful and manipulating scenarios:<br />
∑<br />
EL = s¯n (S i )v¯n (S i ) +<br />
∑¯n∈ ∑ ∑ ∑ ∑<br />
y ¯m,¯n,t (S i )r ¯m (S i )−<br />
¯N S i ∈S<br />
¯m∈ ¯M ¯n∈ ¯N S i ∈S t∈T<br />
∑<br />
( sˆn (S i )vˆn (S i ) +<br />
∑ˆn∈ ∑ ∑ ∑ ∑<br />
y ˆm,ˆn,t (S i )r ˆm (S i )).<br />
ˆN S i ∈S<br />
ˆm∈ ˆM ˆn∈ ˆN S i ∈S t∈T<br />
Figure 1 shows the percentage efficiency loss and the percentage decrease<br />
of the number of successful bidders with a varying number of manipulating<br />
participants as a function of the manipulation factor. For instance, if β = 40%<br />
of the participants manipulate their valuations and reservation prices by λ =<br />
6%, the resulting efficiency loss is 15%. It is apparent that the efficiency loss<br />
is higher when more participants use manipulation and thereby increase the<br />
manipulation factor. With a manipulation factor greater than λ = 25%, the<br />
efficiency curves stagnate. This stagnation results from the fact that either<br />
non-manipulating bidders are the only participants of the allocation (in case<br />
of β < 100%) or no one is allocated further (in case of β = 100%). The<br />
valuation and reservation prices with a manipulation factor λ > 25% are so<br />
low – respectively so high – that no bid from the manipulating participants is<br />
allocated. This means that no manipulated offer for a b<strong>und</strong>le can be matched<br />
with any manipulated request. This effect is supported by the percentage<br />
decrease of the number of successful bidders by manipulating participants. For<br />
instance, if β = 20% of the participants manipulate their bids by λ = 21%,<br />
the number of allocated bids is 17% smaller than in the non-manipulating<br />
scenario. Furthermore, if all participants are manipulating by λ = 30%, the<br />
size decreases by 100%, i.e. the set of allocated participants is empty. The<br />
curves recurrently stagnate with a manipulation factor greater than λ = 25%.<br />
Effiency loss<br />
100<br />
90<br />
80<br />
70<br />
60<br />
50<br />
40<br />
30<br />
20<br />
10<br />
0<br />
0<br />
0,1<br />
0,2<br />
0,4<br />
0,6<br />
0,7<br />
0,8<br />
1<br />
0 0,03 0,06 0,09 0,12 0,15 0,18 0,21 0,24 0,27 0,3<br />
Manipulation factor<br />
Decrease of successful participants<br />
100<br />
90<br />
80<br />
70<br />
60<br />
50<br />
40<br />
30<br />
20<br />
10<br />
0<br />
0<br />
0,1<br />
0,2<br />
0,4<br />
0,6<br />
0,7<br />
0,8<br />
1<br />
0 0,03 0,06 0,09 0,12 0,15 0,18 0,21 0,24 0,27 0,3<br />
Manipulation factor<br />
Fig. 1. Efficiency loss (left) and decrease of the number of successful bidders (right)<br />
by manipulating participants.<br />
In another analysis, the aggregated average utility gain of manipulating partic-<br />
20<br />
144
ipants is measured. This gives information on whether or not the total utility<br />
of participants can be improved through manipulation. The utility which participants<br />
can gain by manipulation only depends on the prices they have to<br />
pay and is thus calculated as<br />
UG =<br />
∑ ∑ˆn∈ ˆN S i ∈S<br />
ˆpˆn (S i ) + ∑<br />
∑<br />
ˆm∈ ˆM S i ∈S<br />
ˆp ˆm (S i ) − (<br />
∑¯n∈ ¯N<br />
∑<br />
S i ∈S<br />
p¯n (S i ) + ∑<br />
∑<br />
¯m∈ ¯M S i ∈S<br />
p ¯m (S i )),<br />
where ˆN and ˆM are the sets of buyers and sellers who are part of the allocation<br />
with manipulating participants and ˆpˆn (S i ) and ˆp ˆm (S i ) are the corresponding<br />
prices. Sets ¯N and ¯M denote the participants of the allocation in the nonmanipulated<br />
scenario with p¯n (S i ) and p ¯m (S i ) as their prices.<br />
The percentage utility gain of manipulating participants in the k-price auction<br />
is shown in figure 2. The total utility of the participants can be increased<br />
by manipulation; at the highest point where all participants manipulate by<br />
λ = 6%, the utility gain is more than 26% compared to the non-manipulating<br />
scenario. Moreover, small manipulation factors between λ = 1% and λ = 9%<br />
always result in a positive utility gain. However, participants have a negative<br />
utility gain when the manipulation factor is greater than λ = 13%. In the worst<br />
scenario, the utility loss is 100% if all participants manipulate by λ = 30%.<br />
Utility gain<br />
40<br />
20<br />
0<br />
-20<br />
-40<br />
-60<br />
-80<br />
-100<br />
0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1<br />
Participants manipulating<br />
0,3<br />
0,21<br />
0,12<br />
0,03<br />
20-40<br />
0-20<br />
-20-0<br />
-40--20<br />
-60--40<br />
-80--60<br />
-100--<br />
80<br />
Manipulation<br />
factor<br />
Fig. 2. Utility gain using k-pricing.<br />
In summary, the simulation has shown it reasonable to believe that participants<br />
will not strongly deviate from their truth valuations and reservation<br />
prices. Although participants’ average utility gain can be improved through<br />
manipulation, the participants increasingly risk not being executed in the<br />
auction (cf. figure 1). This risk actually increases the more participants use<br />
manipulation. The simulation result suggests that the k-pricing schema has<br />
accurate incentive properties resulting in fairly mild allocative efficiency losses.<br />
As such, the pricing schema is a practical alternative to the VCG mechanism<br />
and is highly relevant for an application in the Grid.<br />
21<br />
145
5.4 Computational Tractability<br />
As denoted in section 2, a computational tractable outcome determination<br />
is required. However, the multi-attribute combinatorial winner determination<br />
problem (MWDP) is N P-complete: consider the special case of the MWDP<br />
with multiple buyers, one seller, no quality attributes, one time slot, and<br />
no coupling and maximal divisibility restrictions. In this case, the problem<br />
is equivalent to the combinatorial allocation problem (CAP) formulated by<br />
Rothkopf et al. (1998). CAP is equivalent to the set-packing problem (SPP)<br />
on hypergraphs (Rothkopf et al., 1998) which is known to be N P-complete<br />
(Karp, 1972). Since CAP can be reduced to the MWDP, the MWDP is also<br />
N P-complete.<br />
Due to the N P-completeness of the winner determination problem, the auction<br />
schema is computationally intractable in large-scaled scenarios. For settings<br />
with a limited number of participants, however, the auction schema<br />
seems practicable. Ideally, an upper bo<strong>und</strong>ary of participants can be defined<br />
for which the problem is still computationally tractable within a meaningful<br />
timeframe. In the context of trading services, a meaningful timeframe is<br />
consistent with the maximum time limit that an allocation process may last.<br />
Experiences suggest that an allocation process that is shorter than 4 minutes<br />
is adequate.<br />
Preliminary studies with the auction schema confirmed that solving the winner<br />
determination problem can become computationally intractable, even in<br />
settings with less than 100 participants. This implies that exact solutions of<br />
the winner determination problem may require too much computation time<br />
and cannot be directly applied for the Grid market. To remedy this obstacle,<br />
we have to rely on approximations. A common way is to employ algorithms<br />
that compute solutions of the winner determination problem within a threshold<br />
r away from optimality, say 2%. Such deviations from optimal solutions are<br />
often conceived to be tolerable. However, in some cases even finding solutions<br />
within the specified threshold of 2% becomes impossible within a meaningful<br />
timeframe. In those cases the approximations need to have a termination<br />
condition that forces the algorithm to return the best solution fo<strong>und</strong> within<br />
a predefined time range. Combinatorial auction experiments have shown that<br />
these so-called anytime algorithms perform fairly well. For instance, Andersson<br />
et al. (2000) and Sandholm et al. (2005) report that feasible and approximate<br />
solutions can be determined in a fraction of time that would be required<br />
to find and prove an optimal solution.<br />
The applied standard solver CPLEX provides these functionalities. For our<br />
computational tractability analysis, we perform a series of simulations with<br />
anytime approximations for computing the solutions. The 4 minutes introduced<br />
above will serve as maximum timeframe. To compare the results, we<br />
22<br />
146
Participants<br />
Time Limit 2min Time Limit 4min No Time Limit<br />
µ 2 σ 2 CV 2 µ 4 σ 4 CV 4 µ σ CV<br />
100 11.6s 25.34s 2.18 30.9s 103.28s 3.34 38.25s 143.89s 3.76<br />
200 67.6s 50.09s 0.74 194.9s 178.78s 0.92 407.8s 598.4s 1.47<br />
300 95.4s 42.74s 0.45 307.4s 184.118s 0.6 673.6s 809.74s 1.2<br />
400 110.2s 24.06s 0.22 372.55s 165.07s 0.44 1536.8s 2117.91s 1.38<br />
500 116.58s 13.73s 0.12 374.2s 203.72s 0.54 2146.7s 2593.21s 1.21<br />
600 120s 0s 0 416.06 108.1s 0.26 3786.4s 3046.93s 0.8<br />
Table 7. CPU time required to solve the problem.<br />
also employ a simulation setting with a more optimistic limit of 2 minutes.<br />
As the anytime algorithms diverges from the theoretical optimum, we need a<br />
measure that captures the welfare loss imputed to the approximation. Nevertheless,<br />
computing the theoretical optimum may take very long, so that we use<br />
benchmark solution that lies within a 2% gap from the optimum. If CPLEX<br />
cannot find any feasible solution within the specified timeframe, the problem<br />
instance is rated as intractable. For instance, if no feasible solution is fo<strong>und</strong><br />
within 2 minutes, the problem is measured as an empty allocation with welfare<br />
of 0 and runtime of 2 minutes. This way, we include those hard instances in<br />
our analysis.<br />
In order to perform the simulation, nearly the same random order flow is<br />
used as described above. However, the time attributes are widened in order to<br />
generate harder and more realistic settings. The earliest time slots are within<br />
a range of [1..5], the latest time slots within [6..10], and the number of slots<br />
within a range of [1..5]. In the simulation, the proportions of buyers and sellers<br />
are equal and the number of participants is varied. In total, 20 treatments are<br />
computed and the results are averaged.<br />
Table 7 summarizes the runtime results of the simulations with 2 minutes<br />
time limit, 4 minutes time limit, and benchmark solution with no time limit. 7<br />
The table shows the mean µ, the standard deviation σ, and the coefficient of<br />
variation CV = σ/µ of the CPU times required to solve the winner determination<br />
problems. For example, with 200 participants the average runtime with<br />
2 minutes time limit is µ 2 = 67.6 seconds, with 4 minutes limit µ 4 = 194.9<br />
seconds, and without a time limit µ = 407.8 seconds. Note that the differences<br />
in the runtime of settings with equal number of participants are caused<br />
by the <strong>und</strong>erlying time limits. For 200 participants, several instances require<br />
more than 2 minutes, some even more than 4 minutes to find a solution within<br />
the specified gap. As CPLEX terminates after the predefined timeframe, the<br />
7 The analysis was performed on a Pentium XEON with 3.2GHz and 2GB ram<br />
using CPLEX 9.1 with a MIP gap of 0.02.<br />
23<br />
147
Participants<br />
Time Limit 2min<br />
Time Limit 4min<br />
µ 2 σ 2 CV 2 µ 4 σ 4 CV 4<br />
100 0.09% 0.33% 3.71 0.03% 0.24% 8.01<br />
200 0.27% 0.57% 2.11 0.06% 0.23% 3.67<br />
300 0.86% 0.94% 1.10 0.1% 0.36% 3.59<br />
400 1.85% 3.71% 2 0.31% 0.35% 1.13<br />
500 11.96% 31.14% 2.6 0.55% 0.74% 1.33<br />
600 21.59% 39.27% 1.82 13.79% 33.82% 2.45<br />
Table 8. Percentage Welfare Loss<br />
average runtime in the 2 minutes setting is less than the runtime required for<br />
finding the benchmark solution.<br />
With an increasing number of participants, CPLEX’s average execution time<br />
grows rapidly. This is most obvious in the benchmark settings with no time<br />
limit. For 500 participants, CPLEX requires nearly 2 hours to find a solution<br />
for some instances. In some settings with 600 participants, CPLEX even fails in<br />
finding a benchmark solution within 2 hours. In those cases, the simulation is<br />
stopped and the obtained results are neglected. More precisely, in our analysis<br />
this status occurred in 8 out of 20 instances. As these cases do not produce<br />
benchmark solutions, the obtained results with 600 participants may only have<br />
marginal significance.<br />
Overall, the runtime analysis shows that the winner determination problem is<br />
computationally intractable without the introduction of time limits. Apparently,<br />
delimiting the approximation produces solutions in a meaningful time,<br />
i.e. less than 4 minutes.<br />
In settings with limited time, however, the quality of the (suboptimal) solutions<br />
and accordingly its impact on the welfare changes. In the following the<br />
welfare effect due to the approximations is studied in detail. For this, we define<br />
a welfare loss higher than 5% as unacceptable for our settings, as we already<br />
have approximate solutions via the applied 2% gap.<br />
Table 8 summarizes the welfare loss of the time limited scenarios compared<br />
to the benchmark solution. For instance, for 300 participants the average welfare<br />
loss in the 2 minutes setting is µ 2 = 0.86% and in the 4 minutes limit<br />
µ 4 = 0.1%. With an increasing number of participants, the welfare loss becomes<br />
higher. On the one hand, this is reasoned by the fact that the problem<br />
instances become harder to solve within the given time frame. On the other<br />
hand, feasible solutions cannot always be fo<strong>und</strong> in time which results in a welfare<br />
loss for these infeasible instances of 100%. This explains the big increase<br />
24<br />
148
of the welfare loss from 400 to 500 participants in the 2 minutes settings –<br />
respectively, from 500 to 600 participants in the 4 minutes settings. Regarding<br />
the simulations with a limit of 2 minutes, the welfare loss exceeds our<br />
5% threshold for more than 400 participants. With a 4 minutes limit, this<br />
threshold is exceeded in settings with more than 500 participants.<br />
The analysis of the welfare loss shows, that the anytime algorithm achieves<br />
fairly efficient results: With a 2 minutes time limit, the welfare loss is acceptable<br />
for scenarios with up to 400 participants. For settings with up to 500<br />
participants, a time limit of 4 minutes entails reasonable results. However, the<br />
applied time limits become useless for settings with more than 500 participants.<br />
For such cases, the use of other approximations has to be considered.<br />
To anchor the problem to real Grid applications and to provide insights to<br />
practitioners, we studied the characteristics of PlanetLab 8 and compared<br />
them to our computational results. Essentially, PlanetLab is a testbed for networking<br />
and distributed computing among several scientific institutes worldwide.<br />
The network is state-of-the-art and currently the largest infrastructure<br />
for sharing computational Grid services. The user information was obtained<br />
using the data provided by the All-Pairs-Pings project 9 . We collected and analyzed<br />
the available data from November 2005 until March 2006. As a result,<br />
PlanetLab has currently µ = 170.89 active machines on average which can be<br />
used to perform computational jobs (with a standard deviation of σ = 21.33).<br />
The coefficient of variation (CV) for the values CV = σ/µ = 0.12 shows that<br />
the variability in reference to the size of the mean is low. Thus, the determined<br />
average constitutes to be a stable value.<br />
Transferring the PlanetLab characteristics to the proposed auction schema<br />
implies 170 sellers on average. Assuming an equal number of buyers and sellers,<br />
the average number of participants in the PlanetLab scenario is 340. With<br />
respect to the above presented results, the application of a 2 minutes time<br />
limit is sufficient to fulfill PlanetLab resource bids in a meaningful time. In<br />
cases the bo<strong>und</strong>ary raises up to 500 participants, a time limit of 4 minutes<br />
is preferable. For larger settings, however, alternative approaches have to be<br />
evaluated.<br />
6 Conclusion<br />
The paper at hand proposes the derivation of a multi-attribute combinatorial<br />
exchange for allocating and scheduling services in the Grid. In contrast to<br />
other approaches, the proposed mechanism accounts for the variety of services<br />
8 See http://www.planet-lab.org/ for details.<br />
9 Every 15 minutes, all registered nodes are contacted to check their availability<br />
status. See http://ping.ececs.uc.edu/ping/ for details.<br />
25<br />
149
y incorporating time and quality as well as coupling constraints. The mechanism<br />
provides buyers and sellers with a rich bidding language, allowing for the<br />
formulation of b<strong>und</strong>les expressing either substitutabilities or complementarities.<br />
The winner determination problem maximizes the social welfare of this<br />
combinatorial allocation problem. The winner determination scheme alone,<br />
however, is insufficient to guarantee an efficient allocation of the services. The<br />
pricing scheme must be constructed in a way that motivates buyers and sellers<br />
to reveal their true valuations and reservation prices. This is problematic in<br />
the case of combinatorial exchanges, since the only efficient pricing schedule,<br />
the VCG mechanism, is not budget-balanced and must be subsidized from<br />
outside the mechanism.<br />
This paper develops a new pricing family for a combinatorial exchange, namely<br />
the k-pricing rule. In essence, the k-pricing rule determines the price such that<br />
the resulting surpluses to the buyers and sellers divide the entire surplus being<br />
accrued by the trade according to the ration k. The k-pricing rule is budgetbalanced<br />
but cannot retain the efficiency property of the VCG payments.<br />
As the simulation illustrates, the k-pricing rule does not rigorously punish<br />
inaccurate valuation and reserve price reporting. Buyers and sellers sometimes<br />
increase their individual utility by cheating. This possibility, however, is only<br />
limited to mild misreporting and a small number of strategic buyers and sellers.<br />
If the number of misreporting participants increases, the risk of not being<br />
executed in the auction rises dramatically. As a result, the k-pricing schema<br />
is a practical alternative to the VCG mechanism and highly relevant for an<br />
application in the Grid. The runtime analysis shows that the auction schema is<br />
computationally very demanding. However, the use of approximated solutions<br />
achieves adequate runtime results and fairly mild welfare losses for up to 500<br />
participants. Comparing these results with an existing Grid testbed evinces<br />
the practical applicability of the proposed auction.<br />
This paper is a step towards <strong>und</strong>erstanding the effect and strength of different<br />
multi-attribute combinatorial exchange mechanisms on efficiency. Contributions<br />
include detailing a realistic market definition for Grid services, deriving<br />
the domain-specific requirements from the Grid, defining a winner determination<br />
scheme that meets these requirements, exploring adequate pricing schemes<br />
that contain the right incentives, and evaluating the market mechanism by<br />
means of a simulation.<br />
Future research needs to consider alternative heuristics that simplify the winner<br />
determination problem. Nature-inspired algorithms such as genetic algorithms<br />
may be adequate for an application in the service market. A further<br />
step towards a functioning Open Grid Market would be to confront the mechanism<br />
with real data in a pilot run.<br />
26<br />
150
Acknowledgements<br />
This work has partially been f<strong>und</strong>ed by EU IST progamme <strong>und</strong>er grant 003769<br />
“CATNETS”. The authors are grateful to the anonymous reviewers for the<br />
constructive comments and suggestions that significantly improved the paper.<br />
References<br />
Andersson, A., Tenhunen, M., Ygge, F., 2000. Integer Programming for Combinatorial<br />
Auction Winner Determination. In: ICMAS ’00: Proceedings of<br />
the Fourth International Conference on MultiAgent Systems. IEEE Computer<br />
Society, Washington, DC, USA, pp. 39–46.<br />
Bapna, R., Das, S., Garfinkel, R., Stallaert, J., 2005. A Market Design for<br />
Grid Computing. Tech. rep., Department of Operations and Information<br />
Management, University of Connecticut.<br />
Buyya, R., Abramson, D., Giddy, J., Stockinger, H., 2002. Economic Models<br />
for Resource Management and Scheduling in Grid Computing. The Journal<br />
of Concurrency and Computation: Practice and Experience 14 (13-15),<br />
1507–1542.<br />
Clarke, E., 1971. Multipart pricing of Public Goods. Public Choice 2, 19–33.<br />
Eymann, T., Reinicke, M., Ardaiz, O., Artigas, P., de Cerio, L. D., Freitag,<br />
F., Messeguer, R., L. Navarro, D. R., 2003. Decentralized vs. Centralized<br />
Economic Coordination of Resource Allocation in Grids. In: Proceedings of<br />
the 1st European Across Grids Conference.<br />
Foster, I., Kesselman, C., 2004. The Grid 2 - Blueprint for a New Computing<br />
Infrastructure. Vol. 2. Elsevier.<br />
Foster, I., Kesselman, C., Nick, J., Tuecke, S., 2002. Grid Services for Distributed<br />
System Integration. IEEE Computer 35 (6), 37–46.<br />
Green, J., Laffont, J.-J., 1977. Characterization of Satisfactory Mechanisms<br />
for the Revelation of Preferences for Public Goods. Econometrica 45 (2),<br />
427–438.<br />
Groves, T., 1973. Incentives in teams. Econometrica 41, 617–631.<br />
Jackson, M. O., 2002. Mechanism Theory. In: Encyclopedia of Life Support<br />
Systems. Eolss Publishers, Oxford ,UK.<br />
Karp, R. M., 1972. Reducibility among Combinatorial Problems. In: Miller,<br />
R. E., Thatcher, J. W. (Eds.), Complexity of Computer Computations.<br />
Plenum Press, pp. 85–103.<br />
Kee, Y.-S., Casanova, H., Chien, A., 2004. Realistic Modeling and Synthesis<br />
of Resources for Computational Grids. In: ACM Conference on High<br />
Performance Computing and Networking (SC2004). ACM Press.<br />
Lai, K., 2005. Markets are Dead, Long Live Markets. Tech. Rep. 0502027, HP<br />
Labs.<br />
Ludwig, H., Dan, A., Kearney, B., 2004. Cremona: An Architecture and Library<br />
for Creation and Monitoring of WS-Agreements. In: Proceedings of<br />
27<br />
151
the 2nd International Conference on Service Oriented Computing (ICSOC<br />
2004). pp. 65–74.<br />
Milgrom, P., 2004. Putting Auction Theory to Work. Cambridge University<br />
Press.<br />
Myerson, R. B., Satterthwaite, M. A., 1983. Efficient Mechanisms for bilateral<br />
Trading. Journal of Economic Theory 28, 265–281.<br />
Neumann, D., 2004. Market Engineering – A Structured Design Process for<br />
Electronic Markets. Ph.D. thesis, Economics and Business Engineering.<br />
Karlsruhe, Germany, University of Karlsruhe (TH).<br />
Ng, C., Buonadonna, P., Chun, B. N., Snoeren, A. C., Vahdat, A., 2005. Addressing<br />
Strategic Behavior in a Deployed Microeconomic Resource Allocator.<br />
In: In Proceedings of the 3rd Workshop on Economics of Peer-to-Peer<br />
Systems.<br />
Parkes, D. C., Kalagnanam, J., Eso, M., 2001. Achieving Budget-Balance with<br />
Vickrey-Based Payment Schemes in Exchanges. In: Proceedings of the Seventeenth<br />
International Joint Conference on Artificial Intelligence. pp. 1161–<br />
1168.<br />
Regev, O., Nisan, N., 2000. The POPCORN Market: Online Markets for Computational<br />
Resources. Decision Support Systems 28 (1-2), 177–189.<br />
Rothkopf, M., Pekec, A., Harstad, R., 1998. Computationally Manageable<br />
Combinational Auctions. Management Science 44, 1131–1147.<br />
Sandholm, T., 2002. Algorithm for Optimal Winner Determination in Combinatorial<br />
Auctions. Artificial Intelligence 135 (1–2), 1–54.<br />
Sandholm, T., Suri, S., Gilpin, A., Levine, D., 2005. CABOB: A Fast Optimal<br />
Algorithm for Winner Determination in Combinatorial Auctions. Management<br />
Science 51 (3), 374–390.<br />
Sattherthwaite, M. A., Williams, S. R., 1993. The Bayesian Theory of the<br />
k-Double Auction. In: Friedman, D., J.Rust (Eds.), The Double Auction<br />
Market - <strong>Institut</strong>ions, Theories, and Evidence. Addison-Wesley, Ch. 4, pp.<br />
99–123.<br />
Schnizler, B., Neumann, D., Veit, D., Weinhardt, C., 2005. A Multiattribute<br />
Combinatorial Exchange for Trading Grid Resources. In: Tan, Y. (Ed.), Proceedings<br />
of the 12th Research Symposium on Emerging Electronic Markets<br />
(RSEEM), Amsterdam, Netherlands. pp. 220–240.<br />
Shneidman, J., Ng, C., Parkes, D. C., AuYoung, A., Snoeren, A. C., Vahdat,<br />
A., Chun., B., 2005. Why Markets Could (But Don’t Currently) Solve Resource<br />
Allocation Problems in Systems. In: Proceedings of Tenth Workshop<br />
on Hot Topics in Operating Systems.<br />
Subramoniam, K., Maheswaran, M., Toulouse, M., 2002. Towards a Micro-<br />
Economic Model for Resource Allocation in Grid Computing Systems. In:<br />
Proceedings of the 2002 IEEE Canadian Conference on Electrical & Computer<br />
Engineering.<br />
Vickrey, W., 1961. Counter Speculation, Auctions, and Competitive Sealed<br />
Tenders. Journal of Finance 16, 8–37.<br />
Waldspurger, C., Hogg, T., Huberman, B., Kephart, J., Stornetta, W., 1992.<br />
28<br />
152
Spawn: A Distributed Computational Economy. IEEE Transactions on Software<br />
Engineering 18 (2), 103–117.<br />
Wellman, M. P., Walsh, W. E., Wurman, P. R., MacKie-Mason, J. K., 2001.<br />
Auction Protocols for Decentralized Scheduling. Games and Economic Behavior<br />
35, 271–303.<br />
Wolski, R., Plank, J., Brevik, J., Bryan, T., 2001. Analyzing Market-Based<br />
Resource Allocation Strategies for the Computational Grid. International<br />
Journal of High Performance Computing Applications 15 (3), 258–281.<br />
29<br />
153
A Discriminatory Pay-as-Bid Mechanism for Efficient Scheduling in the Sun N1<br />
Grid Engine<br />
Jochen Stößer † , Philipp Bodenbenner † , Simon See ‡ , Dirk Neumann †<br />
† <strong>Institut</strong>e of Information Systems and Management<br />
Universität Karlsruhe (TH)<br />
Englerstr. 14, 76131 Karlsruhe, Germany<br />
lastname@iism.uni-karlsruhe.de<br />
‡ Asia Pacific Sciene & Technology Center<br />
Sun Microsystems, Inc.<br />
50 Nanyang Avenue, Singapore 639798<br />
simon@apstc.sun.com.sg<br />
Abstract<br />
Grid computing is a promising concept to increase the<br />
efficiency of existing computing systems and to cut down on<br />
IT expenses by allowing the dynamic access to computer<br />
resources across geographical and organizational bo<strong>und</strong>aries.<br />
These inter-organizational settings require a scheduling<br />
strategy for flexibly and efficiently matching resource requests<br />
to idle resources. Market-based mechanisms promise<br />
a good fit to grids’ strategic and dynamic nature by allowing<br />
resource requesters to express valuations in addition<br />
to technical metrics. The contribution of this paper<br />
is twofold: We present a discriminatory pay-as-bid market<br />
mechanism by Sanghavi and Hajek [14] and analytically<br />
show that it outperforms market-based proportional share –<br />
the currently most prominent grid market mechanism – with<br />
respect to both provider’s surplus and allocative efficiency.<br />
We further illustrate that this mechanism is not a purely theoretical<br />
construct but that it can be integrated into the Sun<br />
N1 Grid Engine, a state-of-the-art grid scheduler.<br />
1 Introduction<br />
Grid computing denotes a computing model that distributes<br />
processing across an administratively and locally<br />
dispersed infrastructure. By connecting many heterogeneous<br />
computing resources, virtual computer architectures<br />
are created, increasing the utilization of otherwise idle resources<br />
[7]. A recent report by The Insight Research Corporation<br />
projects an increase in worldwide grid spending<br />
from $1.84 billion in 2006 to $24.52 billion in 2011 [1].<br />
The Enabling Grids for E-sciencE (EGEE) project is an<br />
intriguing example for the value of grid technology in science.<br />
EGEE aims at developing a grid infrastructure for<br />
more than 240 scientific institutions in 45 countries. The<br />
EGEE grid currently consists of more than 36,000 CPUs<br />
and 5 Petabytes of storage [6].<br />
The business case for grids is illustrated by the example<br />
of Synopsys [19]. Synposys is a world leader in integrated<br />
circuit (IC) design and requires massive computer resources<br />
for its computer-aided design processes, such as regression<br />
testing. In the past, each division within Synopsys maintained<br />
its own, separate computing cluster. The full computing<br />
power of these systems, however, was only needed<br />
in rare occasions to accommodate peak loads on the system.<br />
Obviously, this mode of operation thus led to tremendous<br />
inefficiencies. Synopsys leveraged Sun Microsystem’s<br />
grid technology to connect computing resources across divisional<br />
bo<strong>und</strong>aries, thus creating a virtual pool of computing<br />
resources which can be dynamically accessed on demand.<br />
In consequence, the runtime of regression tests could be reduced<br />
from about 12 hours to 2 hours.<br />
While the grid scenario is closely related to older allocation<br />
schemes for computer resources, such as mainframe<br />
allocation, it is somewhat more general because the grid<br />
resources might be owned by different organizations. In<br />
such inter-organizational settings, scheduling of resource<br />
requests becomes a key challenge. What is needed is a set<br />
of mechanisms that enable users to discover, negotiate, and<br />
pay for the use of grid resources. Classic technical scheduling<br />
mechanisms such as first-come-first-serve or fair share<br />
are solely built on system-centric measures. Consequently,<br />
they do not take into account the strategic and dynamic situation<br />
in grids:<br />
• Scarce resources: By nature, the concept of grids is all<br />
about sharing scarce resources. Excess demand has to<br />
be distributed to these resources so as to maximize the<br />
value provided by the grid system to its users.<br />
• Decentralized control: The scarce resources are spread<br />
across organizational bo<strong>und</strong>aries. There is no central-<br />
154
ized and complete knowledge about the state and the<br />
availability of these resources but the system depends<br />
on these organizations to report and act in a goodnatured<br />
manner so as to be able to realize this value.<br />
• Self-interested agents: The resource requesters and the<br />
organizations contributing their idle resources to a grid<br />
will generally try to selfishly maximize their individual<br />
benefit from participating in the system.<br />
In the light of these characteristics, market-based mechanisms<br />
are deemed promising to provide a better fit to<br />
grids’ strategic and dynamic nature by allowing resource requesters<br />
to express valuations in addition to technical metrics.<br />
Ultimately, prices are formed which help to balance<br />
the dynamic demand and supply in grids. The system can<br />
thus induce resource requesters to report truthfully and to<br />
distribute excess demand over time. To this end, the contribution<br />
of this paper is twofold:<br />
• Mechanism design: We present a discriminatory payas-bid<br />
market mechanism by Sanghavi and Hajek [14]<br />
and analytically derive conditions <strong>und</strong>er which it outperforms<br />
market-based proportional share – the currently<br />
most prominent grid market mechanism – with<br />
respect to both provider’s surplus (Proposition 1) and<br />
allocative efficiency (Proposition 2).<br />
• Integration into Sun N1GE: We show that this mechanism<br />
is not a purely theoretical construct but that it<br />
can be integrated into state-of-the-art grid schedulers<br />
to economically enrich the current allocation logics.<br />
We illustrate the basic design considerations for the<br />
case of the N1 Grid Engine (N1GE), the scheduler of<br />
Sun Microsystem’s grid platform.<br />
This paper is structured as follows. In Section 2, we discuss<br />
related work in the field of grid scheduling, before we<br />
introduce the pay-as-bid mechanism in Section 3. We provide<br />
an in-depth analysis of this mechanism and compare it<br />
to market-based proportional share. We present two design<br />
options for how to integrate this mechanism in the Sun N1<br />
Grid Engine in Section 4. We subsequently propose extensions<br />
to the basic mechanism and discuss problems which<br />
may emerge in this context. Section 5 concludes the paper<br />
and points to future work.<br />
2 Related Work<br />
Current resource allocation schemes in grids can be<br />
distinguished into technical and market-based schedulers.<br />
Technical schedulers are based solely on system-centric<br />
measures and aim at maximizing resource utilization and/or<br />
balancing the system load. In contrast, market-based schedulers<br />
introduce economic principles to grids in order to<br />
maximize the economic value provided by such systems;<br />
auctions or negotiations explicitly involve the users in the<br />
allocation process. Such market mechanisms must be carefully<br />
tailored towards the pecularities of the application environment<br />
and the trading object. Consequently, a suite of<br />
mechanisms has been proposed for a range of grid application<br />
scenarios (see [13] and [20] for surveys).<br />
In [15], [3] and [2], the scheduling problem in grids is<br />
formalized as an NP-hard periodic combinatorial allocation<br />
problem. In [3] and [17], heuristics are developed to migitigate<br />
this computational complexity. While these mechanisms<br />
account for dependencies between multiple grid resources<br />
(e.g. CPU, memory and bandwidth), they rely on<br />
strong technical information assumptions, such as knowledge<br />
about the time constraints and the resource requirements<br />
of applications.<br />
A f<strong>und</strong>amentally different approach are mechanisms<br />
which almost continuously assign resource shares to applications<br />
based on one-dimensional input only, e.g. single<br />
values which represent the users’ valuations. Market-based<br />
proportional share is the currently most prominent proxy of<br />
such mechanisms [5, 9, 16]. With an allocation rule purely<br />
based on economic reasoning, e.g. the prominent Vickrey<br />
mechanism, all available resources would be given to the<br />
resource request with the highest valuation. From a technical<br />
viewpoint, however, avoiding starvation 1 may be an<br />
important consideration. Combining the economic and the<br />
technical viewpoint, it may be desirable to give “better” service<br />
to high-value processes but to also give at least “some”<br />
service to low-value processes in order to avoid starvation.<br />
In the remainder of this section, the Sun N1 Grid Engine<br />
(N1GE), a state-of-the-art technical scheduler, and marketbased<br />
proportional share are presented in more detail.<br />
2.1 Sun N1 Grid Engine<br />
The N1GE is a distributed resource <strong>management</strong> and<br />
scheduling system developed by Sun Microsystems [18].<br />
Being an extension of the Solaris operating system, it administers<br />
and dynamically allocates the shared pool of heterogeneous<br />
resources such as computing power, memory<br />
and licensed software within an organization. The usage of<br />
these resources is managed so as to best achieve the goals of<br />
the organization, such as productivity, timeliness and level<br />
of service. The N1GE has been employed for setting up<br />
grids comprising a size of aro<strong>und</strong> 500-2,000 CPUs.<br />
The N1GE scheduler consists of a waiting queue with<br />
pending jobs and a technical scheduler which subsequently<br />
assigns waiting jobs to idle resources (cf. Figure 1). The<br />
user submits a job together with a specification of the tech-<br />
1 In scheduling theory, starvation denotes the fact that low-priority processes<br />
are prevented from doing any progress because all resources are<br />
assigned to other higher-value processes.<br />
155
their reported valuations’ fraction of the overall reported<br />
valuation across all resource requesters: User i with reported<br />
valuation w i will receive a fraction of of<br />
w i Pn<br />
j=1 wj<br />
the available resource when a group of n users is competing<br />
for resource access. Systems using proportional share as<br />
allocation scheme have been proposed in [5], [9] and [16].<br />
The problem with market-based proportional share is that<br />
it remains allocatively inefficient which we will illustrate in<br />
Section 3.3 by means of a numerical example.<br />
In the following, an alternative mechanism is explored –<br />
a so-called “discriminatory pay-as-bid mechanism” [14] –<br />
that may improve on these present mechanisms.<br />
Figure 1: Scheduling process in the N1GE<br />
nical requirements of the job. After receiving the job requests,<br />
the scheduler places the jobs in the waiting list of<br />
pending jobs. The position of a job in the waiting list is<br />
determined by the job’s priority. This priority value is calculated<br />
by the scheduler using a pre-defined and static mix<br />
of different policies. A sample policy mix may comprise<br />
manually (by the administrator) set shares for individual<br />
users, user groups, a department or a project (also called<br />
Entitlement policy), an increase of priority for jobs which<br />
will reach their deadline soon or that have been waiting for a<br />
long time (Urgency policy). Additionally, users may be able<br />
to sort their own jobs according (Custom policy, POSIX)<br />
[4].<br />
An example policy mix can look like this<br />
P mix = W e ∗ P e + W u ∗ P u + W c ∗ P c<br />
where P mix is the dispatch priority, P e is the normalized<br />
entitlement priority (on an interval between 0 and 1) and W e<br />
is the entitlement weighting factor. P u , W u , P c and W c are<br />
defined accordingly for the urgency and custom priorities.<br />
The key drawback of technical schedulers is that static<br />
priorities are manually set up and thus do not reflect the<br />
fluctuating demand in the system, thus leading to inefficient<br />
allocations from an economic viewpoint. To alleviate<br />
this problem and to increase efficiency, proportional share<br />
mechanisms have been introduced to grid systems.<br />
2.2 Proportional Share<br />
Proportional share allows for resource distribution with<br />
shares of unequal size for different users accounting for<br />
varying importance among them. Whereas scheduling according<br />
to pre-set, fixed shares for different users remains<br />
technical, market-based proportional share mechanisms dynamically<br />
base the resource share on the users’ reported<br />
valuations, their “bids”. The total amount of available resources<br />
is distributed among the requesters according to<br />
3 A Discriminatory Pay-as-Bid Mechanism<br />
Let vector w = (w 1 , . . . , w n ) represent the positive<br />
bids of the users. The users receive shares x =<br />
(x 1 , . . . , x n ), x j ∈ R 0 ∑ n<br />
+,<br />
j=1 x j = 1, of the perfectly<br />
divisible good. These shares are calculated according to the<br />
pre-specified allocation mechanism τ. Thus, x i = τ i (w) is<br />
the quantity user i is allocated for a given bid vector w.<br />
As is common in mechanism design, the users are assumed<br />
to have quasi-linear utility functions: U i (x) =<br />
v i x i − c(x i ), with v i ∈ R + and linear price function<br />
c(x i ) = p i x i where p i is user i’s unit price, i.e. the price<br />
user i would have to pay if she got the whole resource unit<br />
(x i = 1). Let U P (x) be the provider’s utility function.<br />
For evaluating and comparing market mechanisms, we<br />
first need to define the user behavior, i.e. the users’ reporting<br />
of w, and a metric. For the former, we choose the<br />
widely used solution concept of Nash equilibria. In a Nash<br />
equilibrium w NE , no user i can benefit by unilaterally deviating<br />
from wi<br />
NE . We interpret Nash equilibria as the final<br />
outcome of an iterative process. After each stage, the requesters<br />
can adjust their bids based on feedback about the<br />
other requesters’ bids. Ultimately, we assume that every<br />
user knows the vector v = (v 1 , . . . , v n ) consisting of all<br />
users’ valuations.<br />
A common metric for a mechanism’s performance is<br />
its performance ratio in its Nash equilibrium. The performance<br />
ratio of mechanism τ is defined as<br />
∑<br />
U(τ(w NE ))<br />
i<br />
U ∗ =<br />
U i(τ(w NE )) + U P (τ(w NE ))<br />
U ∗ ,<br />
i.e. the worst-case ratio of the social welfare generated by<br />
the specific mechanism if all bidders play their Nash strategy<br />
wi NE divided by the theoretical optimum U ∗ .<br />
A key issue when considering the use and (probably<br />
more importantly) the usability of markets is the question<br />
of how users come up with their valuation functions and interact<br />
with the market, i.e. express their valuations in order<br />
to eventually arrive at some sort of equilibrium, such as a<br />
Nash equilibrium. Human users may be released from the<br />
156
urden of having to issue requests and offers manually. Instead,<br />
software agents may serve so as to hide the grids’<br />
complexity from human users by automatically trading resources<br />
based on the current resource consumption of applications<br />
and configurable bidding rules which automatically<br />
derive corresponding valuations [10, 12].<br />
From a mechanism design perspective, we are looking<br />
for a mechanism with maximum performance ratio, i.e. the<br />
mechanism that maximizes the market’s (and thus the grid<br />
system’s) value across all users. Sanghavi and Hajek [14]<br />
propose an allocation mechanism τ sh and show that it generates<br />
the optimal performance ratio. While this mechanism<br />
has been proposed for the allocation of network bandwidth,<br />
the setting essentially generalizes to the allocation of any divisible<br />
resource. In the remainder of this section, we will introduce<br />
this mechanism and compare it to the market-based<br />
proportional share mechanism with respect to provider’s<br />
revenue and overall welfare. In the following section, we<br />
will then discuss possibilities of how such mechanisms can<br />
be integrated into state-of-the-art grid schedulers.<br />
3.1 Two Users<br />
For a scenario with two bidders l (low bidder) and h<br />
(high bidder), τ sh allocates shares of the perfectly divisible<br />
resource as follows:<br />
τl sh (w l , w h ) = w l<br />
and τh<br />
sh (w l , w h ) = 1 − w l<br />
2w h 2w h<br />
Allocation scheme τ sh is complemented by a so called payas-bid<br />
pricing scheme such that c(x i ) = p i x i = w i , i =<br />
l, h. For two buyers, the worst case performance ratio of τ sh<br />
adds up to 87.5% when both buyers have linear valuation<br />
functions [14]. In comparison the worst case efficiency of<br />
the proportional share mechanism is 82.84% [8].<br />
Assume the low bidding user l has a quasi-linear valuation<br />
function U l (x l ) = v l x l − w l with v l ∈ R + . Further<br />
assume the high bidding user h to be characterized by utility<br />
function U h (x h ) = v h x h − w h with v h ∈ R + , v h ≥ v l .<br />
Lemma 1. In the Nash equilibrium w sh of the pay-as-bid<br />
mechanism τ sh , user l bids wl<br />
sh = v2 l<br />
2v h<br />
and receives a share<br />
of τl sh (w sh ) = v l<br />
2v h<br />
, whereas user h bids wh<br />
sh = v l<br />
2 , thus<br />
receiving τh<br />
sh(wsh<br />
) = 1 − v l<br />
2v h<br />
.<br />
Proof. In the Nash equilibrium with bid vector w sh ,<br />
∂U i(τ sh (w sh ))<br />
∂wi<br />
sh<br />
∂U l (τ sh (w sh ))<br />
∂wl<br />
sh<br />
= 0, i = l, h. Consequently,<br />
=<br />
v l<br />
2w sh<br />
h<br />
− 1 = 0 ⇔ w sh<br />
h<br />
= v l<br />
2<br />
and ∂U h(τ sh (w sh ))<br />
= v hw sh<br />
∂wh<br />
sh<br />
l<br />
− 1 = 0 ⇔ w sh<br />
2(wh sh)2<br />
l<br />
=<br />
(<br />
2 vl<br />
) 2<br />
v h 2 ⇔ w<br />
sh<br />
l<br />
= v2 l<br />
2v h<br />
. Inserting w sh in τ sh directly<br />
yields τl<br />
sh (w sh ) = v l<br />
and τh<br />
sh(wsh<br />
) = 1 − v l<br />
2v h<br />
2v h<br />
.<br />
The mechanism τ sh allocates the resource shares in such<br />
a way that, in the Nash equilibrium w sh , the low-bidding<br />
user l is pushed to zero utility:<br />
U l (τ sh (w sh )) = v l τl<br />
sh (w sh ) − wl sh v l<br />
= v l − v2 l<br />
= 0.<br />
2v h 2v h<br />
The high-bidding user h obtains utility of<br />
U h (τ sh (w sh )) = v h τh<br />
sh (w sh ) − wh sh = v h − v l ,<br />
while revenue amounts to<br />
r sh = w sh<br />
l<br />
+ w sh<br />
h<br />
= v2 l + v lv h<br />
2v h<br />
.<br />
This constitutes the provider’s utility U P (.) assuming a<br />
quasi-linear provider valuation function and zero reservation<br />
prices.<br />
The central result of our analysis is that, from a resource<br />
provider’s point of view, the pay-as-bid mechanism dominates<br />
market-based proportional share <strong>und</strong>er certain conditions<br />
both with respect to provider’s surplus and welfare.<br />
To be able to compare the results of both mechanisms,<br />
we first derive the provider’s surplus generated by marketbased<br />
proportional share in the Nash equilibrium. In doing<br />
so, we assume that, as the mechanism by Sanghavi and Hajek,<br />
the proportional share allocation rule is also complemented<br />
by the pay-as-bid pricing rule, as proposed in [5].<br />
Lemma 2. If combined with a linear uniform pricing<br />
scheme ∂ci(w)<br />
∂w i<br />
= 1 ∀i (e.g. pay-as-bid), in the Nash<br />
equilibrium w ps of the proportional share mechanism τ ps ,<br />
( ) 2<br />
user l bids w ps<br />
l<br />
= v lv h<br />
v l +v h<br />
− v<br />
vh<br />
l v l +v h<br />
and receives a<br />
share of τ ps<br />
l<br />
(w ps ) = v l<br />
v l +v h<br />
, whereas user h bids w ps<br />
h =<br />
v l<br />
(<br />
)<br />
vh<br />
2,<br />
v l +v h<br />
thus receiving τ<br />
ps<br />
h (wps ) =<br />
v h<br />
v l +v h<br />
.<br />
Proof. With proportional share, τ ps<br />
l<br />
(w) =<br />
τ ps<br />
h (w) = w h<br />
w l +w h<br />
.<br />
Thus, in the Nash equilibrium w ps ,<br />
w<br />
v ps<br />
h l<br />
(w ps<br />
h +wps<br />
w ps<br />
l<br />
w l<br />
w l +w h<br />
∂U l (τ ps (w ps ))<br />
∂w ps<br />
l<br />
and<br />
− 1 = 0 ⇔ √ v<br />
l ) 2 l w ps<br />
h<br />
= w ps<br />
l<br />
+ w ps<br />
h<br />
⇔<br />
= √ v l w ps<br />
h<br />
− wps h . Analogously, ∂U h (τ ps (w ps ))<br />
=<br />
∂w ps<br />
h<br />
w<br />
v ps<br />
l h<br />
− 1 = 0 ⇔ v<br />
(w ps<br />
h +wps l ) 2 h w ps<br />
l<br />
= (w ps<br />
l<br />
+ w ps<br />
h )2 ⇔<br />
( √vl )<br />
( )<br />
v h w ps<br />
h<br />
− wps h<br />
= v l w ps<br />
h<br />
⇔ w ps<br />
h<br />
= v<br />
vh<br />
2.<br />
l v l +v h<br />
Inserting w ps<br />
h<br />
above directly yields, w ps<br />
l<br />
= v lv h<br />
v l +v h<br />
−<br />
( )<br />
v<br />
vh<br />
2.<br />
l v l +v h<br />
Inserting w ps<br />
h<br />
and wps<br />
l<br />
into τ ps (w ps ) returns τ ps<br />
l<br />
(w ps ) =<br />
√ √<br />
vl w h −w<br />
√ h<br />
vl w h<br />
= 1 −<br />
wh<br />
vl<br />
= 1 − v h<br />
v l +v h<br />
= v l<br />
v l +v h<br />
and<br />
τ ps<br />
h (wps ) = 1 − τ ps<br />
l<br />
(w ps ) = v h<br />
v l +v h<br />
.<br />
=<br />
157
Consequently, market-based proportional share generates<br />
revenue of r ps = w ps<br />
l<br />
+ w ps<br />
h<br />
= √ v l w h = v lv h<br />
v l +v h<br />
.<br />
Based on Lemma 1 and Lemma 2, we can now state our<br />
first central result:<br />
Proposition 1. For two users with linear valuation functions<br />
with slopes v l and v h (v l , v h ∈ R + and v l ≤ v h ),<br />
in the unique Nash equilibria the discriminatory pay-as-bid<br />
mechanism generates a larger provider’s revenue than proportional<br />
share (r sh ≥ r ps ) iff<br />
Proof.<br />
( √ 2 − 1)v h ≤ v l ≤ v h .<br />
r sh − r ps = v2 l + v lv h<br />
2v h<br />
− v lv h<br />
v l + v h<br />
≥ 0<br />
⇔ (v l + v h )(v 2 l + v lv h ) − 2v l v 2 h<br />
2v h (v l + v h )<br />
≥ 0<br />
⇔ (v l + v h ) 2 ≥ 2v 2 h<br />
⇔ v l ≥ ( √ 2 − 1)v h<br />
Furthermore, as defined earlier, v h ≥ v l and thus r sh ≥<br />
r ps ⇔ ( √ 2 − 1)v h ≤ v l ≤ v h .<br />
This result is of significant importance in the context of<br />
enterprise/campus grid environments, which are the main<br />
application domains of N1GE. Especially in those scenarios,<br />
we hypothesize that the users can be assumed to have<br />
rather similar valuations for grid resources. Finally, we can<br />
use these results to also assess the allocative efficiency that<br />
is generated in the Nash equilibrium for two jobs, and in<br />
line with the results of Sanghavi and Hajek [14] we state<br />
the following proposition:<br />
Proposition 2. For two users with linear valuation functions<br />
with slopes v l and v h (v l , v h ∈ R + and v l ≤ v h ) the<br />
discriminatory pay-as-bid mechanism generates an equal<br />
or larger total welfare in the Nash equilibrium compared to<br />
the proportional share mechanism with pay-as-bid pricing,<br />
that is U(τ sh (w sh )) ≥ U(τ ps (w ps )) for all combinations<br />
(v l , v h ).<br />
Proof. From our previous results, U(τ sh (w sh )) = U l (.) +<br />
U h (.)+U P (.) = 0+v h −v l + v2 l +v lv h<br />
2v h<br />
= v2 l +2v2 h −v lv h<br />
2v h<br />
and<br />
U(τ ps (w ps )) = U l (.) + U h (.) + U P (.) = (v l τ ps<br />
l<br />
(w ps ) −<br />
w ps<br />
l<br />
) + (v h τ ps<br />
h (wps ) − w ps<br />
h ) + wps l<br />
+ w ps<br />
h<br />
= v lτ ps<br />
l<br />
(w ps ) +<br />
v h τ ps<br />
h (wps ) = v2 l +v2 h<br />
v l +v h<br />
.<br />
Thus, U(τ sh (w sh )) − U(τ ps (w ps )) ≥ 0 ⇔<br />
v 2 l +2v2 h −v lv h<br />
2v h<br />
(v l − v h ) 2 ≥ 0.<br />
− v2 l +v2 h<br />
v l +v h<br />
≥ 0 ⇔ v 2 l<br />
+ v 2 h ≥ 2v lv h ⇔<br />
Consequently, the discriminatory pay-as-bid mechanism<br />
of Sanghavi and Hajek [14] not only provides us with a better<br />
performance ratio (i.e. worst-case bo<strong>und</strong>) than proportional<br />
share, but it outperforms the latter independently of<br />
the choice of v l and v h .<br />
If considered separately from the allocation scheme,<br />
pay-as-bid pricing is a uniform pricing rule as defined in<br />
Lemma 2. However, if combined with allocation scheme<br />
τ sh , the resulting mechanism as a whole produces discriminatory<br />
unit prices; the buyer with a lower bid pays a higher<br />
unit price than the high bidder. This volume discount encourages<br />
high bidders to bid higher, and thus closer to<br />
their true valuation, compared to a scenario with uniform<br />
prices where users can potentially benefit from shading their<br />
bids downwards.The discriminatory pay-as-bid mechanism<br />
is not a “fair” allocation mechanism in that it does not allocate<br />
the resource in proportion to the submitted bids but<br />
subsidizes the high bidding users. However, this is justified<br />
by the increase in overall efficiency.<br />
3.2 n Users<br />
An extension of the above mechanism from two to n<br />
buyers was developed in [14]. This mechanism still has the<br />
property of a “volume discount”, i.e. higher bidders pay<br />
lower prices.<br />
For n buyers and a given payment vector w =<br />
(w 1 , . . . , w n ) , the following allocation rule is proposed:<br />
τ sh<br />
i (w) = w i<br />
w max<br />
∫ 1<br />
0<br />
∏<br />
j≠i<br />
(<br />
1 − s w j<br />
w max<br />
)<br />
ds<br />
with at least two w i ≥ 0 and w max being the maximum<br />
bid. This allocation rule simplifies to the optimal mechanism<br />
for two buyers given above for n = 2.<br />
In contrast to the case for two buyers, it is hard to determine<br />
an exact value for the worst case efficiency for an unlimited<br />
number of buyers. Instead, [14] calculate the interval<br />
[0.8703, 0.875] as bo<strong>und</strong>s for the worst case efficiency.<br />
The proposed mechanism is still close to the theoretical<br />
maximum worst case efficiency, i.e. 87.5%. But a guarantee<br />
that mechanism τ ∗ is the optimal one can no longer be<br />
given.<br />
Even if machines are assumed to be obedient, a “lying<br />
auctioneer” could be a problem in this mechanism (cf.<br />
[11]). The allocation rule is based on the the highest bid,<br />
w max , which is not publicly known and could thus be manipulated<br />
by the provider to change the allocation in his favor.<br />
Therefore, in practice it might be necessary to somehow<br />
publish and verify w max .<br />
158
3.3 Numerical Example<br />
In this section, proportional share and the discriminatory<br />
pay-as-bid mechanism are compared concerning their allocation<br />
and the resulting efficiencies by means of a simple<br />
numerical example. Table 1 provides a brief overview of<br />
the results of this example.<br />
There are two divisions within a company – divisions L<br />
and H – which execute computational jobs on a shared pool<br />
of computing resources.<br />
Division H temporarily demands more resources than<br />
division L, which is reflected in a higher valuation for<br />
the computing resources: U L (x L ) = 2.1x L − w L and<br />
U H (x H ) = 5x H − w H . The provider’s valuation function<br />
is given by U P (x) = w L + w H . Each division sends one<br />
resource request to the central market-based scheduler. Attached<br />
to both jobs are the corresponding valuations. Now<br />
we determine the allocation of resource shares to the jobs<br />
for both market-based mechanisms.<br />
For the two jobs and the given valuation functions, the<br />
proportional share mechanism arrives at the Nash equilibrium<br />
with bid vector w ps = (w ps<br />
L , wps H<br />
) ≈ (0.437, 1.041).<br />
In this equilibrium point, x ps<br />
L<br />
= 0.296 is allocated to division<br />
L and x ps<br />
H<br />
= 0.704 is allocated to division H. None<br />
of the divisions has an incentive to unilaterally deviate from<br />
these bids. The unit prices are p ps<br />
L<br />
= pps H<br />
= 1.478. Hence,<br />
the proportional share mechanism generates an overall social<br />
welfare of U(x ps ) = 4.142. This corresponds to a<br />
performance ratio of 82.84% compared to the optimal allocation.<br />
In this optimal allocation, the high bidding user<br />
is given a resource share of 1.0 whereas the low bidder receives<br />
nothing. This allocation would create the maximum<br />
social welfare of 5.<br />
The discriminatory pay-as-bid mechanism reaches the<br />
Nash equilibrium for a bid vector w sh = (wL sh,<br />
wsh H ) =<br />
(0.441, 1.05). With these bids a resource share of x sh<br />
H =<br />
L =<br />
0.79 is assigned to division H and the remaining x ps<br />
0.21 are allocated to L. This allocation generates a total<br />
social welfare U(x sh ) = 4.391, which stands for a performance<br />
ratio of 87.82%. Applying the pay-as-bid-rule to this<br />
allocation exemplifies the “volume discount” for the high<br />
bidding division H. It pays a unit price of p ps<br />
H = 1.329<br />
whereas division L has to pay a notably higher unit price of<br />
p sh<br />
L = 2.1.<br />
Whereas in this example the provider’s revenue created<br />
by the discriminatory pay-as-bid mechanism (r sh = 1.491)<br />
is only slightly higher than with the proportional share<br />
mechanism (r ps = 1.478), overall welfare increases by almost<br />
5% from U(x ps ) = 4.142 to U(x sh ) = 4.391. While<br />
here both the high bidder and the provider benefit from<br />
switching to the pay-as-bid mechanism, as showed above,<br />
this gain mainly comes at the expense of the low bidder,<br />
whose utility becomes zero and who is thus indifferent to<br />
Optimal Bids<br />
Allocation<br />
Unit prices<br />
Utilities Users<br />
Provider’s Revenue<br />
Prop. Share<br />
w ps<br />
L<br />
w ps<br />
H<br />
x ps<br />
L<br />
Pay-as-Bid<br />
= 0.437 wsh L = 0.441<br />
= 1.041 wsh H = 1.05<br />
= 0.296 xsh L = 0.21<br />
x ps<br />
H = 0.704<br />
p ps<br />
L<br />
= pps H = 1.478<br />
U L(x ps<br />
L ) = 0.185<br />
U H(x ps<br />
H ) = 2.479<br />
xsh H = 0.79<br />
p sh<br />
L = 2.1<br />
p sh<br />
H = 1.329<br />
UL(xsh L ) = 0<br />
UH(xsh H ) = 2.9<br />
r ps = 1.478 r sh = 1.491<br />
Social Welfare U(x ps ) = 4.142 U(x sh ) = 4.391<br />
Performance<br />
ratio<br />
82.84% 87.82%<br />
Table 1: Numerical example<br />
not participating at all.<br />
In certain cases, the provider might be willing to actually<br />
sacrifice revenue in return for a higher social welfare.<br />
E.g. in the case of Synopsys, attributing the resources to the<br />
division with the higher valuation may be more important<br />
than creating revenue from internal sources. Especially in<br />
cases when provider revenue increases, one option might be<br />
to “subsidize” the lower bidder in order to convince this bidder<br />
to participate in the pay-as-bid mechanism. It will be an<br />
interesting avenue for future research to explore how such<br />
an incentive may be implemented in the pay-as-bid mechanism<br />
and how it changes the users’ strategic considerations.<br />
4 Integration into the Sun N1 Grid Engine<br />
The following section is dedicated to the integration of<br />
the market-based mechanism into the technical scheduling<br />
environment of N1GE. Two possible approaches are evaluated:<br />
1) a modular extension of N1GE, with an additional<br />
market-based policy and 2) the displacement of the current<br />
technical scheduler with the discriminatory pay-as-bid<br />
mechanism. The analysis is concluded with a comparison<br />
of both approaches in Section 4.3.<br />
4.1 The Pay-as-Bid Mechanism as Additional<br />
Policy<br />
In the following section, the application of the discriminatory<br />
pay-as-bid mechanism as a new policy in N1GE is<br />
discussed. The objective is to use the market-based mechanism<br />
as an instrument for partioning of the priority value.<br />
Analogous to the current N1GE system the users submit<br />
their jobs along with a specification document to state the<br />
job’s resource requirements. In addition the users send a<br />
159
one-dimensional bid (a single real number) to signal their<br />
valuation of the submitted job. These bids are then used by<br />
the discriminatory pay-as-bid mechanism to allocate shares<br />
of the priority value, which then again are used to determine<br />
the order in the waiting list of pending jobs. After<br />
sorting the jobs according to their assigned priority value<br />
the technical scheduler traverses the waiting list and picks<br />
a job for execution. Since the jobs can have different resource<br />
requirements the first job that fits the specifications<br />
of a currently available resource is chosen. Thus it could<br />
happen that a job that is heading the waiting list is not chosen<br />
due to its resource requirements. But still, taking up<br />
one of the front slots in the waiting list increases the possibility<br />
of early execution. Thus it is an incentive for jobs,<br />
respectively their owners, to compete for a high share of the<br />
priority value. As long as the job is not released for execution<br />
by the technical scheduler, a user can always abort his<br />
own jobs.<br />
In addition to this new policy, the currently available<br />
policies in N1GE will still be at the administrator’s disposal.<br />
This can be rather easily be achieved by adding the<br />
new discriminatory pay-as-bid policy to the existing policy<br />
mix (see figure 2). The value which is determined by the<br />
market-based mechanism is weighted and added to the total<br />
priority value of a job:<br />
ˆP mix = P mix + W sh ∗ N sh<br />
Thereby, ˆP mix is the new dispatch priority, P mix the priority<br />
value generated with the current N1GE policy mix<br />
and N sh , W sh the discriminatory pay-as-bid priority (on<br />
an interval between 0 and 1) respectively the corresponding<br />
weighting factor for this new policy.<br />
The main objective of the integration process is to preserve<br />
the economic properties of the discriminatory payas-bid<br />
mechanism. Therefore, the approach is to incorporate<br />
the mechanism mostly unchanged into a N1GE environment<br />
that is tailored to the needs of the mechanism. To<br />
guarantee a so<strong>und</strong> concurrence of N1GE system and the discriminatory<br />
pay-as-bid mechanism as a new policy, a number<br />
of extensions and modifications have to be considered<br />
on both sides. Adaptation of the discriminatory pay-as-bid<br />
mechanism requires a number of considerations in different<br />
areas. The major challenges are evaluated next and suitable<br />
solutions are proposed. Issues concerning the payment and<br />
the enforcement thereof, are not in the scope of this work<br />
and thus it plays only a tangential role in the evaluation.<br />
4.1.1 Re-evaluation of the Priority Value<br />
Recalculation of the priority values of all pending jobs is<br />
necessary whenever a new job enters the waiting list. Every<br />
new bid changes the priority values of all waiting jobs.<br />
In addition, a re-evaluation falls due for each updated bid<br />
that is received in the system. A job that leaves the waiting<br />
list for any reason does not require an update of the<br />
waiting list. The re-evaluation is processed periodically according<br />
to a pre-defined interval. A continous evaluation<br />
of newly arrived jobs would put to much additional load on<br />
the provider. Jobs, arriving at the waiting list during such<br />
an interval, are gathered and inserted into the waiting list<br />
within the next interval. In the worst case, utilisation of<br />
this periodic allocation scheme delays all arriving jobs by<br />
the pre-defined interval. This is <strong>und</strong>esirable especially for<br />
time critical and urgent jobs. Currently, the bidding interval,<br />
or scheduler interval as it is called in N1GE, is set to<br />
2-3 seconds. The testbed environments work with a 15 second<br />
interval on default. Consequently, these values seem to<br />
be reasonable references for the re-evaluation interval.<br />
4.1.2 Bid Updating<br />
Figure 2: Modified scheduling process in N1GE with the discriminatory<br />
pay-as-bid mechanism as additional policy<br />
Since re-evaluation of the priority value is indispensable in<br />
the N1GE scenario anyway, bid updating imposes no further<br />
computational effort on the scheduler. But it tremendously<br />
increases the communication effort. For each bid<br />
that is updated an additional message, carrying the single<br />
real value, has to be sent by the user and processed by the<br />
provider. However, allowing bid updates cannot interfere<br />
with the economic properties of the mechanism, since the<br />
Nash equilibrium is considered to be the final point of repeated<br />
plays. Thus, bid updates can even improve the speed<br />
of convergence since users can always adjust their bids towards<br />
the equilibrium. These adjustments are based on the<br />
behaviour of the other jobs in the queue. The bid updates<br />
can be interpreted as part of the ’myopic best response’ bidding<br />
strategy which leads to the Nash equilibrium. A received<br />
bid, respectively bid update, for a particular job is<br />
valid until another modified bid for this job is submitted by<br />
the user. Another feature of bid updates is the prevention<br />
160
of job starvation. Whenever job starvation impends the job,<br />
the user can intervene and update the bid.<br />
4.1.3 Feedback<br />
For a convergence to the Nash equilibrium point, the users<br />
need feedback on their bids to see what resource shares they<br />
received. Using this partial market information, they will<br />
adjust their bids in ”myopic best responses” to finally reach<br />
the Nash equilibrium point. Consequently, an accurate feedback<br />
is essential to preserve this equilibrium concept. By<br />
now, N1GE scheduler does not support this kind of direct<br />
feedback. The reporting tool for the waiting list, named qstat<br />
in N1GE, shows the priority value for each job. This<br />
priority value is the accumulated value for all policies that<br />
are part of the policy mix which is employed in the specific<br />
scenario. Thus, the users are not able to determine the fraction<br />
of the priority value that was calculated based on their<br />
bid. The speed of convergence to the Nash equilibrium can<br />
be further improved by publishing the whole waiting list<br />
in an anonymized form. This enables the users to monitor<br />
both the priority value and the corresponding position in the<br />
waiting list.<br />
4.2 The Pay-as-Bid Mechanism as Scheduler<br />
In this scenario the discriminatory pay-as-bid mechanism<br />
is applied as a scheduler to directly allocate the resources<br />
to the bidding users. In contrast to the scenario assumed<br />
so far, the mechanism does not calculate the respective<br />
priority value for each bidding user, but it determines<br />
the actual share of resources a user gets.<br />
The major challenge, that has to be mastered in this scenario,<br />
is the architectural limitation of N1GE, which only<br />
allows a single task to be executed on each slot at a time.<br />
Therefore resources can no longer be assumed as being perfectly<br />
divisible. There are two solution concepts to handle<br />
this restriction of N1GE. Firstly, the N1GE’s limitation<br />
can be attacked from the mathematical side. This requires<br />
a modification of the mechanism’s allocation rule to allow<br />
discrete shares only. Likewise, the problem can be tackled<br />
from the technical side. Employing virtualization tools<br />
could restore the perfect divisibility of the resources. Both<br />
solution concepts are discussed in the following – showing<br />
advantages and disadvantages of both.<br />
4.2.1 Discrete Resources<br />
The main challenge is to modify the current mechanism in<br />
such a way that it can be used for discrete resources. The<br />
calculated shares may only equal a multiple of 1 m , where<br />
m is the number of currently idle resources. By putting up<br />
a linear integer program a completely new mechanism is<br />
created. Since the discriminatory pay-as-bid mechanism is<br />
already the optimal mechanism (at least for two users) the<br />
linear integer program can not improve the current allocation<br />
and will most likely result in losses in the user’s utilities<br />
and overall efficiency. In addition, further issues have<br />
to be resolved. Firstly, the impact on the computational<br />
tractability has to be analysed. This is mainly dependent<br />
on the complexity class of such a linear integer program.<br />
Secondly, the convergence to a (Nash) equilibrium needs a<br />
more detailed examination. Taking all the considerations<br />
into account, this approach of restricting the mechanism to<br />
discrete resources is not very promising. Solving the linear<br />
integer program does hardly scale and will get computational<br />
intractable for a large number of jobs.<br />
4.2.2 Virtualization<br />
This approach tries to solve the restriction from the technical<br />
perspective. Virtualization could be the solution to<br />
the ”indivisibility constraint”. It is a technique for hiding<br />
the physical characteristics of the <strong>und</strong>erlying computing<br />
resources from the way in which other systems or end<br />
users interact with those resources. Not only homogeneous<br />
resources can be joined to form a virtual resource,<br />
with virtualization any type of resource can be combined.<br />
State-of-the-art virtualization tools, such as e.g. VMWare<br />
(http://www.vmware.com), can reach an almost perfect divisibility.<br />
Instead of directly addressing resources, the<br />
scheduler communicates with a virtual machines which<br />
again manages the resources as such. To the scheduler,<br />
the available resources appear to be one big resource. This<br />
would again fit the requirements of the discriminatory payas-bid<br />
mechanism, which is designed for allocation of a single<br />
unit of a perfectly divisble resource.<br />
In the following, a ’virtual resource’ is referred to as<br />
a single logical representation of multiple resources. Furthermore,<br />
it is assumed that virtualization is employed as<br />
a mean to enable the usage of the discriminatory pay-asbid<br />
mechanism for scheduling in N1GE. The adaptation of<br />
the mechanism to discrete resources is discarded due to the<br />
drawbacks of such an approach discussed earlier. Figure 3<br />
shows the modified N1GE scheduling process.<br />
The challenges in this application scenario differ essentially<br />
from the ones identified in the first scenario. For application<br />
of the discriminatory pay-as-bid mechanism as a<br />
direct scheduler, N1GE’s central architecture and workflow<br />
has to be radically changed. The market mechanism is not<br />
integrated as an modular extension but directly embedded<br />
into N1GE’s core structure.<br />
4.2.3 Mode of Allocation<br />
The scheduler allocates the jobs for a single time slot only.<br />
Consequently, a new allocation is calculated for all cur-<br />
161
to a fixed price, per job and time slot, paid in advance.<br />
4.2.5 Feedback<br />
Finally, a comprehensive feedback is the base for sustaining<br />
the convergence to the Nash equilibrium. The user needs to<br />
be informed about the share that he receives. This enables<br />
the user to see the impact of his bid on the resource share<br />
that is finally allocated to him. Analogous to the ”policy<br />
scenario” the provider has to put the level of allocated resource<br />
share at the user’s disposal.<br />
4.3 Implications<br />
Figure 3: Modified scheduling process in N1GE with the discriminatory<br />
pay-as-bid mechanism as direct scheduler<br />
rently running jobs after each time slot. This mode of allocation<br />
allows a flexible adjustment of short-time changes<br />
in the demand and supply situation. None of the resources<br />
is blocked for more than one time slot. The downside is<br />
an increased effort, both on the users’ and provider’s side.<br />
The users need to constantly monitor the allocation process<br />
for their running jobs and have to submit bids for every allocation<br />
interval. Otherwise, when no bid is received for<br />
a particular job, this job is suspended. Furthermore, the<br />
users are no longer able to determine the total execution<br />
time and corresponding payments for their jobs in advance.<br />
Moreover, the allocation for a single time slot burdens the<br />
provider with migration and re-prioritization of (nearly) all<br />
jobs after each allocation interval. However, this type of allocation<br />
does not affect the economic properties of the market<br />
mechanism. This can be easily proved since the method<br />
equals the initially proposed scenario by [14], in which the<br />
bandwidth of a network link was allocated for a single predefined<br />
time slot.<br />
4.2.4 Job Length<br />
A further factor that is not directly employed in the allocation<br />
scheme yet, is the job length. A major problem in this<br />
context is the determination of the job length in advance,<br />
i.e. before execution of the job. This is particularly a problem<br />
in an environment with heterogeneous resources. As<br />
already discussed in the previous section, the job length can<br />
indirectly be incorporated by allocating the resource for a<br />
single time slot. The total payment of the user is determined<br />
by summing up the fractional payments made for each allocation<br />
interval. Consequently, the job length does not need<br />
to be known in advance. Furthermore, it can be guaranteed<br />
that the user only pays for units of the resource that he really<br />
used, which adds an additional notion of fairness compared<br />
The two integration scenarios discussed in the previous<br />
subsections represent different approaches of extending the<br />
N1GE with economic features. Introducing the discriminatory<br />
pay-as-bid mechanism as an additional policy constitutes<br />
an modular extension of the N1GE. Most of the current<br />
architecture and structure retains unchanged. Solely<br />
the payment system requires major modifications. The policy<br />
mix still covers the existing policies besides the newly<br />
introduced market-based principle. In contrast, the direct<br />
scheduling scenario requires modifications and replacement<br />
of core components. Consequently, the implementation effort<br />
is much higher with this approach. In addition, the allocated<br />
resources are limited to a single type whereas the<br />
modular extension supports heterogeneous resources. On<br />
the other hand, giving the discriminatory pay-as-bid mechanism<br />
direct access to the resources entails a multitude of advantages.<br />
For instance, a waiting list is dispensable since all<br />
jobs get a share of the resource. Furthermore, the additional<br />
in-between scheduler can be left out which removes complexity<br />
from the scheduling process. Both scenarios offer<br />
promising enhancements of N1GE. The modular extension<br />
offers a quick solution for incorporation of the discriminatory<br />
pay-as-bid mechanism, whereas the higher flexibility<br />
of the second approach comes at the expense of higher implementation<br />
effort.<br />
5 Conclusion<br />
The extended model of the Sanghavi/Hajek pay-as-bid<br />
mechanism is a promising addition to the N1GE scheduler.<br />
Employing a market-based mechanism for resource allocation<br />
in grids offers new possibilities on both sides, for<br />
providers as well as for buyers. Current technical schedulers<br />
require an administrator to specify user weights based<br />
on these users relative importance, regardless of the dynamic<br />
demand and supply situation, leading to inefficiencies.<br />
To this end, the Sanghavi/Hajek pay-as-bid mechanism<br />
allows flexible reactions to changes in the demand and<br />
supply situation. Moreover, it offers an elaborated pricing<br />
162
scheme where prices reflect the current market situation and<br />
induce users to report their true valuations to the system.<br />
The administrator no longer needs to adjust the weights<br />
manually but the users can directly express the urgency of<br />
their jobs. Furthermore, the market prices can be leveraged<br />
for usage-based accounting of shared computer resources.<br />
In comparison to other market-based mechanisms, the<br />
discriminatory pay-as-bid mechanism scores with its easeof-use.<br />
In addition, it has an increased worst case performance<br />
ratio as compared to market-based proportional<br />
share and is close to the theoretical maximum. Above all,<br />
the extended model imposes a very low communicational<br />
and computational burden on the scheduling process and<br />
allows for real-time allocations.<br />
Further work has to be done on analyzing the extensions<br />
to the basic mechanism and their impact on the mechanism’s<br />
economic properties. We explicitly mentioned the<br />
need to incentivize the low bidding user to participate in the<br />
grid. It would be very interesting to actually implement the<br />
mechanism within the N1GE scheduler. This would allow<br />
us to examine the mechanism’s behavior and performance<br />
in practice. A decentralized version of the mechanism<br />
would be desirable to support decentralized waiting queues<br />
as well. This might be necessary to keep the N1GE scheduler<br />
applicable for very large clusters (> 20, 000 cores),<br />
which will be demanded in the near future.<br />
Acknowledgements<br />
We would like to thank the anonymous reviewers for<br />
their valuable comments. This research was partially supported<br />
by the EU IST programme <strong>und</strong>er grant #034286<br />
SORMA – “Self-Organizing ICT Resource Management”.<br />
References<br />
[1] Anonymous. Grid Computing: A Vertical Market Perspective<br />
2006-2011. Technical report, The Insight Research Corportation,<br />
Boonton, NJ, USA, 2006.<br />
[2] A. AuYoung, B. Chun, A. Snoeren, and A. Vahdat. Resource<br />
allocation in federated distributed computing infrastructures.<br />
Proceedings of the 1st Workshop on Operating<br />
System and Architectural Support for the On-demand IT InfraStructure,<br />
2004.<br />
[3] R. Bapna, S. Das, R. Garfinkel, and J. Stallaert. A Market<br />
Design for Grid Computing. Technical report, Technical<br />
report, Department of Operations and Information Management,<br />
University of Connecticut, 2005.<br />
[4] C. Chaubal. Scheduler Policies for Job Prioritization in<br />
the Sun N1 Grid Engine 6 System. Technical report,<br />
Sun BluePrints Online, Sun Microsystems, Inc., Santa<br />
Clara, CA, USA. http://www.sun.com/blueprints/1005/819-<br />
4325.pdf, 2005.<br />
[5] B. N. Chun and D. E. Culler. Market-based proportional<br />
resource sharing for clusters. Technical report, University<br />
of California at Berkeley, CA, USA, 2000.<br />
[6] Enabling Grids for E-sciencE (EGEE) project. Website,<br />
2007. Available online at http://www.eu-egee.org/; visited<br />
on August 29th 2007.<br />
[7] I. Foster, S. Tuecke, and C. Kesselman. The Anatomy of<br />
the Grid: Enabling Scalable Virtual Organizations. International<br />
Journal of High Performance Computing Applications,<br />
15(3):200–222, 2001.<br />
[8] R. Johari and J. N. Tsitsiklis. Efficiency Loss in a Network<br />
Resource Allocation Game. Mathematics of Operations Research,<br />
29(3):407–435, 2004.<br />
[9] K. Lai, B. Huberman, and L. Fine. Tycoon: A Distributed<br />
Market-Based Resource Allocation System. Arxiv preprint<br />
cs.DC/0404013, 2004.<br />
[10] J. K. MacKie-Mason and M. P. Wellman. Automated markets<br />
and trading agents. In L. Tesfatsion and K. L. Judd,<br />
editors, Handbook of Computational Economics.<br />
[11] R. T. Maheswaran and T. Basar. Coalition formation in proportional<br />
fair divisible auctions. In Proceedings of the Second<br />
International Joint Conference on Autonomous Agents<br />
and Multiagent Systems, pages 25–32, 2003.<br />
[12] D. Neumann, S. Lamparter, and B. Schnizler. Automated<br />
bidding for trading grid services. In Proceedings of the European<br />
Conference on Information Systems (ECIS), 2006.<br />
[13] D. Neumann, J. Stößer, and C. Weinhardt. Bridging the<br />
Adoption Gap – Developing a Roadmap for Trading Grids.<br />
Electronic Markets, 2007, forthcoming.<br />
[14] S. Sanghavi and B. Hajek. Optimal Allocation of a Divisible<br />
Good to Strategic Buyers. 43rd IEEE Conference on<br />
Decision and Control, 2004.<br />
[15] B. Schnizler, D. Neumann, D. Veit, and C. Weinhardt.<br />
Trading Grid Services-A Multi-attribute Combinatorial Approach.<br />
European Journal of Operational Research, forthcoming,<br />
2006, forthcoming.<br />
[16] I. Stoica, H. Abdel-Wahad, K. Jeffay, S. Baruah, J. Gehrke,<br />
and C. Plaxton. A proportional share resource allocation<br />
algorithm for real-time, time-shared systems. IEEE Real-<br />
Time Systems Symposium, 1996.<br />
[17] J. Stößer, D. Neumann, and A. Anandasivam. A truthful<br />
heuristic for efficient scheduling in network-centric grid os.<br />
In Proceedings of the 15 th European Conference on Information<br />
Systems (ECIS), 2007.<br />
[18] Sun Microsystems, Inc. Sun N1 Grid Engine<br />
6 User’s Guide. Santa Clara, CA, USA.<br />
http://docs.sun.com/app/docs/doc/817-6117/, 2004.<br />
[19] Sun Microsystems, Inc. Grid Computing Solutions.<br />
Website, 2007. Available online at<br />
http://www.sun.com/software/grid/; visited on August<br />
29th 2007.<br />
[20] R. Wolski, J. Plank, J. Brevik, and T. Bryan. Analyzing<br />
Market-Based Resource Allocation Strategies for the Computational<br />
Grid. International Journal of High Performance<br />
Computing Applications, 15:258–281, 2001.<br />
163
Decentralized Online Resource Allocation for Dynamic Web Service Applications<br />
Jochen Stößer, Christine Rössle, Dirk Neumann<br />
<strong>Institut</strong>e of Information Systems and Management (<strong>IISM</strong>)<br />
Universität Karlsruhe (TH)<br />
Englerstr. 14, 76131 Karlsruhe, Germany<br />
{stoesser,roessle,neumann}@iism.uni-karlsruhe.de<br />
Abstract<br />
Economic scheduling mechanisms introduce economic<br />
principles to Grids and thus promise to increase the<br />
systems’ overall utility by explicitly taking into account<br />
strategic, inter-organizational settings. Current economic<br />
scheduling mechanisms, however, do not fully satisfy the requirements<br />
of interactive Grid applications. In this paper,<br />
we introduce a video surveillance environment and show<br />
how its functionality can be encapsulated in Grid services<br />
which are traded dynamically using an economic scheduling<br />
mechanism. We suggest a mechanism from the general<br />
machine scheduling domain and showcase its applicability<br />
to the Grid context. However, the mechanism suffers from<br />
some limitations due to the distinct properties of Grids. We<br />
point to these limitations and propose extensions to the basic<br />
mechanism which may alleviate these drawbacks.<br />
1. Introduction<br />
Distributed computing environments such as compute<br />
clusters are a powerful concept to tackle complex computational<br />
problems. In recent years, the concept of Grids has<br />
emerged, extending these computing environments to comprise<br />
not only computing resources within an organization<br />
but across administrative domains to further increase efficiency<br />
and flexibility [6].<br />
Currently, mainly physical resources such as processing<br />
power, memory and storage are being shared in Grids.<br />
However, to be deployed in Grids, applications must be able<br />
to achieve Quality of Service assertions despite running on<br />
heterogeneous Grid resources and native platforms [6]. The<br />
Open Grid Services Architecture (OGSA) describes a shift<br />
from physical resources to Web services being shared in the<br />
Grid [7]. Web services are used to hide the <strong>und</strong>erlying infrastructures’<br />
heterogeneity from the Grid application/user<br />
and aggregate basic physical resources to more complex and<br />
elaborate services. As illustrated in [12], the technology gap<br />
between the Grid community and the Web services community<br />
has continuously been narrowed and recently resulted<br />
in the OGSA WSRF Basic Profile 1.0 in which the two technologies<br />
converge into so called Grid services or OGSA<br />
services which can be seen as stateful extensions to classic<br />
Web services [8]. A recent and prominent example for<br />
the application of Web services in the Grid context is Amazon’s<br />
Elastic Compute Cloud that allows Web service-based<br />
deployment of applications on Amazon’s compute platform<br />
[1].<br />
A further example for such a Grid service-based application<br />
with distinct properties and requirements is a video<br />
surveillance environment as described in [15]. Imagine an<br />
office building which accommodates several organizations,<br />
each operating video surveillance cameras. The aim of<br />
the surveillance system is to detect persons and track their<br />
movement across the facility. The analysis of the resulting<br />
video streams requires vast amounts of computing power.<br />
However, to avoid each organization having to deploy its<br />
own, separate cluster and to increase resource utilization,<br />
the organizations deploy one common inter-organizational<br />
compute Grid. In order to perform target separation, the<br />
analysis of a video stream performs a set of tasks which run<br />
sequentially in a pipeline: motion detection, body detection,<br />
direction estimation – in which direction is the person moving?<br />
– and field of vision detection – what is the identity of<br />
the spotted person? Each of these tasks can be encapsulated<br />
Figure 1. Grid service pipeline<br />
164
in a Grid service which can then be instantiated multiple<br />
times, possibly spread across multiple physical machines.<br />
Each video stream runs sequentially through these Grid services<br />
as illustrated in figure 1.<br />
If computer resources are scarce, a scheduling strategy<br />
is needed that decides which Grid service should be allocated<br />
to which camera feed at what time. Classic technical<br />
scheduling mechanisms such at first-come-first-serve or fair<br />
share are solely built on system-centric measures. Consequently,<br />
they do not take into account the strategic dimension<br />
in Grids, i.e. self-interested organizations with heterogeneous<br />
preferences and objectives [5]. For instance, during<br />
some periods of time, the required security level may<br />
vary within organizations. With technical scheduling mechanisms,<br />
these organizations cannot express this variation<br />
in their valuation of the shared Grid services. Economic<br />
scheduling mechanisms are deemed promising to provide a<br />
better fit to the Grid’s inter-organizational nature by allowing<br />
both service requesters and providers to express valuations<br />
in addition to technical metrics. Ultimately, prices are<br />
formed which help to balance dynamic demand and supply<br />
in the Grid. The system can thus induce service requesters<br />
to distribute excess demand over time. In addition, the system<br />
gives service providers incentives to contribute idle resources<br />
to the Grid.<br />
This paper is structured as follows. In section 2, the<br />
requirements for an economic scheduling mechanism for<br />
dynamic Web service applications are outlined. Section 3<br />
discusses related work on this class of scheduling mechanisms.<br />
In section 4 we introduce a mechanism for machine<br />
scheduling proposed by Heydenreich et al. in [11] and subsequently<br />
showcase its general applicability for interactive<br />
Grid settings such as the video surveillance application at<br />
hand in section 5. We will point to some limitations and extend<br />
this mechanism in the Grid context in section 6 before<br />
we recapitulate the main points in section 7.<br />
2. Requirements<br />
The video surveillance Grid service application poses a<br />
number of requirements towards an economic scheduling<br />
mechanism [16]:<br />
• Immediacy: The timing of the allocation is crucial.<br />
The allocation of services in this scenario is time critical<br />
as the monitored objects are dynamic in relatively<br />
constant movement and the workflow of services is<br />
sensitive to the order of execution [15]. For instance,<br />
the direction estimation process may not be executed<br />
prior to the body detection process. Consequently, a<br />
so called online mechanism is needed which allocates<br />
requests as soon as they arrive in the system instead of<br />
pooling and allocating requests and offers periodically.<br />
• Allocative efficiency: This requirement addresses that<br />
there should be no “waste” of resources; the system is<br />
supposed to make optimal use of its resources.<br />
• Decentralized scheduling removes the need for a central<br />
bottleneck which may not scale sufficiently or<br />
cause the whole system to fall over, an aspect that is<br />
crucial for large scale applications.<br />
• Incentive compatibility: participants cannot benefit<br />
from cheating the mechanism, i.e. reporting any valuation<br />
for a service other than their true valuation.<br />
• A mechanism is said to be individually rational if its<br />
users cannot suffer any loss in utility from participating.<br />
• The payment scheme of a budget-balanced mechanism<br />
is designed such that it does not require any subsidies;<br />
the payments to service providers are covered by the<br />
payments from service requesters. Any mechanism<br />
must be both individually rational and budget-balanced<br />
in order to be sustainable over time.<br />
In the following, we are interested in economic scheduling<br />
mechanisms that satisfy those requirements.<br />
3. Related Work<br />
There are two main classes of economic scheduling<br />
mechanisms with respect to the mode of allocation: mechanisms<br />
which execute periodically and mechanisms which<br />
execute continuously.<br />
Hitherto, periodic economic mechanisms have received<br />
the most interest from the economic mechanism design<br />
community. The prototype SPAWN implements a market for<br />
computing resources in which each workstation auctions off<br />
idle computing time to multiple applications by means of a<br />
Vickrey auction [17]. All resources are allocated to at most<br />
one application at a time regardless of this application’s actual<br />
demand which yields low resource utilization. POP-<br />
CORN is a Web-based market for computing power which<br />
implements a Vickrey auction as well as two double auctions<br />
[14]. While these two mechanisms focus on the trading<br />
of processing power only, Schnizler et al. elaborate a<br />
multi-attribute combinatorial exchange (MACE) in which<br />
users are allowed to request and offer arbitrary b<strong>und</strong>les of<br />
Grid resources and can specify quality attributes on these<br />
resources [16]. MACE implements an exact mechanism.<br />
The scheduling problem in this combinatorial setting is NPhard,<br />
the pricing scheme of MACE yields approximately<br />
incentive compatible prices.<br />
The BELLAGIO system also implements an exact combinatorial<br />
exchange for computing resources [3]. Being an<br />
exact mechanism it shares the computational drawbacks of<br />
165
MACE. In the mechanism proposed by Bapna et al., multiple<br />
requesters and providers can trade both computing<br />
power and memory for a sequence of time slots [4]. First,<br />
an exact mechanism is introduced. By imposing both fairness<br />
constraints and one common valuation across all resource<br />
providers, the search space in the <strong>und</strong>erlying combinatorial<br />
allocation problem is structured as to establish one<br />
common, incentive compatible price per time slot for all<br />
accepted requests. Additionally, this introduces a linear ordering<br />
across all jobs and time which reduces the complexity<br />
of the allocation problem, which, however, still remains<br />
NP-hard. To mitigate this computational complexity, a fast,<br />
greedy heuristic is proposed at the expense of both incentive<br />
compatibility and efficiency. All of these approaches may<br />
be applicable for batch applications where immediacy can<br />
be sacrificed in return for increased economic efficiency.<br />
However, due to periodic allocation, the complexity of the<br />
<strong>und</strong>erlying allocation problem and the necessary time to actually<br />
compute the allocation, all of these mechanisms are<br />
inappropriate for interactive Grid settings and do not satisfy<br />
the required immediacy needed for the video surveillance<br />
Grid service application at hand. Consequently, a continuous<br />
economic mechanism is required.<br />
The most prevalent continuous economic mechanisms –<br />
also called online mechanisms – are based on the concept<br />
of market-based proportional share as proposed in [5] and<br />
[13]. Each requester receives a fraction of the total available<br />
resources according to her share in the total reported<br />
valuation across all requesters; i.e., if requester i out of n<br />
requesters reported a valuation of v i , she will receive a fraction<br />
amounting to v i / ∑ n<br />
j=1 v j of the resource. The problem<br />
with market-based proportional share is that it does<br />
generally not support Quality of Service assertions. The<br />
mechanism holds the risk that requesters are not able to<br />
obtain the necessary resources [13]; the share of resources<br />
provided to a requester, i.e. the actual service level returned<br />
to the requester, will generally be below or above this requester’s<br />
required service level. More importantly, the basic<br />
form of market-based proportional share as suggested in<br />
[5] does not support advance reservation and may result in<br />
high latency which is fatal for interactive applications [13].<br />
The AUCTION SHARE mechanism proposed by Lai et al.<br />
extends the basic market-based proportional share mechanism<br />
as to support low latency, advance reservations and an<br />
incentive compatible pricing scheme [13].<br />
In summary, the employed mechanisms proposed for<br />
Grids do not fully satisfy all requirements stated in section<br />
2.<br />
4. A Decentralized Local Greedy Mechanism<br />
A Grid environment consists of a set of individual users<br />
(agents) which we will assume to act rationally and selfishly,<br />
meaning the agents intend to maximize their individual<br />
benefit from participating in the Grid. In the following,<br />
this setting will be introduced as an instance of the class of<br />
machine scheduling problems P| r j | ∑ w j C j [9].<br />
Agents submit (computational) tasks to the system, so<br />
called jobs. A job j is specified by the tuple t j =<br />
(r j , p j , w j ) where r j ∈ R + 0 denotes the job’s release date,<br />
p j ∈ R + the job’s processing time and w j ∈ R + the job’s<br />
weight, i.e. the job’s cost of waiting for one additional unit<br />
of time. When requesting resources for the execution of a<br />
job, the selfish agent 1 j may strategically submit the request<br />
(its type [10]) ˜t j = (˜r j , ˜p j , ˜w j ) ≠ t j to the system.<br />
The Grid infrastructure consists of m identical, parallel<br />
machines M = {1, ..., m}. We restrict the strategic analysis<br />
of this setting to selfish behavior on the demand side only<br />
and consequently assume that machines always report true<br />
information about their status to the system.<br />
Each of the jobs J = {1, ..., n} can be executed on any of<br />
the m machines. Jobs are assumed to be numbered in order<br />
of their release dates, i.e. j < k ⇒ ˜r j ≤ ˜r k . Each machine<br />
can process one job at a time. Preemption of jobs is not<br />
allowed, meaning once a job has been started, it cannot be<br />
interrupted externally.<br />
For this setting, Heydenreich et al. [10] propose a DE-<br />
CENTRALIZED LOCAL GREEDY MECHANISM (DLGM)<br />
which aims at maximizing the agents’ overall “happiness”<br />
by minimizing the sum of the jobs’ weighted completion<br />
times ∑ j∈J w jC j – the costs induced by the system –<br />
where C j denotes j’s actual completion time.<br />
DLGM comprises the following steps:<br />
1. At its chosen release date ˜r j , job j communicates<br />
˜w j and ˜p j to every machine m ∈ M. It is assumed<br />
that a machine can postpone its release date, therefore<br />
˜r j ≥ r j . The processing time can only be overstated<br />
(˜p j ≥ p j ), e.g. by adding unnecessary calculations,<br />
but not <strong>und</strong>erstated since this could easily be detected<br />
and punished by the mechanism [10]. The weight ˜w j<br />
can be reported arbitrarily.<br />
2. Based on the received information (˜r j , ˜p j , ˜w j ), the machines<br />
communicate a (tentative) completion time Ĉj<br />
and a (tentative) payment ˆπ j to the job. The tentativeness<br />
is due to the fact that later arriving jobs might<br />
overtake job j. This leads to a final (ex-post) completion<br />
time C j ≥ Ĉj and a final (ex-post) payment<br />
π j ≤ ˆπ j as compensation payments by overtaking<br />
jobs might occur (see below). The local scheduling<br />
on each machine follows the ”weighted shortest processing<br />
time first” (WSPT) policy. Jobs are assigned<br />
a priority value according to their ratio of weight and<br />
processing time: job j has a higher priority than job k<br />
1 In the following we will use the terms “agent” and “job” interchangeably.<br />
166
⇔ ˜w j /˜p j ≥ ˜w k /˜p k and is thus inserted in front of k<br />
into the waiting queue at the machine. Ties may be resolved<br />
by time of arrival or randomly, for instance. Let<br />
H(j) denote the set of jobs with higher priority than<br />
j and L(j) be the set of jobs with lower priority than<br />
j, respectively. Furthermore, let S j denote the point in<br />
time when job j is started to be processed and b i (t) the<br />
remaining processing units of the currently executed<br />
job at time t on machine i. j → i indicates that job j<br />
is assigned to machine i. Then (tentative) completion<br />
times Ĉj(i) and payments ˆπ j (i), j ∈ J and i ∈ M, can<br />
be computed as<br />
∑<br />
Ĉ j (i) = ˜r j + b i (˜r j ) +<br />
˜p k + ˜p j<br />
k∈ H(j),k→i,k
formance can be assured; the objective value ∑ j∈J w jC j –<br />
which is to be minimized – resulting from the online mechanism<br />
where information about jobs is gradually becoming<br />
available is compared to the the optimal solution by an<br />
offline mechanism whose computations are based on prior<br />
knowledge of the entire information space. Heydenreich et<br />
al. [10] derive a value of ρ = 3.281 for DLGM, meaning the<br />
objective value of DLGM is guaranteed to be no more than<br />
ρ = 3.281 times the objective value of an optimal offline<br />
mechanism.<br />
5. Sample Scenario<br />
In this section, we provide a numerical example to<br />
demonstrate the applicability of the DLGM to the video<br />
surveillance environment introduced in section 1. In our<br />
example setting, there are five cameras performing video<br />
stream analysis with the aim of target separation. A job<br />
can in this context be interpreted as a segment of a video<br />
stream which needs to be analyzed. According to the Grid<br />
service pipeline illustrated in figure 1, each job requests to<br />
be assigned sequentially to the four task types. For the sake<br />
of simplicity, we confine our example to the first two tasks<br />
in the pipeline, i.e. motion detection (MD) and body detection<br />
(BD) (cf. figure 1). Furthermore, the critical job<br />
restriction is neglected. Each of the tasks is encapsulated<br />
in a corresponding Grid service. We assume that we have<br />
two machines for each Grid service, i.e. MD 1 and MD 2<br />
for motion detection and BD 1 and BD 2 for body detection.<br />
The different job types are generated sequentially in<br />
the given order at each camera: only when a MD-job has<br />
been processed and the result has been transferred back to<br />
the camera, a BD-job can be generated. This illustrates<br />
the interactive processing mode of this Grid service application.<br />
We can partly avoid combinatorial problems since<br />
it is not possible that e.g. a direction estimation job is executed<br />
without the <strong>und</strong>erlying motion detection job being<br />
completed. Note, however, that it is still possible that the<br />
MD-part of a camera feed is being analyzed but the camera<br />
is not able to subsequently obtain the BD-service. A job<br />
which is released at time t can be started being processed at<br />
time t + 1. The order in which jobs with identical release<br />
dates are being considered by the machines is determined<br />
randomly; the same holds for the cameras’s choice between<br />
two machines if both yield the same utility. Table 1 lists<br />
all the generated jobs with the respective camera and service<br />
type as well as their type (˜r, ˜p, ˜w) and the priority ratio<br />
˜w/˜p.<br />
The waiting queues for each service and timestep are illustrated<br />
in figure 2; empty queues have been omitted, active<br />
jobs are highlighted. At time r = 0, cameras 1 to 3<br />
each generate a MD-job. All jobs report to the two MDmachines<br />
which compute tentative completion times and<br />
job camera service ˜r ˜p ˜w ˜w/˜p<br />
1MD 1 MD 0 2 6 3<br />
2MD 2 MD 0 4 5 1.25<br />
3MD 3 MD 0 1 6 6<br />
4MD 4 MD 1 1 10 10<br />
5MD 5 MD 2 2 7 3.5<br />
3BD 3 BD 2 2 11 5.5<br />
1BD 1 BD 3 2 5 2.5<br />
4BD 4 BD 3 1 8 8<br />
Table 1. Job generation at the cameras in<br />
chronological order<br />
Figure 2. Sample waiting queues<br />
job Ĉ j (MD 1 ) ˆπ j (MD 1 ) û j (MD 1 ) choice<br />
Ĉ j (MD 2 ) ˆπ j (MD 2 ) û j (MD 2 )<br />
1MD 0+0+0+2=2 0 -12 MD 1<br />
0+0+0+2=2 0 -12<br />
2MD 0+0+2+4=6 0 -30 MD 2<br />
0+0+0+4=4 0 -20<br />
3MD 0+0+0+1=1 1*6=6 -6-6=-12 MD 2<br />
0+0+0+1=1 1*5=5 -6-5=-11<br />
4MD 1+1+0+1=3 0 -30 MD 2<br />
1+0+0+1=2 1*5=5 -20-5=-25<br />
5MD 2+0+0+2=4 0 -28 MD 1<br />
2+0+0+2=4 1*10=10 -28-10=-38<br />
Table 2. Motion Detection (MD) service<br />
168
job Ĉ j (BD 1 ) ˆπ j (BD 1 ) û j (BD 1 ) choice<br />
Ĉ j (BD 2 ) ˆπ j (BD 2 ) û j (BD 2 )<br />
3BD 2+0+0+2 = 4 0 -44 BD 1<br />
2+0+0+2 = 4 0 -44<br />
1BD 3+1+0+2 = 6 0 -30 BD 2<br />
3+0+0+2 = 5 0 -25<br />
4BD 3+1+0+1 = 5 0 -40 BD 2<br />
3+0+0+1 = 4 1*5 = 5 -32-5 = -37<br />
Table 3. Body Detection (BD) service<br />
payments (cf. table 2). With no other jobs being present at<br />
any machine, camera 1 selects MD 1 to execute job 1MD<br />
since it yields a higher utility. Camera 2’s job 2MD is<br />
being considered next; due to the occupation of MD 1 by<br />
job 1MD, 2MD is scheduled on the other machine MD 2 .<br />
Since both jobs are allocated to idle machines, they do not<br />
need to compensate any other jobs. Next, job 3MD is evaluated.<br />
Both MD-machines now already have one job waiting<br />
to be executed from the next timestep on. But since<br />
MD 3 ’ s priority value exceeds the other two jobs’ priority,<br />
it can overtake them. With a payment of $5, job 2MD is<br />
”cheaper” to compensate and thus 3MD is scheduled on<br />
machine MD 2 in front of 2MD. The execution of job<br />
1MD on MD 1 and 3MD on MD 2 will start in the following<br />
period r = 1. The next job being generated is 4MD by<br />
camera 4 which shifts job 2MD at MD 2 further back in the<br />
waiting queue due to its higher priority and consequently<br />
pays $5 in compensation. MD 2 is chosen instead of MD 1<br />
since it can guarantee an earlier completion time. Two new<br />
jobs emerge at time r = 2, another MD-job at camera 5 as<br />
well as the first BD-job 3BD arising as a result from the already<br />
completed MD-job 3MD. Camera 5 chooses MD 1<br />
since no compensation payments are due, while the completion<br />
time is the same as for MD 2 . 3BD can choose from<br />
the two empty BD-machines (cf. table 3). Concerning the<br />
two BD-jobs 1BD and 4BD being generated in r = 3,<br />
1BD is considered first and placed at the empty machine<br />
BD 2 . It is, however, immediately being displaced by the<br />
higher-prioritized job 4BD which gains more utility from<br />
the earlier completion time feasible at BD 2 than from waiting<br />
at BD 1 even though it has to compensate 1BD with<br />
$5.<br />
6. Limitations and Possible Extensions<br />
Despite some striking features outlined in section 4,<br />
DLGM also suffers from drawbacks. In this section, we<br />
show a few of the identified limitations of the basic mechanism<br />
and point to extensions that may alleviate the problems.<br />
First, DLGM does not fully account for the interorganizational<br />
nature of Grids; strategic and self-interested<br />
behavior is modeled on the demand side only. Jobs have private<br />
information about their type and may misreport about it<br />
in hope for a utility gain. The supply side, i.e. the machines<br />
offering their service for the execution of jobs, is assumed<br />
to be ”obedient”. Since jobs are the ones determining the<br />
allocation by choosing their favorite location for execution,<br />
the machines’ action space is restricted to (truthfully) reporting<br />
tentative completion times and payments to the jobs<br />
upon request. Neither are they granted the right to select the<br />
jobs they want to execute themselves, nor does the design<br />
of the mechanism leave room for strategic behavior in terms<br />
of misstating completion time and/or payments. Moreover,<br />
with the basic setting, machines do not get any reward for<br />
contributing to the system. Their behavior is assumed to<br />
be beneficial and altruistic. Consequently, DLGM is only<br />
suitable for scheduling scarce resources that are <strong>und</strong>er centralized<br />
control. The design of a mechanism which accounts<br />
for strategic behavior on both sides of the market serves as<br />
a goal for future research.<br />
Another weakness of DLGM becomes observable when<br />
taking a closer look at the example provided in the previous<br />
section. Since job 2MD queues first at machine MD 2 , it<br />
does not displace any present job and this exempts it from<br />
having to make an immediate payment. However, it subsequently<br />
receives compensation payments amounting to $10<br />
from the overtaking jobs 3MD and 4MD. 2MD could<br />
thus have exploited the payment scheme by adding unnecessary<br />
computing instructions at the end of its regular code<br />
or even creating a whole artificial job with entirely senseless<br />
computations. A relatively simple remedy for preventing<br />
jobs from exploiting the mechanism to earn money may<br />
be to add a variable cost component to the payment scheme.<br />
This can be done by introducing z i ∈ R + which equals the<br />
operating costs per timestep for machine i. When extending<br />
the scenario to feature strategically acting machines which<br />
can misreport about Ĉj and ˆπ j , machines may be allowed to<br />
individually report their operating costs z i . To incorporate<br />
the operating costs in the mechanism, the tentative payment<br />
ˆπ j has to be extended:<br />
∑<br />
ˆπ j = ˜p j (˜z i +<br />
˜w k ).<br />
k∈ L(j),k→i,kr j<br />
The introduction of z (respectively ˜z) does on the one side<br />
have a positive effect on jobs. They are provided with an<br />
incentive to claim only as much processing time as needed<br />
since every unit of useless computation now costs money.<br />
On the other hand, it becomes possible to at least cover<br />
the expenses the machines incur for executing the jobs and<br />
computing the priority values, (tentative) completion times<br />
and (tentative) payments as well as the continuous interjob<br />
compensations. Assuring incentive compatibility on the<br />
supply side is yet another issue to be tackled in future work.<br />
169
Allowing preemptiveness of jobs may further increase<br />
the efficiency of the mechanism. In DLGM in its current<br />
form, a job which is being processed at a machine may not<br />
be interrupted until it is completed. Grid applications like<br />
the video surveillance scenario demand for the possibility to<br />
freeze the execution of a running job if a job with a higher<br />
priority enters the queue. As long as a job can be stopped<br />
at any point in time, the procedure of the mechanism needs<br />
to be adjusted only slightly: When a job j asks a machine<br />
for Ĉj and ˆπ j , the machine also considers the priority value<br />
of the currently executed job l . If the priority of the incoming<br />
job is higher, i.e. ˜w j /˜p j > ˜w l /˜p l , the current job l is<br />
“frozen”, its processing time set to the remaining processing<br />
time, and it is put back into the queue. Enabling suspension<br />
of jobs requires additional memory space for saving the<br />
preliminary results of a job; they are transmitted to the job<br />
initiator only when the job is completed to avoid communicational<br />
overload. The formula for computing the tentative<br />
completion time of a job j has to be changed to<br />
∑<br />
Ĉ j (i) = ˜r j +<br />
˜p k + ˜p j .<br />
k∈ H(j),k→i,k
in a scalable computing cluster. IEEE Trans. Parallel Distrib.<br />
Syst., 11(7):760–768, 2000.<br />
[3] A. AuYoung, B. Chun, A. Snoeren, and A. Vahdat. Resource<br />
allocation in federated distributed computing infrastructures.<br />
In Proceedings of the 1st Workshop on Operating<br />
System and Architectural Support for the Ondemand IT InfraStructure,<br />
October 2004.<br />
[4] R. Bapna, S. Das, R. Garfinkel, and J. Stallaert. A Market<br />
Design for Grid Computing. INFORMS Journal on Computing,<br />
forthcoming, 2006.<br />
[5] B. N. Chun and D. E. Culler. Market-based proportional<br />
resource sharing for clusters. Technical report, Berkeley,<br />
CA, USA, 2000.<br />
[6] I. Foster, C. Kesselman, J. Nick, and S. Tuecke. The physiology<br />
of the grid: An open grid services architecture for<br />
distributed systems integration. Technical report, Globus<br />
Project, www.globus.org/research/papers/ogsa.pdf, 2002.<br />
[7] I. Foster, H. Kishimoto, A. Savva, D. Berry, A. Djaoui,<br />
A. Grimshaw, B. Horn, F. Maciel, F. Siebenlist, R. Subramaniam,<br />
J. Treadwel, and J. V. Reich. The open grid services<br />
architecture, version 1.5. Technical report, Open Grid<br />
Forum, July 24 2006.<br />
[8] I. Foster, T. Maguire, and D. Snelling. OGSA WSRF basic<br />
profile 1.0. Technical report, Open Grid Forum, May 1 2006.<br />
[9] R. L. Graham, E. L. Lawler, J. K. Lenstra, and A. H. G. R.<br />
Kan. Optimization and approximation in deterministic sequencing<br />
and scheduling theory: a survey. Annals of Discrete<br />
Mathematics, 5:287–326, 1979.<br />
[10] B. Heydenreich, R. Müller, and M. Uetz. Decentralization<br />
and mechanism design for online machine scheduling.<br />
In L. Arge and R. Freivalds, editors, SWAT, volume<br />
4059 of Lecture Notes in Computer Science, pages 136–147.<br />
Springer, 2006.<br />
[11] B. Heydenreich, R. Müller, and M. Uetz. Games and mechanism<br />
design in machine scheduling : an introduction. 2006.<br />
[12] J. Joseph, M. Ernest, and C. Fellenstein. Evolution of grid<br />
computing architecture and grid adoption models. IBM Systems<br />
Journal, 43(4):624–645, 2004.<br />
[13] K. Lai, B. A. Huberman, and L. Fine. Tycoon: A distributed<br />
market-based resource allocation system. Technical report,<br />
April 5 2004.<br />
[14] O. Regev and N. Nisan. The popcorn market - an online market<br />
for computational resources. In ICE ’98: Proceedings of<br />
the first international conference on Information and computation<br />
economies, pages 148–157, New York, NY, USA,<br />
1998. ACM Press.<br />
[15] E. Rosenberg and A. Eilat. Video surveillance grid application,<br />
version 1.0. Technical report, Correlation Systems<br />
Ltd., December 17 2006.<br />
[16] B. Schnizler, D. Neumann, D. Veit, and C. Weinhardt. Trading<br />
grid services - a multi-attribute combinatorial approach.<br />
European Journal of Operational Research, forthcoming,<br />
2006.<br />
[17] C. A. Waldspurger, T. Hogg, B. A. Huberman, J. O. Kephart,<br />
and W. S. Stornetta. Spawn: A distributed computational<br />
economy. IEEE Trans. Softw. Eng., 18(2):103–117, 1992.<br />
171
Exploiting the Knowledge Economy: Issues, Applications, Case Studies<br />
Paul Cunningham and Miriam Cunningham (Eds)<br />
IOS Press, 2006 Amsterdam<br />
ISBN: 1-58603-682-3<br />
Self-Organizing ICT Resource Management<br />
– Policy-based Automated Bidding<br />
Dirk NEUMANN<br />
<strong>Institut</strong>e of Information Systems and Management (<strong>IISM</strong>), Universität Karlsruhe (TH),<br />
Engler Str. 14, 76131 Karlsruhe, Germany<br />
Tel: +49 721 608 8377, Fax: + +4 721 608 8399, Email: Neumann@iism.uni-karlsruhe.de<br />
Abstract: For decades markets have been proposed for allocating computer<br />
resources in distributed systems. None of these approaches has made it into practice.<br />
One reason for the adoption failure that has been rarely discussed is the bidding<br />
process. Theoretic approaches assume that the bidders know their bids exactly. This<br />
includes the specific resource, which is needed at some time in the future, as well as<br />
the price. This assumption simplifies reality and can thus not contribute to the<br />
development of Grid markets. What is needed to establish a prospering Grid market<br />
are rules how to conduct the bidding. Since demand & supply are dynamic, manual<br />
bidding is too slow to accommodate abrupt shifts. This paper introduces a policy<br />
based autonomous agent approach for automated bidding. By means of the policies<br />
resource providers and consumers can specify how Grid resources are being traded.<br />
1. Introduction<br />
The vision of a complete virtualization of Information and Communication Technology<br />
(ICT) infrastructures by the provision of resources like computing power or storage over<br />
Grid infrastructures will make the development of Open Grid Markets necessary. Over the<br />
Open Grid Market idle or unused resources (e.g. computational resources) can be supplied<br />
(e.g. as services) and client demand can be satisfied not only within organization but also<br />
among multiple administrative domains.<br />
In conventional IT environments, the decision “who shares with whom, at what time” is<br />
orchestrated via central server that uses scheduling algorithms in order to maximize<br />
resource utilization. Those central scheduling algorithms, however, have problems when<br />
organizational bo<strong>und</strong>aries are crossed and information about demand and supply are<br />
manipulated. In particular, if demand exceeds supply, the scheduling algorithms fail to<br />
allocate the resources efficiently.<br />
The further virtualization of ICT infrastructure and services requires technical solutions<br />
that allow for an economically efficient allocation of resources. Economic mechanisms can<br />
be the basis for technological solutions that handle these problems by setting the right<br />
incentives to reveal information about demand and supply accurately. Market or pricing<br />
mechanisms foster information exchange and can therefore attain efficient allocations.<br />
The idea of introducing markets in distributed systems is certainly not new. Despite<br />
previous efforts, none of the market-based approaches has made it into practice. The<br />
reasons can briefly be summarized by the following three arguments: (1) inadequate market<br />
design, (2) insufficient support to use the markets, and (3) improper coupling between the<br />
market and state of the art middleware.<br />
The SORMA project (Self-Organizing ICT Resource Management, www.sormaproject.eu)<br />
will implement a comprehensive Open Grid Market by addressing all three<br />
arguments. Firstly, the economic model provides an so<strong>und</strong> market structure. Secondly, the<br />
self-organization model deals with interaction between the Grid-application and the market<br />
Copyright © 2006 The Authors 172
y providing intelligent tools. Thirdly, the economic middleware model builds the bridge<br />
between the self-organization, the economic model, and state of the art Grid infrastructure.<br />
Integrating those three models into the SORMA system, the Open Grid Market is expected<br />
to take off in practice and help realizing the benefits of Grid technologies. Potential target<br />
groups of the Open Grid Markets range from server farms, over large companies to SMEs,<br />
which strive for lowering the total cost of ownership for ICT resources.<br />
The design of Open Grid Markets and related issues attempting to move business to<br />
Grid are currently hotly debated <strong>und</strong>er the label Grid Economics. In this context, this paper<br />
concentrates on the self-organization model and shows how policies can be used to<br />
automate the bidding process. This paper is unique as it proposes a novel approach on<br />
bidding in Grid and reconciles research on Grid markets and on technical middleware.<br />
The remainder of the paper is structured as follows. In section 2 the SORMA system,<br />
enabling self-organized ICT resource <strong>management</strong> is illustrated. In section 3, the policybased<br />
working of the bidding component is showed. Section 4 concludes with a summary<br />
and a brief outlook.<br />
2. The SORMA System<br />
The lessons learned from the previous efforts to establish markets in distributed system<br />
gave rise to the SORMA project. The overall objective of SORMA is the development of<br />
methods and tools for establishing an efficient market-based allocation for ICT resources in<br />
a more efficient way in order to enable resource accessibility for all users and to increase<br />
user’s satisfaction, profit and productivity.<br />
Grid Application<br />
Payment Interface<br />
Open Grid<br />
Market<br />
Resource<br />
Monitoring<br />
Clearing &<br />
Billing<br />
Security<br />
Contract<br />
Management<br />
Market<br />
Connector<br />
Intelligent Tools<br />
Economic Grid Middleware<br />
Grid Market<br />
Directory<br />
Bidding<br />
Component<br />
Resource<br />
Decomposition<br />
Accounting<br />
Resource<br />
Auditing<br />
Grid<br />
Middleware<br />
Extended<br />
Resource<br />
Management<br />
Common Virtualization Middleware<br />
(Globus, UNICORE,...)<br />
Resource Fabrics<br />
Figure 1: The SORMA System<br />
SORMA uses concepts from autonomic computing [2, 9], such that the ICT resource<br />
allocation process is automatically and autonomously conducted. Once the bidding process<br />
is virtualized, the ICT system of an organization will self-organize its resource<br />
<strong>management</strong>: overcapacity is offered on the market, while <strong>und</strong>ercapacity initiates a buying<br />
process. Markets and self-organization go thus hand in hand.<br />
The SORMA system combines all those areas of self-organizing ICT resource<br />
<strong>management</strong> in a holistic way. This is reflected by the high-level architecture, which<br />
consists of the following building blocks: Open Grid Market, Intelligent Tools and<br />
Economic Grid Middleware (Figure 1). In the following, the focus is on the bidding<br />
component. Since bidding without a market is meaningless, the architecture of the Open<br />
Grid Market is presented.<br />
Copyright © 2006 The Authors 173
2.1. Open Grid Market<br />
The Open Grid Market consists of the service consumers and providers, representing<br />
demand and supply, and of the software system that implements the market functionalities.<br />
Service consumers are those entities that request specific services. For example, a bionumerical<br />
simulation application requests computation and storage services. As it is the<br />
goal to have a self-organizing ICT resource <strong>management</strong>, the service consumer is<br />
considered to be the specific Grid application, which provides runtime data of the<br />
requestor’s system. Based on this data, service requests are initiated including detailed<br />
service type requirements (e.g. storage service) and quality of service constraints (e.g. ><br />
300GB free space for 4 hours). It is assumed that approximate quality and time constraints<br />
of the requested services can be determined using prediction models [e.g. 1].<br />
On the service provider side, computer nodes host a set of Grid services (e.g.<br />
computation service or storage service) and provide information about their current quality<br />
characteristics and availability. The node includes a resource manager that provides runtime<br />
data about scheduling, allocating, and monitoring of local resources. This resource manager<br />
makes use of prediction models in order to determine the approximate future availability of<br />
the <strong>und</strong>erlying computational resources.<br />
The Open Grid Market component aggregates the bids from service providers and users<br />
into an allocation of resources and corresponding prices. Crucial for the working of the<br />
Open Grid Market is the clearing mechanism component, which implements the logic how<br />
the market is resolved (i.e. auction formats). In the SORMA framework, the clearing<br />
mechanism can be arbitrarily defined. The MACE auction is deemed promising for a use in<br />
Grids, as it allows service requesters and providers the simultaneous submission of bids on<br />
heterogeneous services expressing substitutes and complements [8]. Bids, encoded as WS-<br />
Agreement offers, are submitted to an order book. The auction determines an allocation and<br />
the corresponding prices either regularly or continuously.<br />
The objective of the allocation process in MACE is the maximization of social welfare,<br />
i.e. the difference between the buyers' valuations and the sellers' reservation prices. The<br />
problem is formulated as a linear Mixed Integer Program and, thus, can be solved by<br />
standard optimization solvers. The outcome of the winner determination model is allocative<br />
efficient, as long as buyers and sellers reveal their valuations truthfully. The incentive to set<br />
bids according to the valuation is induced by an efficient pricing mechanism. MACE<br />
implements a k-pricing scheme and as such determines prices for buyers and sellers on the<br />
basis of the difference between their bids. For instance, presume a buyer wants to buy a<br />
computation service for 5 and a seller wants to sell a computation service for at least 4. The<br />
difference between these bids is ζ =1, i.e. ζ is the surplus of this transaction and can be<br />
distributed among the participants. This schema can be applied to MACE and results in an<br />
approximate efficient outcome [8].<br />
2.2. Bidding Component<br />
When describing the Open Grid Market, it was mentioned that service consumer and<br />
providers submit bids that express their demand and supply situation. This implies that the<br />
Grid applications and the computing nodes have the capabilities to determine reasonable<br />
bids. In Grid markets generating reasonable bids can become very complex for several<br />
reasons: For example demand and supply fluctuates dynamically and is in most of the cases<br />
unknown, valuations are unknown and bidding strategies have not yet developed.<br />
The bidding component in SORMA consists of two elements: a Bidding Agent that<br />
generates bids and forwards them to the clearing mechanism and an Administration<br />
Interface that allows the control and configuration of the agent [7].<br />
Copyright © 2006 The Authors 174
• The Administration Interface specifies user policies. Policies are mostly static and do<br />
not change frequently. They define general rules <strong>und</strong>er which condition, when and how<br />
bids are being generated. These policies are either entered manually via a human control<br />
interface or imported from other sources in a company.<br />
• A Bidding Agent implements a bid generation process, which is based on policies as<br />
well as runtime data. It is also responsible for submitting bids to the market mechanism.<br />
The agent interacts with the Open Grid Market in order to gain auction feedback (e.g.<br />
current highest price) which influences its bidding decisions. The agent specifies all<br />
attributes of an order (e.g. price and bidding time) autonomously.<br />
The Bidding Agent integrates policies as well as runtime information via an ontology.<br />
Ontologies are logic-based representation languages that executable calculi. This is required<br />
by the agent in order to query and reason about this information during run-time. To capture<br />
system information as well as policies we rely on the Web Ontology Language (OWL) and<br />
a decidable fragment of the Semantic Web Rule Language (SWRL) defined in [6].<br />
3. Policy-based Bidding<br />
This section introduces the reasoning behavior of the bidding agent, which autonomously<br />
derives one or several bids based on system and market information. In essence, the<br />
generation is guided by policies. Policies are considered as declarative rules defined by the<br />
user to adapt and control the behavior of the system. Policies are thus adequate to represent<br />
the logic (i.e. the business model) for bidding. Policies are specified by the administrator<br />
using the human control interface. The bidding component receives system information<br />
from the local resource manager of an application or computational nodes.<br />
By means of policies this system information is interpreted and appropriate bidding<br />
activities (e.g. submission or manipulation of bids) are initiated. The general bid generation<br />
process performed by the bidding component is distinguished into four steps (c.f. Figure 2).<br />
Demand<br />
Curve<br />
Market<br />
Information<br />
Demand<br />
Curve<br />
Grid<br />
Ontology<br />
System Information<br />
Monitoring<br />
Attribute<br />
Determination<br />
Bid expansion<br />
Bid specification<br />
Bidding<br />
Policies<br />
Pricing, QoS<br />
Policies<br />
Expansion<br />
Policies<br />
Bid Generation Process<br />
User Policies<br />
Figure 2: Bid Generation Process [7]<br />
In a first step, monitoring information, derived from the resource manager, is explored<br />
to assess whether a bid is submitted. This decision is based on (1) the current and on the<br />
predicted utilization of the resource or (2) on the expected demand of the application and on<br />
the preferences of the user.<br />
In case demand or supply situation requires the submission of one or more bids, the<br />
concrete properties of the bids are determined by taking information from the market<br />
mechanism, as well as utilization, and user policies into account (second step).<br />
In a third step, the bid is expanded by means of a domain ontology that contains<br />
taxonomical and composition information. This step assures that different levels of<br />
abstraction in descriptions of bids do not lead to inefficiencies in the market. In addition, it<br />
reflects the fact that several Grid services can be offered and requested within one order.<br />
Finally, before forwarding to the market the bid is serialized using a WS-Agreement<br />
template to assure interoperability with the market mechanism.<br />
Copyright © 2006 The Authors 175
3.1. Information Gathering<br />
The bid generation process is triggered by a decision event. This decision event depends on<br />
the actual and expected utilization of the ICT resource or the demand of the application,<br />
respectively. The functions, which represent either the availability of the resources or the<br />
demand of an application are denoted as load curves D(t). Load curves are obtained from<br />
the resource manager by means of prediction models.<br />
Based on the load curves and on the bidding policies defined by the user a decision is<br />
made. For example, a simple bidding policy may specify that a bid must be submitted to the<br />
Open Grid Market in case the demand is predicted to exceed a certain threshold υc within<br />
the interval t 0 to t 1 . That means, a bid has to be submitted in case D(t) > υc holds at least for<br />
one t with t 0 < t < t 1 . The policy specifying if a bid b has to be generated within a certain<br />
planning interval p based on threshold value υ as well as the overall capacity c:<br />
issueBid(?b,?p) ← PlanningInterval(?p), lowerBo<strong>und</strong>(?p,?t1), upperBo<strong>und</strong>(?p,?t2), Time(?t),<br />
load(?t,?dt), greaterThan(?t,?t1), greaterThan(?t2,?t),hasThreshold(?b,?v), hasCapacity(?b,?c),<br />
multiply(?v,?c,?e), greaterThan(?dt,?e)<br />
The length of the interval should be determined according to the minimal slot that can<br />
be requested from one single service provider or that can be provided to a single requester,<br />
respectively. Long-range planning intervals may thereby entail inaccuracies due to load<br />
changes during the interval. Thus, it is referred to mean values. For each interval this step<br />
will be executed only once. Moreover, machine learning approaches can be applied in order<br />
to adapting υ with respect to changing environments.<br />
3.2. Bid Formulation<br />
After the bid formulation has been initiated by the monitoring step the attribute values for a<br />
concrete bid have to be determined. Within each bid the type of Grid service that is required<br />
or offered, quality of service criteria that have to be fulfilled, a time interval in which the<br />
resource is offered/requested, and a range of prices which are acceptable (e.g. maximal or<br />
minimal price) has to be specified in a way that it is optimal with respect to the goals of the<br />
agent. These goals are influenced by three aspects: The user preferences reflected by the<br />
system policies, the system monitoring information, and the current market information. In<br />
the following we show how these three information sources can be integrated in order to<br />
derive attribute values for the individual attributes.<br />
Static Storage Service<br />
Storage Service<br />
Dynamic Storage Service<br />
f,g<br />
1<br />
HDD Service DVD Service RAM Service<br />
0<br />
0<br />
Figure 3: Storage Service Ontology and Eagerness Policies<br />
l(t 0 ,t 1 ), t 0 -t<br />
Service Functionality: The service functionality determines which service is provided<br />
by the computational node, or requested by an application. This information is directly<br />
derived from the resource manager and can be modeled by referring to the corresponding<br />
concept of a Grid ontology, which is based on the JSDL and domain-specific extensions as<br />
shown for the storage service example in the left panel of Figure 3. In a subsequent step, the<br />
relations between the different concepts in the ontology are exploited to improve the<br />
matchmaking of services in the market.<br />
Price: In literature, it is frequently assumed that bidders know their valuations. From<br />
these valuations bidding strategies are derived that are either static such as so-called Zero-<br />
Intelligence bidders or highly adaptive. Unfortunately, for Grid services it is difficult to<br />
Copyright © 2006 The Authors 176
make this assumption that the valuations are known. Thus, we have to refer to a<br />
constructive way to first approximate the valuations.<br />
In our approach we assume that the price for bids is determined by three factors: (a) the<br />
load curve and the resultant eagerness factor, (b) the current price of similar services in the<br />
market and finally (c) the pricing policies of the user.<br />
(a) In order to determine the load that is required from the application within the considered<br />
time frame [t 0 ,, t 1 ] the equation l( t , t ) = max ( D( t)<br />
) is applied to ensure that the demand for<br />
0<br />
1<br />
[ t , t ]<br />
t∈<br />
0 1<br />
that entire period can be fulfilled. For the provider, the minimum is vice versa used to<br />
avoid overstraining the resource.<br />
The eagerness factor µ specifies how quickly consumers or providers will make<br />
concessions in the price. Thus, the eagerness factor can be represented by the equation µ<br />
= f(l(t 0 ,t 1 )) + g(t 0 – t), where the functions f and g can be considered as eagerness<br />
policies, which assign to each load D(t) and each time span t 0 - t a corresponding factor<br />
µ. The right panel of Figure 3 reflects common eagerness policies for the storage<br />
example, which rests on the intuition that the pressure to identify a corresponding<br />
request increases with growing load and decreases with shortening remaining time. The<br />
range of the eagerness factor is defined between zero and a positive rational number R.<br />
According to the policies in our running example this might lead to f(300GB) = 0.3 and<br />
g(60s-0s)=0.6. Consequently, we get an eagerness factor of µ=0.9.<br />
(b) The eagerness factor µ is now used to adapt the current market price m of a service with<br />
the same functionality and similar quality characteristics. This is achieved by means of<br />
the following equation, where p represents the proposed price: p = m µ. Note that in<br />
case µ is greater than one, the price of the bid might surpass the current market price. In<br />
our running example this leads to 90% of the current market price.<br />
However, since the price depends heavily on the quality of the service and it is not<br />
always possible to find services with similar quality characteristics in the market, there<br />
is a need for calculating prices for different quality levels. How this can be done based<br />
on user policies is presented in [3].<br />
(c) Assuring that the price does not adopt any <strong>und</strong>esired value, the provider might<br />
additionally specify pricing policies, which define the valid range for the service prices.<br />
Those pricing policies are not limited to linear prices but can also discriminatory prices<br />
to skim off consumer surplus of heterogeneous users.<br />
In this paper it is referred to rather simple pricing policies. The pricing policies (b) and<br />
(c) can be expressed by the following statements:<br />
price(?s,?p) ← Service(?s), minPrice(?s,?p min ), maxPrice(?s,?p max ), marketPrice(?s,?m),<br />
hasEagernessFactor(?µ), multiply(?m,?µ,?p), greaterThan(?p,?p min ), smallerThan(?p,?p max )<br />
price(?s,?p) ← Service(?s), minPrice(?s,?p), marketPrice(?s,?m), hasEagernessFactor(?µ),<br />
multiply(?m,?µ,?e), smallerThan(?e,?p)<br />
price(?s,?p) ← Service(?s), maxPrice(?s,?p), marketPrice(?s,?m), hasEagernessFactor(?µ),<br />
multiply(?m,?µ,?e), greaterThan(?e,?p)<br />
Quality of Service (QoS): The service offers and requests need to specify the guaranteed<br />
or requested QoS level, since this information is crucial for the subsequent agreement.<br />
However, QoS attributes are highly domain dependent and thus may vary from one service<br />
to another. Our notion of QoS includes attributes ranging from the capacity of a storage<br />
service to the response time or error rate of a computational service.<br />
Information about quality requirements or quality guarantees can be automatically<br />
derived from the computing node or from the application, respectively. For example, as<br />
indicator for the capacity that can be offered or is required in the time interval [t 0 ,, t 1 ] the<br />
value l(t 0 ,, t 1 ) can be used. For this purpose we use the formula cap = φ l(t 0 ,, t 1 ), where the<br />
rate φ allows systematic over-/<strong>und</strong>er-selling or over-/<strong>und</strong>er-buying. In many scenarios φ is<br />
required; especially when contractual penalties are high (typically φ < 1) or working to<br />
Copyright © 2006 The Authors 177
capacity (typically φ > 1) is important. In case of a storage provider a rate of φ = 0.95<br />
might be reasonable to reduce the risk of overselling. This means in the example we would<br />
offer storage capability of cap = φ l(t 0 ,, t 1 ) = 0.95 * 300GB = 285GB.<br />
Time Interval: Since the time interval [t 0 , t 1 ] represents the minimal period which can<br />
be sold to a single consumer or bought from a single service provider, the interval [t 0 , t 1 ]<br />
should also be the slot that is offered or requested in the bid. Moreover, the market<br />
mechanism being used within the framework supports aggregation of quality criteria and<br />
time. This means, several time slots can be aggregated in order to reach an agreement in the<br />
market. Hence, the issue of choosing slots that are too short is not relevant in this context.<br />
3.3. Bid Expansion<br />
Having generated the initial bid, the aim of this section is to improve the probability for<br />
finding a suitable request in the market by a semantic pre-processing of requests: On the<br />
one hand, a Grid service taxonomy is used to introduce alternative bids that can also be<br />
used to describe the corresponding service (vertical bid expansion). On the other hand,<br />
equivalence relations explicitly modelled in an ontology are used to address requests that<br />
could also be fulfilled by a combination of different services (horizontal bid expansion).<br />
Vertical bid expansion: Since offers and requests might be defined on different levels<br />
of abstraction pure syntactic matching may fail. Typically offers can be described very<br />
detailed since providers know exactly what kind of service they provide (e.g. Hard Disk<br />
Service), whereas customers usually define their requests on a rather abstract level (e.g.<br />
Storage Service). Consequently, in the market there will be no match between Hard Disk<br />
Service and Storage Service since the terms<br />
are compared only using strings. Therefore,<br />
we employ a Grid service ontology (c.f.<br />
Figure 3) that serves as backgro<strong>und</strong><br />
knowledge for adding additional XORrelated<br />
bids. By introducing these alternative<br />
bids according to Algorithm 1 we can<br />
overcome the different levels of abstraction<br />
while avoiding inappropriate matches.<br />
Figure 3 shows a taxonomy for storage services which is used to exemplify how bids<br />
can be vertically expanded. Note that since we use formal ontologies the taxonomic<br />
relations either are modelled explicitly or they are derived by means of automatic<br />
classification done by a reasoner. According to Algorithm 1 in case of offers we introduce<br />
an additional XOR-related bid for each concept that is passed when traversing to the root of<br />
the hierarchy. For instance, a HDD Service offer is expanded by adding an offer for a Static<br />
Storage Service and an offer for a Storage Service. This is necessary since a HDD Service<br />
also fulfills the functionality of a Static Storage Service and a Storage Service and thus<br />
should be matched with a request for the latter. In case of a request we have to traverse to<br />
the leafs of the taxonomy since more specific services also fulfill the functionality of the<br />
more general services, e.g. a HDD Service meets the requirements for a Storage Service<br />
request. However, since this expansion could lead to a high number of alternative bids it<br />
might be necessary to limit the depth of such an expansion, which can be done by means of<br />
an expansion policy.<br />
Horizontal bid expansion: Often a requested Grid service can not only be fulfilled by<br />
one single service, but also by a combination of different services. To account for this, the<br />
Grid ontology allows expressing composite services and equivalence relations: A service is<br />
either atomic or composite. Further, an equivalence relation can be defined between<br />
different services. This allows the explicit specification which services provide the same<br />
functionality. By means of the equivalent_to and contains relations new alternative bids can<br />
Copyright © 2006 The Authors 178
e introduced automatically to improve the probability or finding a suitable match in the<br />
market. For example, a mainframe computing service containing computation and storage<br />
can be expanded by adding a XOR-related b<strong>und</strong>le request the two constituting services [4].<br />
4. Conclusion<br />
The presented SORMA project primarily aims at promoting the widest and best possible<br />
use of Grid applications and services. This is attempted by devising incentive mechanisms<br />
that ICT resources are offered and requested in Grid. This requires that the service<br />
providers are compensated for offering their ICT resources by the service consumers. It is a<br />
premise of this paper that markets can <strong>und</strong>ertake this task better than any other coordination<br />
mechanism. This is especially the case if the bidding process is fully automated, such that<br />
the market itself will handle any unforeseen problem in the resource <strong>management</strong> process.<br />
By bringing autonomic and Grid computing as well as economic principles closer<br />
together, market-based approaches have the potential to achieve an economy-wide adoption<br />
of Grid technology and will thus leverage the exploitation of potential Grid technology<br />
offers. To achieve this potential, many Grid Economics related obstacles must be<br />
overcome: markets can only work well if bids accurately reflect demand and supply. If this<br />
is not the case, the market price cannot reflect the correct situation. Thus, the market price<br />
looses its capacity to direct scarce resources to the bidders who value them most.<br />
This paper is unique by recommending an autonomic computing approach for bidding<br />
in Grid markets. By defining policies the business models of the service consumers and<br />
providers can be represented. The proposed bid generation process is currently quite simple<br />
and referring to observations and straightforward rules.<br />
As the next steps, the bid generation process will be implemented and coupled with<br />
available Grid markets and middleware extensions. Subsequently, validation and economic<br />
testing will be performed, gradually improving the bid generation process (incorporating<br />
minimum probabilities of receiving the resources).<br />
References<br />
[1] Kee, Y.-S., H. Casanova, and A. Chien. Realistic Modeling and Synthesis of Resources for<br />
Computational Grids. in ACM Conference on High Performance Computing and Networking. 2004.<br />
[2] Kephart, J.O. and D.M. Chess, The Vision of Autonomic Computing. Computer, 2003. 36(1): p. 41-50.<br />
[3] Lamparter, S., A. Eberhart, and D. Oberle. Approximating Service Utility from Policies and Value<br />
Function Patterns. in International Workshop on Policies for Distributed Systems and Networks. 2005:<br />
IEEE Computer Society.<br />
[4] Lamparter, S. and B. Schnizler. Trading Services in Ontology-driven Markets. in 21st Annual ACM<br />
Symposium on Applied Computing. 2005. Dijon, France.<br />
[5] Ludwig, H., A. Dan, and B. Kearney. Cremona: An Architecture and Library for Creation and<br />
Monitoring of WS-Agreements. in 2nd International Conference on Service Oriented Computing<br />
(ICSOC 2004). 2004.<br />
[6] Motik, B., U. Sattler, and R. Studer, Query Answering for OWL-DL with Rules. Journal of Web<br />
Semantics: Science, Services and Agents on the World Wide Web, 2005. 3: p. 41-60.<br />
[7] Neumann, D., S. Lamparter, and B. Schnizler. Automated Bidding for Trading Grid Services. in<br />
European Conference on Information Systems (ECIS). 2006. Goeteborg, Sweden.<br />
[8] Schnizler, B., D. Neumann, and C. Weinhardt. Resource Allocation in Computational Grids - A Market<br />
Engineering Approach. in WeB 2004. 2004. Washington.<br />
[9] Sterritt, R., et al., A concise introduction to autonomic computing. Advanced Engineering Informatics,<br />
2005. 19(3): p. 181-187.<br />
Copyright © 2006 The Authors 179
A Decentralized Online Grid Market – Best myopic vs. rational response<br />
Dirk Neumann 1 , Nikolay Borissov 1 , Jochen Stoesser 1 ,<br />
and Simon See 2<br />
1 <strong>Institut</strong>e of Information Systems and Management, University Karlsruhe (TH), Germany<br />
2 Sun Microsystems, Singapore<br />
{neumann, stoesser, borissov}@iism.uni-karlsruhe.de; simon.see@sun.com<br />
1. Introduction<br />
Grid computing is a new computing paradigm where processing is distributed across locally dispersed<br />
infrastructures. By connecting many heterogeneous computing resources, virtual computer architectures are<br />
created, increasing the utilization of otherwise idle resources [1]. Grid computing extends the traditional parallel<br />
and distributed computing paradigms as it deals with resources belonging to different administrative domains.<br />
One of the key problems in Grids is the scheduling of jobs, i.e. the decision about what jobs are to be<br />
executed on which machines at what time. Grid computing is hampered by the fact that different administrative<br />
units are involved. Priorities of jobs are communicated by the respective agents, thus opening up possibilities for<br />
strategic misreporting of true job priorities. In these heterogeneous and decentralized settings, markets are<br />
supposed to work well as the pricing mechanisms of markets may force the participants to reveal their true<br />
priorities [2]. Looking at existing market mechanisms, however, there are currently only few mechanisms which<br />
perform real-time scheduling for Grid applications, while at the same time appropriately accounting for the<br />
arising economic design challenges.<br />
To this end, the contributions of this paper are threefold. Firstly, we present the Decentralized Local Greedy<br />
Mechanism (henceforth “DLGM”) by Heydenreich et al. [3], a mechanism from the general machine scheduling<br />
domain which exhibits several desirable properties, and propose its use for Grid environments. Secondly, the<br />
theoretical analysis of DLGM is based on the assumption of simple agents behaving according to a so called<br />
“myopic best response strategy”. By means of a simulation with learning agents, we show that an analysis using<br />
myopic best response strategies overestimates the performance of the mechanism and is thus not an appropriate<br />
solution concept for modeling real-world market-based scheduling mechanisms. As a byproduct of this analysis,<br />
we illustrate the performance benefits of market-based schedulers as opposed to purely technical schedulers<br />
which are solely based on system-centric measures. Thirdly, we point at limitations of the mechanism for the<br />
practical use and suggest remedies that may help to mitigate these drawbacks.<br />
1<br />
180
This paper is structured as follows. In Section 2, we present related work on the design of Grid market<br />
mechanisms. In Section 3, we present the DLGM mechanism and the concept of myopic best response equilibria<br />
used to analyze its economic worst-case performance. In Section 4, we model agents with learning capabilities,<br />
and contrast the DLGM for myopic best response agents with agents with learning capabilities by means of a<br />
numerical simulation. In Section 5, we point to limitations and possible extensions of the basic DLGM<br />
mechanism, before we conclude the paper in Section 6.<br />
2. Related Work<br />
Economic scheduling mechanisms can be distinguished into two classes according to their mode of<br />
allocation: mechanisms which execute periodically, so called “offline mechanisms”, and mechanisms which<br />
execute continuously, so called “online mechanisms” [4].<br />
In classic mechanism design, offline mechanisms have received the most interest. Examples comprise the<br />
simple Vickrey auction, English auction, Dutch auction, and double auctions [5, 6]. However, many modern<br />
applications in E-Commerce, and in particular in Grid scheduling, involve complementarities between goods.<br />
Examples of the resulting combinatorial mechanisms are MACE [7, 8], Bapna et al [9], and Stoesser et al [10,<br />
11]. All these mechanisms involve a scheduling problem that is NP-hard.<br />
Online mechanism design also allows agents to enter or leave the mechanism dynamically. Parkes et al.<br />
propose an NP-hard online Vickrey-Clarke-Groves (VCG) mechanism [12]. In a subsequent work, Parkes and<br />
Singh provide an approximation that retains the mechanism’s truth-telling property [13]. The computational<br />
complexity of such mechanisms drives researchers and practitioners to look at simpler allocation models which<br />
can be adapted to real-world scenarios and requirements. Prominent examples are proportional share<br />
mechanisms [14], where the users receive a share of the computer resource proportional to their valuation’s<br />
fraction of the overall valuation across all users. A related online mechanism is the so-called pay-as-bid<br />
mechanism proposed in [15]. In pay-as-bid mechanisms, the agents submit only a single real value as bids which<br />
at the same time make the agents’ final payments.<br />
Heydenreich et al. propose a Decentralized Local Greedy Mechanism (DLGM) [3]. DLGM is a promising<br />
economic mechanism for scheduling interactive Grid applications due to its polynomial runtime, budgetbalanced<br />
payments and incentive compatibility. In the following section, we will introduce the DLGM<br />
mechanism in more detail as we will use it to contrast the game theoretic solution concept of myopic best<br />
response strategies with a learning-based approach.<br />
2<br />
181
3. A Decentralized Local Greedy Mechanism<br />
3.1. The Setting<br />
In the following, a Grid environment consists of a number of individual agents, which are assumed to act<br />
rationally in the sense that they strive for maximizing their individual benefit from participating in the Grid.<br />
Agents submit computational jobs to the Grid, where a job j is specified by the tuple (its “type”) t j = (r j , p j ,w j ).<br />
The job’s release date is denoted by r j ∈<br />
+<br />
+<br />
R o , the job’s processing time by p j ∈ R , and the agent’s cost of<br />
+<br />
waiting for one additional unit of time until the job is finished (its “weight”) by w j ∈ R<br />
. The quasi-linear utility<br />
function of agent j is defined to be<br />
+<br />
u ( C , π | t ) = −w<br />
C − π , where C j ∈ R , C j ≥ p j , marks j’s completion<br />
j<br />
j<br />
j<br />
j<br />
j<br />
j<br />
j<br />
time, and π<br />
j<br />
∈<br />
+<br />
R o is j’s payment. When requesting resources for the execution of job j, the agent may<br />
~<br />
t =<br />
~ r ,<br />
~<br />
p , w<br />
~<br />
≠ t<br />
strategically choose to submit the request ( )<br />
j j j j j<br />
to the Grid. In the following, we will use thze<br />
terms “job” and “agent” interchangeably. We consider a decentralized setting where the action space of agent j<br />
~<br />
does not only consist of reporting its type t j , but also of subsequently choosing a machine at which to queue.<br />
We will henceforth denote agent j’s action, or strategy, by s j , and we will also write ( s,<br />
t)<br />
for agent j’s utility<br />
given the strategy vector s = ( s ,..., ) and the true types t ( t ,..., )<br />
1<br />
s n<br />
= .<br />
The Grid infrastructure consists of m identical, parallel machines M = {1, ..., m}. The strategic analysis of<br />
this setting is restricted to strategic behavior on the demand side only. Machines are assumed to report truthfully<br />
about their status. Each job can be executed on any machine, but each machine can execute at most one job at a<br />
time.<br />
1<br />
t n<br />
u j<br />
3.2. The Mechanism<br />
For this setting, we propose the use of the DLGM mechanism suggested by [3]. The mechanism aims at<br />
minimizing the total weighted completion time Σw j C j across all jobs. DLGM comprises the following three<br />
steps:<br />
Step 1 – Job submission<br />
At release date<br />
~<br />
r j , j reports w ~ j and<br />
~<br />
p j to every machine m ∈ M.<br />
Step 2 – Real-time planning<br />
Based on this information, the machines perform real-time planning based on a local scheduling policy. Job j<br />
3<br />
182
has a higher priority value than k ⇔<br />
w<br />
~<br />
~<br />
p<br />
j<br />
j<br />
≥<br />
w<br />
~<br />
~<br />
p<br />
k<br />
k<br />
. Then j is scheduled before job k in the waiting queue.<br />
Depending on the current local waiting queue which resulted from the previous strategies<br />
s − j<br />
of all other agents,<br />
and j’s report ~ t , machine i reports the tentative completion time ˆ ~<br />
~<br />
)<br />
j<br />
C ( i | s− , t to agent j. Cˆ<br />
( i | s<br />
−<br />
, t ) consists<br />
of j’s own runtime plus the runtime of the job which is currently being executed on machine i (if any) and the<br />
aggregated runtimes of the jobs which would be queuing in front of job j. Furthermore, the mechanism computes<br />
a tentative payment ˆ<br />
~<br />
π ( i | s , t ) according to the prominent Vickrey principle. Agent j has to compensate all<br />
j<br />
− j<br />
j<br />
agents whose jobs are currently waiting at machine i and which are delayed due to j being assigned to i for their<br />
additional waiting cost. These values are only tentative as later arriving jobs might displace job j. Consequently,<br />
the tentative utility of agent j at machine i at time ~ ) ~ ) ~<br />
~<br />
r is u i | s , t , t ) = −w C ( i | s , t ) −<br />
) π ( i | s , t )<br />
j<br />
Step 3 – Machine selection<br />
j<br />
j<br />
j<br />
j<br />
(<br />
− j j j<br />
j j − j j j − j j<br />
Upon receiving information about its tentative completion time and required payment from the machines, the<br />
agent makes a binding decision to queue at a machine i, and pays ˆ<br />
~<br />
π ( i | s , t ) to the delayed jobs.<br />
j<br />
− j<br />
j<br />
j<br />
j<br />
j<br />
.<br />
3.3. Solution Concept and Theoretical Performance<br />
When evaluating the performance of a market mechanism, we first need to fix a solution concept which<br />
~<br />
defines the agents’ behavior, i.e. their reports of t<br />
j based on tj and their valuation for a given allocation.<br />
In the model of Heydenreich et al. [3], agents do not remember the outcomes of earlier market interactions<br />
but are somewhat “myopic” in the sense that they only consider the current situation. Agents choose the action<br />
which maximizes their tentative utility, given the machines’ tentative feedback. Formally:<br />
Definition 1 (“Myopic best response equilibrium”): A strategy profile s is called a myopic best response<br />
equilibrium if, for all agents j ∈ J, all agent types t, all strategies<br />
instead of s j , uˆ<br />
(( s , s ), t)<br />
≥ uˆ<br />
((<br />
~<br />
s , s ), t)<br />
j j − j<br />
j j − j<br />
.<br />
s − j<br />
, and all strategies<br />
~<br />
s<br />
j which j could play<br />
As shown in [3], without knowledge about the future, at time<br />
~<br />
rj<br />
it is a utility maximizing strategy sj for agent<br />
) ~<br />
j ∈ J to report its true type t j to the system and to choose the machine i which maximizes u i | s , t , t ) . We<br />
will henceforth refer to s j as being a “myopic best response strategy”.<br />
A common metric for a mechanism's performance is its “performance ratio”:<br />
(<br />
j − j j j<br />
Definition 2 (“Performance ratio”): Let Z A<br />
(J)<br />
be the objective value generated by an online mechanism A<br />
4<br />
183
for a given sequence J of jobs and let Z<br />
* ( J ) be the (optimal) objective value generated by an omniscient offline<br />
mechanism for this given job sequence. Then, for the class ψ of all possible job sequences, the performance<br />
⎧Z A<br />
A ( J ) ⎫<br />
ratio of mechanism A is ρ = ρ Ψ<br />
= sup ⎨ | J ∈ Ψ⎬<br />
.<br />
*<br />
⎩ Z ( J ) ⎭<br />
Assuming all agents play their equilibrium strategy, DLGM achieves a performance ratio of ρ = 3.281.<br />
4. Evaluation<br />
In this section, we examine two main questions which arise from the theoretical results in [3]:<br />
Myopic best response vs. rational response: Given agents with learning capabilities, does the concept of<br />
myopic best response equilibria adequately model agent behavior? Or do learning capabilities lead to some other<br />
form of equilibrium, which may be called a “rational response equilibrium”?<br />
Worst-case vs. average case: What is the average case performance of the DLGM as opposed to the worstcase?<br />
How does it compare to FIFO as a purely technical allocation mechanism, and how does it change if we<br />
introduce agents with learning capabilities?<br />
4.1. Data Generation<br />
For simplification, we assume that agents can only misreport about their weight w . As presented above, one<br />
j<br />
central result is that myopic best response agents then truthfully report<br />
w<br />
~<br />
= w .<br />
j<br />
j<br />
Our aim is to simulate rational agents with learning capabilities instead which may strategically misreport<br />
about<br />
w<br />
j<br />
based on their past behavior and payoffs. We will refer to these strategies as “rational response<br />
strategies”. To model the learning capabilities of these agents, we chose a reinforcement learning-based<br />
approach with a greedy selection policy [16]. This approach comprises the following two phases:<br />
• Exploration phase: In this phase, the agents simultaneously explore the strategy space and heuristically try<br />
to learn the expected payoffs of various strategies to finally decide on an expectedly optimal strategy. Within<br />
this exploration phase, we perform 25 runs of the market. In each run k, every agent j reports<br />
k<br />
w ~ j<br />
and p to<br />
j<br />
the market at time r j<br />
and, upon having received the feedback about its tentative completion time and<br />
) ~<br />
payment from each machine, selects the machine i which maximizes its tentative utility u i | s , t , t ) ,<br />
where<br />
k<br />
w ~<br />
j<br />
(<br />
j − j j j<br />
is an integer which is randomly drawn from the uniform distribution with support [0, 2w j ] in order<br />
to model both the <strong>und</strong>er- and overstating of weights. Subsequent to each run k, agent j determines its actual<br />
5<br />
184
k<br />
ex-post utility (i.e. its “reward”) u ((<br />
~<br />
s , s ), t)<br />
for ro<strong>und</strong> k. After all 25 runs, agent j greedily determines its<br />
j j − j<br />
best strategy * *<br />
k<br />
s as s = arg max u ((<br />
~<br />
s , s ), t)<br />
. Since the strategy space of agent j consists of strategically<br />
j j<br />
j j − j<br />
k<br />
reporting<br />
w ~<br />
j<br />
only, we can simplify this to the greedy optimal strategy being the report of<br />
~ *<br />
w which yielded<br />
j<br />
the maximum ex-post utility across all 25 runs.<br />
• Exploitation phase: In this phase, the agents try to “exploit” the information which they obtained in the<br />
previous exploration phase. Using a greedy selection policy, agent j assumes that, since the report of<br />
~ *<br />
w<br />
j<br />
maximized its ex-post utility given its true weight<br />
w in the past, reporting a fraction of<br />
j<br />
~ for the weights<br />
of all subsequent jobs will also maximize its future payoff. In order to examine this strategy, we also ran the<br />
exploitation phase for 25 ro<strong>und</strong>s. In each ro<strong>und</strong>, we generated one job per agent based on the same<br />
distributions as in the exploration phase as specified in Table 1.<br />
In order to compare the myopic best response strategy and the rational response strategy for the DLGM<br />
mechanism, we ran six settings, whereas each setting comprised 5 machines. To increase the competition in the<br />
market, across the course of these settings, we successively increased the Poisson-based arrival rate of jobs from<br />
λ = .2 to λ = 1as listed in Table 1, which also specifies the random choices of the processing times and weights.<br />
Each job sequence was then fed into both the DLGM mechanism with agents playing myopic best response<br />
strategies and in the DLGM mechanism with agents playing what we call rational response strategies. Moreover,<br />
we simulated a simple FIFO mechanism as an example of a technical scheduler to determine the benefit of<br />
involving users into the scheduling process by enabling them to dynamically submit weights for jobs instead of<br />
assigning static weights.<br />
Setting S1 S2 S3 S4 S5 S6<br />
Arrival time λ of the Poisson-based release dates r<br />
j .2 .5 .6 .7 .8 1<br />
Actual number of jobs 208 440 559 686 795 1050<br />
Processing time p<br />
j<br />
Lognormal distribution with mean 9 and variance 3<br />
Weights w<br />
j<br />
Normal distribution with mean 9 and variance 3<br />
Table 1: Simulation settings<br />
w *<br />
j<br />
w<br />
j<br />
4.2. Results<br />
The results of the simulations are summarized in Table 2. As aforementioned, we tested a FIFO mechanism<br />
as variant of a technical scheduler, the DLGM with myopic best response and the DLGM with learning agents in<br />
six settings with varying competition. The utilization being defined as the number of jobs where the mechanism<br />
6<br />
185
is busy divided by the total number of time slots represents a good metric for expressing the level of competition.<br />
As all three mechanisms do not impose reservation prices, utilization is identical for all three mechanisms. For<br />
an arrival rate of λ = . 2 .with a job sequence of 208 jobs, the machines are only used 38 % of the time. When the<br />
size of the job sequence increases to 559 jobs, the utilization rate grows up to 77 %, and the machines are fully<br />
utilized for settings with more than 700 jobs.<br />
When competition is low, technical schedulers achieve as good welfare (i.e. total weighted completion time)<br />
as DLGM does. However, with increasing utilization, technical schedulers are starting to perform worse than<br />
market mechanisms. When competition is extremely high, FIFO generates about 13 % higher overall waiting<br />
costs than DLGM with agents playing rational response strategies. This result is rather straightforward, as<br />
technical schedulers do not account for the relative urgency of jobs. When utilization is low, however, the<br />
welfare loss incurred by technical schedulers is insignificant. In the absence of machine prices, the use of market<br />
mechanisms for scheduling is accordingly only advantageous if excess demand for resources exist. Although<br />
there are strong indicators that demand for Grid resources exceeds supply [17], most papers in the so-called Grid<br />
economy overestimate the goodness of market mechanisms [18].<br />
Furthermore, another important result of our simulation is that myopic best response is an adequate proxy for<br />
rational bids only when utilization is low. Once competition increases, agents have an even stronger incentive to<br />
manipulate their weights reported to the mechanism. More specifically, the jobs have an incentive to <strong>und</strong>erstate<br />
their true weight, due to the fact that they have to compensate all other jobs in the queue for being scheduled<br />
earlier. The longer the waiting queue becomes, the more expensive truthful reporting becomes, as all jobs in the<br />
schedule needs to be compensated for their delay. This effect grows for the setting with more than 1,000 jobs, as<br />
learning jobs report on average only 19 % of their true weight. Naturally, the DLGM looses much of its power to<br />
prioritize jobs according to their urgency. However, when competition is high, the results of a realistic DLGM,<br />
where jobs learn to adjust their weight reports, are still better than for technical schedulers but way less than the<br />
concept of myopic-best response would suggest. This leads us to the conclusion that best myopic response<br />
behavior is only an adequate concept when competition is low – in those cases, however, the use of market<br />
mechanisms does not make sense, as technical schedulers are performing equally well.<br />
Our simulation also yields that the fraction of jobs that receive an overall positive payment does not depend<br />
on the learning capabilities of the jobs. When competition is low, the fraction of jobs earning positive<br />
compensation payments is as low as 1 %, and grows up to almost 40 %, when competition gets fierce. The<br />
employment of compensation payments opens up a way how to cheat the mechanism, as we will show in the<br />
7<br />
186
next section. This possibility does generally hold and is not restricted to myopic best response.<br />
Another important result of our simulation is that the planning quality of the DLGM becomes negligible<br />
when competition gets fierce. Essentially, the DLGM seems to be nice as it reports back a tentative completion<br />
time. As competition becomes fierce, this tentative completion time is overly extended. The lateness metric<br />
measures the time difference between ex-post and tentative completion time. As Table 2 shows, the lateness<br />
extends considerably so that the tentative completion time loses its power for prediction. Jobs cannot rely on<br />
those time reports, but should add considerably more time. Intuitively, lateness occurs in the same way for<br />
learning jobs.<br />
Metric Mechanism\Setting S1 S2 S3 S4 S5 S6<br />
Utilization All mechanisms 38% 77% 91% 100% 100% 100%<br />
%-bid of best<br />
myopic<br />
response weight<br />
DLGM with learning agents 102% 90 % 65 % 25 % 22 % 19 %<br />
Aggregated<br />
waiting cost<br />
Positive<br />
Compensations<br />
Lateness<br />
FIFO -916,634 -1,993,694 -2,676,904 -3,633,538 -5,046,646 -8784801<br />
DLGM with myopic best<br />
-7042589<br />
-916,701 -2,001,682 -2,650,274 -3,340,492 -4,370,345<br />
response<br />
DLGM with learning agents -916,717 -2,001,787 -2,663,577 -3,485,629 -4,676,194 -7802760<br />
DLGM with myopic best<br />
response<br />
1% 7% 16% 18% 23% 34%<br />
DLGM with learning agents 1% 8% 18% 20% 25% 36%<br />
DLGM with myopic best<br />
response<br />
.002 .073 1.13 8.72 20.73 58.99<br />
DLGM with learning agents .0021 .1056 1.198 8.6589 19.9821 58.79<br />
Table 2: Simulation results<br />
5. Limitations of the Mechanism and Possible Extensions<br />
As the simulation results suggest, despite exhibiting some striking features, DLGM also suffers from various<br />
drawbacks. In this section, we outline a few of the identified limitations of the basic mechanism and point to<br />
extensions that may alleviate these problems.<br />
First, DLGM does not fully account for the inter-organizational nature of Grids; strategic and self-interested<br />
behavior is modeled on the demand side only. The supply side, i.e. the machines offering their service for the<br />
execution of jobs, is assumed to be “obedient”. Moreover, with the basic setting, machines do not get any reward<br />
for contributing to the system. The design of a mechanism which accounts for strategic behavior on both sides of<br />
the market serves as a goal for future research.<br />
Another weakness of DLGM is that jobs may be able to exploit its payment scheme. If a job does not displace<br />
any present job, e.g. because it reported a low valuation, it does not have to make an immediate payment.<br />
However, it may subsequently receive compensation payments from the overtaking jobs. A relatively simple<br />
remedy for preventing jobs from exploiting the mechanism to earn money may be to add a variable cost<br />
+<br />
component to the payment scheme. This can be done by introducing z ∈R<br />
i<br />
which equals the operating costs<br />
8<br />
187
per timestep for machine i. When extending the scenario to feature strategically acting machines which can<br />
misreport about<br />
Ĉ j<br />
and<br />
πˆ<br />
j<br />
, machines may be allowed to strategically report their operating costs z i<br />
~ . To<br />
incorporate these operating costs in the mechanism, the tentative payment<br />
ˆ π ( i | s<br />
j<br />
− j<br />
~ ⎛<br />
, t ) =<br />
~<br />
p ⎜ ~ z +<br />
∑ w<br />
~<br />
j j i<br />
k∈L(<br />
j ), k→i,<br />
k<<br />
j,<br />
⎝<br />
k<br />
S k > r<br />
~ j<br />
⎞<br />
⎟<br />
⎠<br />
πˆ<br />
j<br />
has to be extended to<br />
Allowing preemption of jobs may further increase the efficiency of the mechanism. In DLGM in its current<br />
form, a job which is being processed at a machine may not be interrupted until it is completed. Grid applications<br />
demand for the possibility to freeze the execution of a running job if a job with a higher priority enters the queue.<br />
Deadline constraints are a further possible extension to the mechanism. With interactive settings,<br />
applications / users need the jobs to be computed in a timely manner and the validity of the job is limited. An<br />
extension to the basic mechanism is thus to expand a job’s (agent’s) type to t = r , p , w , d ) where<br />
+<br />
d ∈R<br />
j<br />
, d r + p<br />
j j j<br />
j<br />
(<br />
j j j j<br />
~<br />
~<br />
≥ , is j’s deadline and agent j reports t = (<br />
~<br />
r ,<br />
~<br />
p , w<br />
~<br />
, d ) to the machines. This additional<br />
j<br />
j<br />
j<br />
j<br />
j<br />
information can then be used to adjust the priority of jobs when ordering the waiting queue such that jobs which<br />
approach their deadline receive a higher priority. Assume a new job j reports to the system. Then, at time<br />
~<br />
rj<br />
, the<br />
priority values of all jobs have to be recomputed. A measure for the urgency of a particular job k is its distance to<br />
the point in time at which it needs to be started at the latest in order to still be finished before its deadline, i.e.<br />
~<br />
d −<br />
~<br />
p −<br />
~<br />
r . The smaller this distance, the more the priority value of job k needs to be boosted. One possibility<br />
k<br />
k<br />
k<br />
is to use<br />
~<br />
p<br />
k<br />
w<br />
~<br />
k<br />
~<br />
( d −<br />
~<br />
p −<br />
~<br />
r )<br />
k k k<br />
as priority value for job k at time rj<br />
~ w<br />
~<br />
instead of<br />
k<br />
~<br />
pk<br />
.<br />
Overall, with such extensions, the DLGM mechanism is a promising approach towards scheduling in Grids.<br />
6. Conclusion and Outlook<br />
In Section 2, we briefly presented market mechanisms which have been proposed for the use in Grid<br />
scheduling. In Section 3, we focused on the DLGM mechanism which exhibits several desirable features. In their<br />
analysis, Heydenreich et al. [3] propose the use of the solution concept of myopic best response equilibria in<br />
order to model the behavior of strategic agents. In order to examine the validity of this concept and the<br />
performance of the mechanism in more realistic settings with learning agents, we performed an agent-based<br />
simulation in Section 4. The central result of our analysis is that myopic best response strategies do not<br />
adequately model more realistic agents. Furthermore, myopic best response strategies distort the performance<br />
9<br />
188
analysis of DLGM and yield overly optimistic results. In Section 5, we then pointed to several limitations of the<br />
basic DLGM mechanism if considered for practical settings, and pointed at extensions to this basic mechanism<br />
which may help to mitigate these drawbacks.<br />
The presented results point to several interesting possibilities for future research. From the mechanism design<br />
perspective, it would be interesting to evaluate the suggested extensions in more detail. We plan to implement<br />
the mechanism in the Sun N1 Grid Engine, a state-of-the-art Grid scheduler by Sun Microsystems. This may<br />
help in identifying both further limitations of this approach and possible remedies and extensions. The agent<br />
perspective suggests two ways to extend on our research. Firstly, one f<strong>und</strong>amental question is how actual users<br />
can discover their true valuations and interact with the mechanism. First research in this direction has been<br />
presented in [19] and [20], who suggest some form of “intelligent” software agents as intermediaries which assist<br />
the user in determining job priorities and communicating with the market. Secondly, the learning capabilities and<br />
strategies of the (software) agents need further refinement and validation.<br />
Bibliography<br />
1. Foster, I., C. Kesselman, and S. Tuecke, The Anatomy of the Grid: Enabling Scalable Virtual Organizations.<br />
International Journal of High Performance Computing Applications, 2001. 15(3): p. 200-222.<br />
2. Sutherland, I.E., A futures market in computer time. Communications of the ACM, 1968. 11(6): p. 449-451.<br />
3. Heydenreich, B., R. Müller, and M. Uetz., Decentralization and mechanism design for online machine scheduling., in<br />
SWAT, L.A.a.R. Freivalds, Editor. 2006, Springer. p. 136-147.<br />
4. Neumann, D., J. Stoesser, and C. Weinhardt, Bridging the Adoption Gap - Developing a Roadmap for Trading Grids.<br />
International Journal of Electronic Markets, 2008. 18(1): p. forthcoming.<br />
5. Grosu, D. and A. Das, Auctioning resources in Grids: model and protocols. Concurrency and Computation: Practice<br />
and Experience, 2006. 18(15): p. 1909-1927.<br />
6. Waldspurger, C.A., et al., Spawn: A distributed computational economy. IEEE Transactions on Software Engineering<br />
Management Journal, 1992. 18(2): p. 103–117.<br />
7. Schnizler, B., et al., Trading Grid Services - A Multi-attribute Combinatorial Approach. European Journal of Operation<br />
Research, 2006: p. forthcoming.<br />
8. Schnizler, B., D. Neumann, and C. Weinhardt. Resource Allocation in Computational Grids - A Market Engineering<br />
Approach. in WeB 2004. 2004. Washington.<br />
9. Bapna, R., et al., A market design for grid computing. INFORMS Journal of Computing, 2006: p. fortcoming.<br />
10. Stoesser, J. and D. Neumann. GREEDEX – A Scalable Clearing Mechanism for Utility Computing. in Networking and<br />
Electronic Commerce Research Conference (NAEC) 2007. Riva Del Garda, Italy<br />
11. Stoesser, J., D. Neumann, and A. Anandasivam. A Truthful Heuristic for Efficient Scheduling in Network-Centric Grid<br />
OS. in European Conference on Information Systems (ECIS 2007). 2007. St. Gallen.<br />
12. Parkes, D.C., S. Singh, and D. Yanovsky. Approximately Efficient Online Mechanism Design. in 18th Annual<br />
Conference on Neural Information Processing Systems (NIPS’04). 2004.<br />
13. Parkes, D.C. and S. Singh. An MDP-based approach to Online Mechanism Design. in 17th Annual Conference on<br />
Neural Information Processing Systems (NIPS’03). 2003.<br />
14. Lai, K., et al., Tycoon: an Implementation of a Distributed, Market-based Resource Allocation System. Working Paper,<br />
2004.<br />
15. Hajek, S.S.a.B. Optimal allocation of a divisible good to strategic buyers. in IEEE Conference on Decision and<br />
10<br />
189
Control. 2004.<br />
16. Kaelbling, L.P., M.L. Littman, and A.W. Moore, Reinforcement Learning: A Survey. Journal of Artificial Intelligence<br />
Research, 1996. 4: p. 237-285.<br />
17. Fu, Y., et al., SHARP: an architecture for secure resource peering. ACM SIGOPS Operating Systems Review, 2003.<br />
37(5): p. 133-148.<br />
18. Buyya, R., D. Abramson, and S. Venugopal, The grid economy. Proceedings of the IEEE, 2005. 93(3): p. 698-714.<br />
19. MacKie-Mason, J.K. and M.P. Wellman, Automated Markets and Trading Agents, in Handbook of Computational<br />
Economics, vol. 2: Agent-Based Computational Economics, L. Tesfatsion and K.L. Judd, Editors. 2006, North-Holland.<br />
20. Neumann, D., S. Lamparter, and B. Schnizler, Automated Bidding for Trading Grid Services, in Proceedings of the<br />
14th European Conference on Information Systems (ECIS). 2006: Gothenburg, Sweden.<br />
11<br />
190
Economically Enhanced Resource Management<br />
for Internet Service Utilities<br />
Tim Püschel 1,2 , Nikolay Borissov 2 ,MarioMacías 1 ,DirkNeumann 2 ,<br />
Jordi Guitart 1 , and Jordi Torres 1<br />
1 Barcelona Supercomputing Center - Technical University of Catalonia(UPC),<br />
c/ Jordi Girona 29, 08034 Barcelona, Spain<br />
{tim.pueschel,mario.macias,jordi.guitart,jordi.torres}@bsc.es<br />
2 <strong>Institut</strong>e of Information Systems and Management (<strong>IISM</strong>)<br />
Universität Karlsruhe (TH), Englerstr. 14, 76131 Karlsruhe, Germany<br />
{pueschel,borissov,neumann}@iism.uni-karlsruhe.de<br />
Abstract. As competition on global markets increases the vision of utility<br />
computing gains more and more interest. To attract more providers it<br />
is crucial to improve the performance in commercialization of resources.<br />
This makes it necessary to not only base components on technical aspects,<br />
but also to include economical aspects in their design. This work<br />
presents an framework for an Economically Enhanced Resource Manager<br />
(EERM) which features enhancements to technical resource <strong>management</strong><br />
like dynamic pricing and client classification. The introduced approach<br />
is evaluated considering various economic design criteria and example<br />
scenarios. Our preliminary results, e.g. an increase in achieved revenue<br />
from 77% to 92% of the theoretic maximum in our first scenario, show<br />
that our approach is very promising.<br />
1 Introduction<br />
Many web applications have strongly varying demand for computing resources.<br />
To fulfil these requirements sufficient resources have to be made available. In<br />
some cases it is possible to run certain tasks at night to achieve a more even<br />
usage, however in many cases it is not feasible to let users wait a long time<br />
for results. This leads to a situation in which utilization is very high during<br />
certain peak times while many resources lay idle during other times. Due to<br />
high competition on global markets many enterprises face the challenge to make<br />
use of new applications and reduce process times on one side and cut the costs<br />
of their IT-infrastructures on the other side [5].<br />
In light of this challenge the idea of utility computing gained interest. Utility<br />
computing describes a scenario where computer resources can be accessed dynamically<br />
in analogy to electricity and water [19]. The more resource providers<br />
offer their resources or services, the more likely it is they can be accessed at<br />
competitive prices. Therefore it is important to attract more providers.<br />
However providers will only offer their services if they can realize sufficient<br />
benefit. With state-of-the-art technology, this assimilation is hampered, as the<br />
B. Benatallah et al. (Eds.): WISE 2007, LNCS 4831, pp. 335–348, 2007.<br />
c○ Springer-Verlag Berlin Heidelberg 2007<br />
191
336 T. Püschel et al.<br />
local resource managers facilitating the deployment of the resources are not<br />
designed to incorporate economic issues (e.g. price).<br />
In recent times, several research projects have started to develop price-based<br />
resource <strong>management</strong> components supporting the idea of utility computing.<br />
Those approaches are devoted to scheduling by utilizing the price mechanism.<br />
Clearly, this means that technical issues such as resource utilization are ignored<br />
for scheduling. In addition, resource <strong>management</strong> is much more comprehensive<br />
than just scheduling. For example Service-Level-Agreement (SLA) <strong>management</strong><br />
is also part of resource <strong>management</strong> that is often omitted in economic approaches.<br />
This plays a role when deciding which already ongoing jobs to cancel<br />
in overload situations to maintain system stability.<br />
To improve performance in the commercialization of distributed computational<br />
resources decisions about the supplied resources and their <strong>management</strong><br />
should be based on both technical and economic aspects [12]. Hence, this paper<br />
is an interdisciplinary work taking into account aspects from computer science<br />
and economics. We will explore the use of economic enhancements such as client<br />
classification and dynamic pricing to resource <strong>management</strong>.<br />
Technical resource <strong>management</strong> systems typically offer the possibility to include<br />
priorities for user groups. In purely price-based schedulers it is not possible<br />
to distinguish important from unimportant partners, as only current price matters<br />
for the allocation. We will motivate that client classification should be integrated<br />
into economically enhanced resource <strong>management</strong> systems. Essentially,<br />
there are two main reasons to do so: First, client classification allows the inclusion<br />
of long-term oriented relationships with strategically important customers<br />
so-called credential components. Second, client classification can be used as an<br />
instrument of revenue <strong>management</strong>, which allows skimming off consumer surplus.<br />
The main contribution of this paper is to show how technical parameters can be<br />
combined into an economically enhanced resources <strong>management</strong> that increases<br />
revenue for the local resource sites.<br />
2 Motivational Scenarios<br />
2.1 Organisation Selling Spare Resource Capacity<br />
An organisation offers different web services for internal use. However the utilization<br />
of the resources is uneven. During certain times there is only a low load,<br />
the resources are almost idle, and at other times users have to wait for their<br />
results very long. Therefore the organisation decides to buy more capacity on<br />
the market when needed and sell its spare capacity when the load is low. When<br />
accepting jobs users from within the organisation should always be preferred.<br />
To provide and communicate a good dynamic evaluation of resources, market<br />
mechanisms are also used for internal users. This also gives internal users an<br />
incentive to run their jobs during times of low utilization. Internal users also<br />
receive a significant discount compared to external users.<br />
192
Economically Enhanced Resource Management for Internet Service Utilities 337<br />
2.2 Resource/Service Provider with Preferred Customers<br />
A big service provider maintains a data center whose resources are sold. The<br />
service provider already has a number of clients but still has a high spare capacity.<br />
Therefore he joins a marketplace to find new clients and optimize capacity<br />
utilization. To maintain the good relations with the current clients and encourage<br />
regular use of its services the provider offers a preferred client contract.<br />
Preferred clients receive a discount on the reservation price and soft preference<br />
when accepting jobs. Soft preference means that their bids are increased by a<br />
certain amount for the winner determination. When offering the same price the<br />
preferred client is chosen over the external client, but the standard client has the<br />
chance to outbid the preferred client. Prices should be calculated dynamically<br />
based on utilization, client classification, estimated demand and further pricing<br />
policies of the provider.<br />
3 Economically Enhanced Resource Management<br />
To improve performance in the commercialization of distributed computational<br />
resources and increase the benefit of service providers we propose the introduction<br />
of various economic enhancements into resource <strong>management</strong>. This chapter<br />
first describes the objectives and requirements for the enhancements, then follows<br />
a description of the key mechanisms that are to be integrated in the EERM.<br />
The chapter ends with a description of the architecture and the components of<br />
the EERM.<br />
3.1 Objectives and Requirements<br />
The main goals of these enhancements are to link technical and economical<br />
aspects of resource <strong>management</strong> and strengthen the economic feasibility. This<br />
can be achieved by establishing more precise price calculations for resources,<br />
taking usage of the resources, performance estimations and business policies into<br />
account. The introduced mechanisms should be be able to deal efficiently with<br />
the motivational scenarios given earlier. This means they have to feature client<br />
classification, different types of priorities for jobs from certain clients, reservation<br />
of a certain amount of resources for important clients, and dynamic calculation<br />
of prices based on various factors. In addition to these requirements the system<br />
should also offer quality of service (QoS) and be able to deal with situations in<br />
which parts of the resources fail. To adapt to different scenarios and business<br />
policies of different situations it should be highly flexible and configurable via<br />
policies.<br />
When designing mechanisms various economic design criteria [4], [20] should<br />
be considered. These following criteria apply to the respective features as well<br />
as the overall system and the market mechanisms it is embedded in.<br />
Individual Rationality. An important requirement for a system is that it is<br />
individual rational on both sides, i.e. both providers and clients have to have a<br />
benefit from using the system.<br />
193
338 T. Püschel et al.<br />
Simplicity and Computational Costs. While the enhancements introduce some<br />
additional factors and they should not introduce any unnecessary complexity.<br />
Similarly client classification, quality of service and dynamic pricing add<br />
some additional computational complexity, however they should not add any<br />
intractable problems and its benefits should outweigh its costs.<br />
Revenue Maximization. A key characteristic for resource providers is revenue<br />
maximization or more general utility maximization.<br />
Incentive Compatibility. Strategic behaviour of clients and providers can be<br />
prevented if a mechanism is incentive compatible. Incentive compatibility means<br />
that no other strategy results in a higher utility than reporting the true valuation.<br />
Efficiency. There are different types of efficiency. The first one considered here<br />
is pareto optimimality. An allocation is considered pareto optimal if no participant<br />
can improve its utility without reducing the utility of another participant.<br />
The second efficiency criterion is allocative efficiency. A mechanism is called<br />
allocative efficient if it maximizes the sum of individual utilities.<br />
3.2 Key Features<br />
The motivational scenarios and the further requirements lead to four key features<br />
the of the EERM presented in this work.<br />
Quality of Service. The first feature is quality of service. This can be broken<br />
down into two aspects. The first aspect is to assure adequate performance during<br />
normal operation of the resources. Overload situations can lead to reduced overall<br />
performance [17] and thereby can result in breaking QoS agreements between<br />
the provider and clients. Thus it is necessary to have a mechanism that ensures<br />
that jobs will not be accepted if they result in an overload situation. The second<br />
aspect of quality of service regards situations in which parts of the resources<br />
fail. To be able to fulfill all SLAs even in situations of partial resource failure it<br />
would be necessary to keep an adequate buffer of free resources. Where this is<br />
not feasible there should at least be a mechanism that ensures that those SLAs<br />
that can be kept with the available resources are fulfilled. This can be done by<br />
suspending or cancelling those jobs that can not be finished in time due to the<br />
reduced availability of resources.<br />
Job Cancellation. Related to QoS is the feature automatic job suspension and<br />
cancellation. It is needed to ensure quality of service in situations where problems<br />
arise, i.e. parts of the resources fail or the estimations of the utilization were to<br />
optimistic. Cancellation of lesser important jobs to free capacity for incoming<br />
jobs with higher importance, i.e. jobs from a client with a higher classification<br />
or a jobs that deliver significantly more revenue is also possible.<br />
Dynamic Pricing. Another enhancement is dynamic pricing based on various<br />
factors. [21] shows an approach for a pricing function depending on a base pricing<br />
rate and a utilization pricing rate. However the price can depend not only on<br />
current utilization but also on other factors such as projected utilization, client<br />
194
Economically Enhanced Resource Management for Internet Service Utilities 339<br />
classification, and projected demand. Pricing should also be contingent on the<br />
demand on the market. This feature can be either implemented in the EERM<br />
or in a dedicated component responsible for trading. This agent would request<br />
a price based on factors including utilization of the resources and client classification.<br />
from the EERM and then calculate a new price taking the situation on<br />
the market into account.<br />
Client Classification. Earlier giving clients different privileges was mentioned,<br />
e.g. by discriminating on factors like price and quality of service. This part<br />
describes the factors that can be used to differentiate various client classes.<br />
Price Discrimination. Price discrimination or customer-dependent pricing is one<br />
way to differentiate between different classes of clients. One idea to achieve this<br />
is introducing Grid miles [16] in analogy to frequent flyer miles. Clients could be<br />
offered a certain amount of free usage of the resources or a 10% discount after<br />
spending a certain amount of money.<br />
Reservation of Resources. For certain users it may be very important to always<br />
have access to the resources. This class of users could be offered a reservation of<br />
a certain amount of resources. One option is to reserve a fixed share of resources<br />
for a certain class of users another possibility is to vary this share depending on<br />
the usage of the system.<br />
Priority on Job Acceptance. Another option is to use a client priority on job<br />
acceptance. When the utilization of the system is low jobs from all classes of<br />
clients are accepted but when the utilization of the resources rises and there is<br />
competition between the clients for the resources, jobs from certain clients are<br />
preferred. There are two types of priorities: strict priorities and soft priorities.<br />
– Strict priority means that if a job from a standard client and a client with<br />
priority compete for acceptance, the job from the client with priority always<br />
wins. Jobs from clients with priority are always preferred, thus there is no<br />
real competition between the different classes of clients.<br />
– Soft priority means jobs from clients with priority are generally preferred but<br />
standard clients have the chance to outbid clients with priority. Thus soft<br />
priority is essentially a discount on the reservation price or bid that may only<br />
apply in certain situation, i.e. when utilization exceeds a certain threshold.<br />
Quality of Service. Another factor where differentiation for classes of clients<br />
is possible is quality of service. For some classes of clients quality of service is<br />
offered, for others not. Offering different levels of quality of service for different<br />
classes of clients is also possible. An example for this would be offering different<br />
risk levels [7].<br />
4 Architecture of the EERM<br />
This part describes the architecture of the EERM that was designed based on<br />
the requirements and key features. It includes a description of the components<br />
195
340 T. Püschel et al.<br />
Fig. 1. EERM Architecture<br />
with the aid of sequence diagrams. An overview of the architecture of the EERM<br />
can be seen in Fig. 1.<br />
The EERM interacts with various other components, namely a Grid Market<br />
Middleware, a Monitoring component and the Resource Fabrics. The Grid<br />
Market Middleware represents the middleware responsible for querying prices<br />
and offering the services on the Grid market. The Monitoring is responsible for<br />
monitoring the state and the performance of the resources and notifying the System<br />
Performance Guard in case of problems. Additionally data collected by the<br />
Monitoring is used by the Estimator component to for its predictions. Resource<br />
Fabrics refers to Grid Middlewares such as Condor [15] or Globus [9].<br />
Economy Agent. The Economy Agent is responsible for deciding whether<br />
incoming jobs are accepted and for calculating their prices. These calculations<br />
can be used both for negotiation as well as bids or reservation prices in auctions.<br />
Figure 2 shows the sequence of a price request. First the Economy Agent receives<br />
a request from a market agent. Then it checks whether the job is technically and<br />
Fig. 2. Sequence Diagram Price Request<br />
196
Economically Enhanced Resource Management for Internet Service Utilities 341<br />
economically feasible and calculates a price for the job based on client category,<br />
resource status, economic policies and predictions of future job executions from<br />
the estimator component.<br />
Estimator. The Estimator component calculates the expected impact on the<br />
utilization of the resources. This is important to prevent reduced performance<br />
due to overload [17], furthermore the performance impact can be used for the<br />
calculation of prices. This component is based on work by Kounev et al. using<br />
online performance models [13],[14].<br />
System Performance Guard. The System Performance Guard is responsible<br />
for ensuring that the accepted SLAs can be kept. In case of performance problems<br />
with the resources it is notified by Monitoring. After checking the corresponding<br />
policies it determines if there is a danger that SLAs cannot be fulfilled. It<br />
then takes the decision to suspend or cancel jobs to ensure the fulfilment of the<br />
other SLAs and maximize overall revenue. Figure 3 shows a typical sequence for<br />
such a scenario. Jobs can also be cancelled when capacity is required to fulfil<br />
commitments to preferred clients.<br />
Policy Manager. To keep the EERM adaptable the Policy manager stores and<br />
manages policies concerning client classification, job cancellation or suspension,<br />
etc. Policies are formulated in the Semantic Web Rule Language (SWRL) [11].<br />
All features of the EERM require the respective components to be able to communicate<br />
with the Policy Manager and base their decisions on the corresponding<br />
policies. A simple example from a pricing policy in SWRL is the following rule<br />
which expresses that if the utilization is between 71% and 100% there is a surcharge<br />
of 50:<br />
Utilization(?utilization)∧InsideUtilizationRange(?utilization, ”71%−100%”)<br />
⇒ SetSurcharge(?utilizationsurcharge, ”50”)<br />
Fig. 3. Sequence Diagram System Performance Guard<br />
197
342 T. Püschel et al.<br />
Economic Resource Management. The Economic Resource Management is<br />
responsible for the communication with the local resource managers and influences<br />
the local resource <strong>management</strong> to achieve a more efficient global resource use.<br />
5 Evaluation<br />
The evaluation of this proposal consists of three parts. First an example scenario<br />
is described, then the results are presented and last the economic design criteria<br />
[4],[20] are considered.<br />
5.1 Example Scenario<br />
The first part of the evaluation considers an example scenario. For this evaluation<br />
there were no actual jobs executed. The policies were manually applied and<br />
revenue, utilization and utilities calculated. The job information given in Table 1<br />
and the following assumptions were used.<br />
The total capacity is 100 and the jobs given in Table 1 will be available during<br />
the run. The system receives information about jobs one timeslot before they<br />
become available. The Gold-client only uses the provider when he is assured<br />
access to a certain capacity. In this case he is guaranteed a total capacity of 60<br />
units, while being able to launch new jobs of up to 30 units per period.<br />
Using this scenario as a basis four policies are evaluated. The following policies<br />
represent the some of the basic options - no enhancements, client classification<br />
with fixed reservation, dynamic pricing with reservation prices, and client classification<br />
with strict priority:<br />
– Case I is an example without any EERM. In this case any job is accepted if<br />
there is enough capacity left to fulfil it.<br />
– Case II features a simple form of client classification, there is a fixed reservation<br />
of 60% of the capacity for the Gold-client. There is no System Performance<br />
Guard.<br />
– Case III features the EERM with utilization-based pricing and the System<br />
Performance Guard, but without client classification. The policy is to only<br />
accept jobs that offer a price higher than 1 currency unit per capacity unit<br />
and time slot if the job results in utilization over 80%. The unit price is<br />
determined by dividing the price by the total capacity used by a job over all<br />
periods.<br />
– In case IV the EERM with client classification and the system performance<br />
guard is used. The policy is to accept only jobs from the Gold-client if the<br />
job would result in utilization higher than 70%. This policy which can be<br />
applied to the first motivational scenario when replacing ”Gold-client” with<br />
”Internal”.<br />
To evaluate the ability of the EERM to adapt to problems with the resources<br />
there is also a second scenario. It includes a reduced capacity of 70 in t=7 and a<br />
capacity of 60 in t=8 due to unpredicted failure of resources. In the cases without<br />
198
Economically Enhanced Resource Management for Internet Service Utilities 343<br />
Table 1. Example scenario<br />
Job Start Time End Time Capacity/t Client Class Price Penalty<br />
A 1 3 55 Standard 330 -82.5<br />
B 1 5 24 Standard 180 -30<br />
C 1 7 20 Standard 140 -17.5<br />
D 2 4 20 Standard 120 -30<br />
E 3 5 15 Standard 90 -22.5<br />
F 3 8 20 Standard 120 -15<br />
G 4 7 20 Standard 160 -40<br />
H 4 9 15 Standard 135 -22.5<br />
I 5 10 30 Standard 180 -22.5<br />
K 5 8 30 Standard 240 -60<br />
L 6 8 30 Standard 90 -11.25<br />
M 6 9 12.5 Standard 50 -6.25<br />
O 8 10 20 Standard 90 -15<br />
P 9 10 21 Standard 84 -21<br />
Q 9 10 30 Standard 90 -15<br />
R 2 6 30 Gold 375 -187.5<br />
S 5 8 30 Gold 300 -150<br />
T 7 10 7.5 Gold 75 -37.5<br />
U 7 9 20 Gold 150 -75<br />
V 9 10 20 Gold 100 -50<br />
the System Performance Guard it is assumed that the SLAs of jobs that end in<br />
periods that are overloaded due to the reduced capacity fail and that jobs that<br />
continue to run for more time can catch up and fulfil their SLAs. To assess the<br />
benefits of the EERM for providers and clients we compare their utility without<br />
and with EERM. Utility functions depend on the preferences of providers and<br />
clients. For the provider revenue is obviously a key criteria. Another factor that<br />
has to be considered is utilization. To allow for maintenance and reduce the<br />
effects of partial resource failure it is not in the interest of the provider to have a<br />
utilization that approaches 100%. In this example an average utilization of 80% is<br />
considered optimal. The following utility function for the provider considers the<br />
revenue per utilization while taking into account the optimal average utilization<br />
of the resources:<br />
u provider =<br />
revenue<br />
n ∗ averageutilization ∗ 1<br />
1+|0.8 − averageutilization<br />
100<br />
|<br />
For the client utility the idea is to weight the capacity needed by a job with<br />
its importance. The importance of a job for a client is expressed in the price per<br />
capacity unit the client is willing to pay for it. This results in the following client<br />
utility function, where P is the set of prices per unit (i.e. 2.5, 2, 1.5, 1):<br />
u client = ∑ l∈P<br />
unitprice l ∗ allocatedjobcapacity l<br />
totaljobcapacity l<br />
199
344 T. Püschel et al.<br />
Table 2. Result of the four cases<br />
Case Completed<br />
Revenue Avg Load u provider u client Revenue Proportion<br />
I A, B, C, G, H, L, M, O , P, Q 1349 89.7 1.37 3.10 76.65%<br />
II C, D, M, O, R, S, T, U, V 1400 71 1.81 3.33 79.55%<br />
III A, B, D, G, H, K, M, O, P, Q 1479 84.7 1.67 3.25 84.03%<br />
IV A, G, H, R, S, T, U, V 1625 73.5 2.08 3.87 92.33%<br />
A* A, E, G, O, Q, R, S, T, U, V 1760 81 2.15 3.53 100%<br />
Table 3. Result with partial resource failure<br />
Case Completed<br />
Failed Revenue Avg Load u provider u client Revenue Proportion<br />
I A, B, H, L, M, P, Q C, G 901.5 79.7 1.13 2.28 56.56%<br />
II D, P, R, M, T, U, V C, S 786.5 66.2 1.04 2.23 49.34%<br />
III A, B, D, G, H, K, P, Q M 1332.75 74.95 1.69 3.05 83.61%<br />
IV A, P, R, S, T, U, V G, H 1351.5 71.2 1.74 3.31 84.79%<br />
A* A, E, P, Q, R, S, T, U, V 1594 71.2 2.06 3.11 100%<br />
5.2 Results<br />
Table 2 shows the results of applying these 4 cases to the scenario given in Table<br />
1. A* is the allocation with maximum revenue that could be achieved if all job<br />
information was available before the first period. The table includes information<br />
about the completed jobs, the realized revenue, the average utilization of the<br />
providers resources, the provider utility, the client utility, and the proportion of<br />
the revenue in comparison with the A*. Cases II, III and IV perform significantly<br />
better than the standard case I without any economical enhancements. Although<br />
the average utilization is lower than in case I the yielded revenue as well the<br />
provider’s and the client’s utility is higher.<br />
Table 3 shows the results of applying the policies to the scenario with failure<br />
of parts of the resources in t=7 and t=8. It can be seen that the improvement in<br />
cases III and IV is even more significant. In this situation case II is worse than case<br />
I regarding revenue, provider utility and client utility. This is caused by the lack of<br />
the System Performance Guard and thereby higher cancellation penalties of the<br />
jobs of the Gold-client. Case III delivers a significantly better utility for providers<br />
and clients. However since it doesn’t include client classification it does not address<br />
the needs of the Gold-client. Case IV again delivers the best results regarding revenue,<br />
provider utility and client utility. Figure 4 shows the revenue proportion in<br />
the standard case (a) and with partial resource failure (b). In both cases III and<br />
IV the benefit of the System Performance Guard can be clearly seen, they deliver<br />
of 83.61% and 84.79% compared to the 56.56% of the standard case I.<br />
5.3 Economic Design Criteria<br />
In cases III and IV of the example both sides have a clear benefit from using<br />
the respective mechanisms, hence individual rationality is achieved in these<br />
200
Economically Enhanced Resource Management for Internet Service Utilities 345<br />
(a)<br />
(b)<br />
Fig. 4. Proportion of maximum revenue in the four cases<br />
cases. The proposed mechanisms introduce some additional complexity but this<br />
is encapsulated in the EERM. They, however, do not introduce any NP-hard<br />
problems into the mechanism and the additional computational cost is limited.<br />
Another key characteristic for resource providers is revenue maximization. Inthe<br />
given example the policy of case IV delivers the highest revenue, provider utility<br />
and client utility among the options.<br />
Whether the criterion of incentive compatibility is fulfilled depends on the<br />
market and classification mechanisms the EERM is embedded in. For the example<br />
a simple pay as you bid scheme is assumed. As this means that clients set<br />
their bids lower than their evaluation to achieve a benefit incentive compatibility<br />
is not given in the example. If, however, embedded in an allocation and pricing<br />
mechanism that is incentive compatible the EERM serves to give more precise<br />
valuations for the jobs.<br />
Efficiency also depends on the mechanisms the EERM is embedded in. For<br />
the example we used a scenario where the provider decides whether jobs are<br />
accepted. Obviously the provider is mainly concerned with maximizing its own<br />
utility. Therefore it does not guarantee the maximum total utility. The EERM<br />
can also be embedded in efficient market mechanisms serving to give a more<br />
precise valuations of the jobs.<br />
6 Related work<br />
There is related work covering different aspects of our work. [8] discusses the<br />
application of economic theories to resource <strong>management</strong>. [1] presents an architecture<br />
for autonomic self-optimization based on business objectives. Elements<br />
of client classification such as price discrimination based on customer characteristics<br />
have been mentioned in other papers [16], [3]. They did however not<br />
consider other discrimination factors. [6] describes data-mining algorithms and<br />
tools for client classification in the electricity grids but concentrate on methods<br />
for finding groups of customers with similar behaviour. An architecture for admission<br />
control on e-commerce websites that prioritizes user sessions based on<br />
predictions about the user’s intentions to buy a product is proposed in [18]. [2]<br />
201
346 T. Püschel et al.<br />
presents research on how workload class importance should be considered for<br />
low-level resource allocation.<br />
One approach to realize end-to-end quality of service is the Globus Architecture<br />
for Reservation and Allocation (GARA) [10]. This approach uses advance<br />
reservations to achieve QoS. Another way to achieve autonomic QoS aware resource<br />
<strong>management</strong> is based on online performance models [13], [14]. They introduce<br />
a framework for designing resource managers that are able to predict the<br />
impact of a job in the performance and adapt the resource allocation in such a<br />
way that SLAs can be fulfilled. Both approaches do not consider achieving QoS<br />
in case of partial resource failure. Our work makes use of the second approach<br />
and adds the mechanism of Job Cancellation.<br />
The introduction of risk <strong>management</strong> to the Grid [7] permits a more dynamic<br />
approach to the usage of SLAs. It allows modelling the risk that the SLA cannot<br />
be fulfilled within the service level agreement. A provider can then offer SLAs<br />
with different risk profiles. However such risk modelling can be very complex. It<br />
requires information about the causes of the failure and its respective probabilities.<br />
Clients need to have the possibility to validate the accuracy and correctness<br />
of the providers risk assessment and risks have to be modelled in the SLAs.<br />
7 Conclusion and Future Work<br />
In this work various economical enhancements for resource <strong>management</strong> were<br />
motivated and explained. We presented a mechanism for assuring Quality of<br />
Service and dealing with partial resource failure without introducing the complexity<br />
of risk modelling. Flexible Dynamic Pricing and Client Classification was<br />
introduced and it was shown how these mechanisms can benefit service providers.<br />
Various factors and technical parameters for these enhancements were presented<br />
and explained.<br />
Furthermore the preliminary architecture for an Economically Enhanced Resource<br />
Manager integrating these enhancements was introduced. Due to the general<br />
architecture and the use of policies and a policy manager this approach can<br />
be adapted to a wide range of situations.<br />
The approach was evaluated considering economic design criteria and using<br />
an example scenario. The evaluation shows that the proposed economic enhancements<br />
enable the provider to increase his benefit. In the standard scenario we<br />
managed to achieve a 92% of the maximum theoretically attainable revenue<br />
with the enhancements in contrast to 77% without enhancements. In the scenario<br />
with partial resource failure the revenue was increased from 57% to 85%<br />
of the theoretical maximum.<br />
The next steps will include refinement of the architecture as well as the implementation<br />
of the EERM. During this process further evaluation of the system<br />
will be done, e.g. by testing the system and running simulations. Another issue<br />
that requires further consideration is the generation of business policies for the<br />
EERM. Further future work in this area includes the improvement of price calculations<br />
with historical data and demand forecasting; the design of mechanisms<br />
202
Economically Enhanced Resource Management for Internet Service Utilities 347<br />
for using collected data to determine which offered services deliver the most revenue<br />
and concentrate on them; the introduction of a component for automatic<br />
evaluation and improvement of client classification.<br />
Acknowledgments<br />
This work is supported by the Ministry of Science and Technology of Spain<br />
and the European Union (FEDER f<strong>und</strong>s) <strong>und</strong>er contract TIN2004-07739-C02-<br />
01 and Commission of the European Communities <strong>und</strong>er IST contract 034286<br />
(SORMA).<br />
References<br />
1. Aiber, S., Gilat, D., Landau, A., Razinkov, N., Sela, A., Wasserkrug, S.: Autonomic<br />
self-optimization according to business objectives. In: ICAC 2004. Proceedings of<br />
the First International Conference on Autonomic Computing, pp. 206–213 (2004)<br />
2. Boughton, H., Martin, P., Powley, W., Horman, R.: Workload class importance<br />
policy in autonomic database <strong>management</strong> systems. In: POLICY 2006. Proceedings<br />
of the Seventh IEEE International Workshop on Policies for Distributed Systems<br />
and Networks, pp. 13–22 (2006)<br />
3. Buyya, R.: Economic-based Distributed Resource Management and Scheduling for<br />
Grid Computing. PhD thesis, Monash University (2002)<br />
4. Campbell, D.E.: Resource Allocation Mechanisms. Cambridge University Press,<br />
London (1987)<br />
5. Carr, N.G.: The end of corporate computing. MIT Sloan Management Review<br />
46(3), 32–42 (2005)<br />
6. Chicco, G., Napoli, R., Piglione, F.: Comparisons among clustering techniques<br />
for electricity customer classification. IEEE Transactions on Power Systems 21(2),<br />
933–940 (2006)<br />
7. Djemame, K., Gourlay, I., Padgett, J., Birkenheuer, G., Hovestadt, M., Kao, O.,<br />
Voß, K.: Introducing risk <strong>management</strong> into the grid. In: eScience2006. The 2nd<br />
IEEE International Conference on e-Science and Grid Computing, Amsterdam,<br />
Netherlands, p. 28 (2006)<br />
8. Ferguson, D.F., Nikolaou, C., Sairamesh, J., Yemini, Y.: Economic models for<br />
allocating resources in computer systems, 156–183 (1996)<br />
9. Foster, I., Kesselman, C.: Globus: A metacomputing infrastructure toolkit. International<br />
Journal of Supercomputer Applications and High Performance Computing<br />
11(2), 115–128 (1997)<br />
10. Foster, I., Kesselman, C., Lee, C., Lindell, B., Nahrstedt, K., Roy, A.: A distributed<br />
resource <strong>management</strong> architecture that supports advance reservations and<br />
co-allocation. In: IWQoS 1999. Proceedings of the 7th International Workshop on<br />
Quality of Service, London, UK, pp. 62–80 (1999)<br />
11. Horrocks, I., Patel-Schneider, P.F., Boley, H., Tabet, S., Grosof, B., Dean, M.: Swrl:<br />
A semantic web rule language combining owl and ruleml. Technical report, W3C<br />
Member submission (2004)<br />
12. Kenyon, C., Cheliotis, G.: Grid resource commercialization: economic engineering<br />
and delivery scenarios. Grid resource <strong>management</strong>: state of the art and future<br />
trends, 465–478 (2004)<br />
203
348 T. Püschel et al.<br />
13. Kounev, S., Nou, R., Torres, J.: Autonomic qos-aware resource <strong>management</strong> in<br />
grid computing using online performance models. In: VALUETOOLS 2007. The<br />
2nd International Conference on Performance Evaluation Methodologies and Tools,<br />
Nantes, France (2007)<br />
14. Kounev, S., Nou, R., Torres, J.: Building online performance models of grid middleware<br />
with fine-grained load-balancing: A globus toolkit case study. In: EPEW<br />
2007. The 4th European Engineering Performance Workshop, Berlin, Germany<br />
(2007)<br />
15. Litzkow, M.J., Livny, M., Mutka, M.W.: Condor - A hunter of idle workstations. In:<br />
Proceedings of the 8th International Conference of Distributed Computing Systems<br />
(1988)<br />
16. Newhouse, S., MacLaren, J., Keahey, K.: Trading grid services within the uk e-<br />
science grid. Grid resource <strong>management</strong>: state of the art and future trends, 479–490<br />
(2004)<br />
17. Nou, R., Julià, F., Torres, J.: Should the grid middleware look to self-managing<br />
capabilities? In: ISADS 2007. The 8th International Symposium on Autonomous<br />
Decentralized Systems, Sedona, Arizona, USA, pp. 113–122 (2007)<br />
18. Poggi, N., Moreno, T., Berral, J.L., Gavaldà, R., Torres, J.: Web customer modeling<br />
for automated session prioritization on high traffic sites. In: Proceedings of the 11th<br />
International Conference on User Modeling, Corfu, Greece (2007)<br />
19. Rappa, M.A.: The utility business model and the future of computing services.<br />
IBM Systems Journal 43(1), 32–42 (2004)<br />
20. Wurman, P.R.: Market structure and multidimensional auction design for computational<br />
economies. PhD thesis, University of Michigan, Chair-Michael P. Wellman<br />
(1999)<br />
21. Yeo, C.S., Buyya, R.: Pricing for Utility-driven Resource Management and Allocation<br />
in Clusters. In: ADCOM 2004. Proceedings of the 12th International Conference<br />
on Advanced Computing and Communications, Ahmedabad, India, pp. 32–41<br />
(2004)<br />
204
SITUATED DECISION SUPPORT FOR MANAGING SERVICE LEVEL<br />
AGREEMENT NEGOTIATIONS<br />
Rustam Vahidov 1 , Dirk Neumann 2<br />
1 John Molson School of Business<br />
Concordia University, Montreal, Canada<br />
rVahidov@jmsb.concordia.ca<br />
2<br />
<strong>Institut</strong>e of Information Systems and<br />
Management (<strong>IISM</strong>)<br />
Universität Karlsruhe (TH)<br />
neumann@iism.uni-karlsruhe.de<br />
Abstract<br />
In this paper we propose a situated decision<br />
support system for efficient negotiating of service<br />
level agreements (SLAs) in Grids. Situated decision<br />
support systems effectively combine human<br />
judgment with autonomous decision making and<br />
action by agents. The intuition of using this approach<br />
for SLA negotiations lies in the monitoring<br />
and controlling of the fleet of local agents<br />
negotiating single services from multiple service<br />
providers by the use of a “manager” agent and<br />
human decision maker. We show through numerical<br />
experiments that the approach performs<br />
well <strong>und</strong>er a set of simplifying assumptions.<br />
1. Introduction<br />
Quite recently, Grid computing has been increasingly<br />
gaining traction in a number of application<br />
areas. Grid computing denotes a computing<br />
model that distributes processing across an administratively<br />
and locally dispersed infrastructure.<br />
The ability to connect numerous heterogeneous<br />
computing devices, allows the creation of<br />
virtual computer architectures from otherwise<br />
idle resources.<br />
According to market surveys is the average productivity<br />
of IT infrastructures extremely low:<br />
mainframes are up to 40 % of the time idle;<br />
UNIX servers even less than 10 % of the time<br />
used; individual PCs reach a utilization rate of at<br />
most 5 % [2]. This daunting ineffectiveness of IT<br />
infrastructures and associated costs entail that<br />
enterprises seek to use Grid technologies to outsource<br />
IT services. Rather than investing and<br />
maintaining proprietary infrastructures enterprises<br />
tap to the Grid and consume IT services<br />
on-demand. Service providers like SUN and<br />
Amazon accommodate this need by offering<br />
CPU hours and storage over web-services.<br />
Outsourcing of IT services is, however, not a<br />
panacea as proposed by critics Nicholas Carr [2].<br />
Any outsourced IT service incurs high risks, as<br />
the deployment is outside the control of the enterprises.<br />
Service providers address this risk by<br />
referring to formal contracts, so-called Service<br />
Level Agreements (SLAs). SLAs define the<br />
common <strong>und</strong>erstanding about services (e.g.,<br />
priorities, responsibilities, guarantees and penalties<br />
once SLAs are violated) between service<br />
providers and consumers. Essentially, SLAs are<br />
the crucial instrument for service consumers to<br />
formulate guarantees on the service quality delivered<br />
by the service provider. Nonetheless,<br />
SLAs are also an important tool for service providers<br />
to advertise free service capacities as well<br />
as to manage their internal resources.<br />
It is thus not asto<strong>und</strong>ing that SLA lifecycle <strong>management</strong><br />
is currently a hot topic [6, 10]. One<br />
major success has been the specification of the<br />
de-facto standard WS-Agreement as a description<br />
language for SLAs by the Open Grid Forum<br />
(OGF). WS-Agreement allows the formulation<br />
of well defined and comprehensive contracts,<br />
which minimizes the risks of disputes resulting<br />
from unclear formulations among service provider<br />
and consumer. Clearly, a description language<br />
is merely prerequisite for well defined<br />
SLAs. What is equally important is a procedure<br />
with which SLAs are created. This becomes even<br />
more important, if SLAs are dynamically<br />
adapted dependent on resource availability and<br />
demand. If the SLAs are well-negotiated, the<br />
likelihood that balanced SLAs are signed is<br />
much higher. In this spirit, the Grid Resource<br />
Allocation Agreement Protocol Working Group<br />
(GRAAP-WG)<br />
(http://forge.gridforum.org/projects/graap-wg) of<br />
OGF set on their agenda for future work to define<br />
so-called negotiation profiles or protocols<br />
for SLA negotiation as part of the WS-<br />
Agreement specification [22].<br />
SLA negotiations can become very complex as<br />
service consumers need to negotiate with multiple<br />
service providers to reserve different resources.<br />
The resource reservations that are<br />
205
needed are typically described via a workflow.<br />
Service consumers either want to have all resources<br />
reserved along the workflow or none at<br />
all. This is reasoned by the fact that failure to<br />
reserve one single constituent of the workflow<br />
results in unsuccessful termination of the service<br />
[15]. Due to this complexity the use of software<br />
agents have been proposed for managing SLA<br />
negotiations [4, 7, 19].<br />
In this work our interest is in one-to many SLA<br />
negotiations involving one service provider and<br />
multiple potential customers. In such settings the<br />
complexity involved in tracking and <strong>management</strong><br />
of multiple on-going negotiations can be<br />
alleviated by means of intelligent support (e.g.<br />
software agents). However, in our opinion, a<br />
balanced approach combining autonomous action<br />
with overall human control and decisionmaking<br />
is a more adequate approach, as it allows<br />
for timely intervention and improves the predictability<br />
of business outcomes. In this respect the<br />
recently introduced model of “situated decision<br />
support” seems to be a promising framework to<br />
apply to SLA negotiations [26, 28]. Situated<br />
decision support advocates abandoning the traditional<br />
“toolbox” type of decision aids in favor of<br />
a more connected and active mode. The major<br />
components of situated decision support system<br />
include sensors, effectors, manager, and active<br />
user interface. In this setup, decision support<br />
expands to include problem sensing and action<br />
generation and monitoring of the implementation<br />
of decisions in addition to the conventional intelligence/design/choice<br />
phases. It also allows<br />
flexibility for effectively combining autonomous<br />
action by the system with judgmental input from<br />
the human decision makers.<br />
Since the fields of decision and negotiation support<br />
are closely related, in this work we show the<br />
potential applicability of the situated decision<br />
support model to managing multiple concurrent<br />
SLA negotiations, where a managing entity<br />
monitors and controls the local negotiation<br />
agents. The contribution of this paper is threefold:<br />
• Firstly, we provide motivation for the adoption<br />
of situated decision support model for<br />
SLA negotiations;<br />
• Secondly, we briefly describe a situated decision<br />
support framework for SLA negotiations.<br />
• Thirdly, we illustrate the approach through<br />
numerical simulation experiments for the<br />
scenario involving two issues.<br />
The remainder of this paper is structured as follows.<br />
Section 2 provides the backgro<strong>und</strong> on<br />
SLAs, SLA negotiation and on agent strategies<br />
in automated negotiations. Section 3 motivates<br />
and describes the situated decision support<br />
framework and shows how it can be used for<br />
SLA negotiation. Section 4 provides an evaluation<br />
on the basis of numerical experiments showing<br />
that situated decision support is advantageous<br />
for SLA negotiations. Section concludes<br />
with a summary and an outlook on future research.<br />
2. Backgro<strong>und</strong><br />
This section provides brief overview of service<br />
level agreements (definition, content of typical<br />
SLAs), SLA negotiations (common protocols)<br />
and agent-based strategies. It further discusses<br />
why situated DSS model is deemed promising<br />
for use in SLA negotiations.<br />
Service Level Agreements<br />
A SLA is defined a contract between a service<br />
provider and consumer that specifies the rights<br />
and obligations of the provider and the penalties<br />
that will be applied if those obligations are not<br />
satisfied.<br />
Since the late 80s SLAs have been used by telecom<br />
operators as contracts with their corporate<br />
customers. With the recent trend of outsourcing<br />
and application service providing, IT departments<br />
have adopted the idea of using service<br />
level agreements with their internal or external<br />
customers.<br />
SLAs for Grid computing are not very different<br />
than those for other services (e.g. computation,<br />
or storage services). But while the elementary<br />
issues of an SLA can be the same, negotiating<br />
SLAs for Grid services is aggravated by the fact<br />
that Grid services are typically composed of<br />
several basic services. This implies that it is<br />
essential needs to manage concurrent SLA negotiations<br />
with multiple service providers [24].<br />
Typical attributes of SLAs for Grid are compiled<br />
in Table 1.<br />
Attributes<br />
Operating System<br />
Attribute Levels<br />
Linux<br />
Solaris<br />
Windows<br />
Service Availability 99,999 %<br />
99,99 %<br />
99,9 %<br />
Price<br />
< 1,000 €/month<br />
< 5,000 €/month<br />
< 10,000 €/month<br />
> 10,000 €/month<br />
206
CPU type<br />
CPU utilization<br />
Storage (RAM)<br />
Bandwidth<br />
Deployment<br />
Co-allocation<br />
Single CPU<br />
Dual CPU<br />
4 CPU<br />
> 4CPU<br />
1,000 CPU hours<br />
5,000 CPU hours<br />
10,000 CPU hours<br />
1 GB<br />
2 GB<br />
4 GB<br />
> 4 GB<br />
1 Mb<br />
10 Mb<br />
100 Mb<br />
1 Tb<br />
Webservice<br />
Source code<br />
others<br />
Service is provided by a<br />
single provider<br />
Service is provided by a<br />
multiple provider<br />
Table 1: Common Attributes and attribute<br />
levels for Grid services<br />
Service Level Agreements Negotiation<br />
It is commonly accepted that the success of SLA<br />
negotiations relates to the specification and implementation<br />
of a protocol that creates legallybinding<br />
agreements. This protocol needs to describe<br />
a set of domain-independent messages,<br />
together with an abstract schema, such that it can<br />
be exchanged by all negotiators [15]. Several<br />
bargaining protocols that meet those requirements<br />
have been proposed by [4, 7, 9, 20, 25].<br />
The bargaining protocols share a very similar<br />
structure but differ in some details. For example<br />
the protocol proposed by [25] involves multiple<br />
ro<strong>und</strong>s among service consumers (e.g., coallocators)<br />
and providers (e.g., schedulers) until<br />
they reach an agreement. The providers post<br />
their offers on request to the consumers, who can<br />
either select among the available offers or enter<br />
re-negotiation by relaxing some constraints.<br />
Another noteworthy effort has been proposed by<br />
[17] who combine the negotiation protocol with<br />
a template-based contract generation process. At<br />
the end of the negotiation protocol a fullyfledged<br />
SLA is generated as WS-Agreement<br />
instance.<br />
Agent-based Negotiation<br />
SLA negotiations can involve a number of participants<br />
both on the consumer as well as the<br />
provider side trying to reach an agreement in<br />
multiple bi-lateral interactions. A given provider,<br />
for example, could have several on-going negotiations<br />
at the same time and has to set the objectives,<br />
constraints and strategies in those negotiation<br />
instances in such a way that maximizes the<br />
achievement of business objectives, while avoiding<br />
over-commitment, i.e. promising more than<br />
the provider can deliver. Clearly, with the growing<br />
complexity of the composite services and<br />
increasing customer base, automated means of<br />
supporting multiple negotiations could help the<br />
providers to better handle SLA negotiation processes.<br />
In this respect, employing intelligent<br />
agents seems to be an adequate approach. Below<br />
we briefly describe some agent-based approaches<br />
to automated negotiations, in particular for the<br />
case of multi-bilateral settings.<br />
Most of the related work in the area of agentbased<br />
negotiation has been devoted to the design<br />
of strategies for agents. The f<strong>und</strong>amental work<br />
has been <strong>und</strong>ertaken without considerations to<br />
SLA negotiations. Nonetheless, as those approaches<br />
are domain-independent, they are<br />
highly relevant for SLA negotiations.<br />
Faratin et al. have proposed a “smart” strategy<br />
for autonomous negotiating agents [8]. Agents<br />
following this strategy would try to make tradeoffs<br />
in a manner that the newly generated offer is<br />
similar to the opponent’s last offer, before trying<br />
a concession. Another work in this direction<br />
seeks to map business policies and contexts to<br />
negotiation goals, strategies, plans, and decisionaction<br />
rules [16].<br />
While fully automated negotiations may not<br />
always be a feasible choice, agents could also act<br />
as intelligent assistants, helping the users by<br />
providing advice, critiquing user’s own candidate<br />
offers as well as the offers by an opponent,<br />
and generating candidate offers for a user to<br />
consider [3, 14].<br />
In multi-bilateral negotiations a negotiator may<br />
be having several concurrent negotiation processes<br />
taking place at the same time and involving<br />
multiple opponents A fuzzy set-theoretic approach<br />
to analysis of alternatives in multibilateral<br />
negotiations has been proposed in [29].<br />
The authors have considered scenarios involving<br />
RFQ sent by one seller to multiple buyers<br />
(agents) with the purpose of deciding which<br />
potential buyers to negotiate with. They used<br />
fuzzy-relational approach to obtain a partial<br />
rank-order of the prospective buyers.<br />
A setup where multiple agents negotiate<br />
autonomously and one agent is designated as a<br />
coordinator has been proposed in [18] and [21].<br />
207
In [18] a buyer agent runs multiple concurrent<br />
negotiation threads that interact with several<br />
sellers. The coordinator agent provides each<br />
thread with reservation values and negotiation<br />
strategy. The threads report back to the coordinator,<br />
which then advises them on the possible<br />
changes to reservation values or strategies. In<br />
[21] a similar approach has been used including<br />
coordinating agents and multiple “sub-buyer”<br />
agents. In [5] a protocol for handling many-tomany<br />
concurrent negotiations for internet services<br />
has been proposed.<br />
In most of the above models of one-to-many<br />
negotiations it is assumed that the party on the<br />
one side can only have one agreement as an<br />
outcome of negotiations with multiple opponents<br />
(i.e. XOR case). In this work we interested in<br />
managing multiple SLA negotiations where each<br />
negotiation may fail or succeed regardless of<br />
others (OR case). In such settings the system<br />
could learn from the agreements made recently<br />
and take into account other relevant information<br />
(e.g. market situation) to direct concurrent negotiations.<br />
3. Situated DSS for SLA Negotiation<br />
When negotiation mechanism is used as one of<br />
the means to conduct daily basic business activities,<br />
much human effort may be required. Employing<br />
automated software components to conduct<br />
or assist human negotiators in conducting<br />
regular SLA negotiations may be a promising<br />
solution to achieve significant cost and time<br />
savings. However, with automated negotiations<br />
there is a potential danger that the overall process<br />
and the important business outcomes may<br />
become somewhat unpredictable. Moreover,<br />
often the course of negotiations depends on other<br />
factors, lying outside the domain of the expertise<br />
or sensory capabilities of the agents. For example,<br />
customer behavior may be heavily affected<br />
by the latest important economic, technologyrelated<br />
and other types of events. This calls for a<br />
type of solution allowing certain degree of automation,<br />
but allowing the control by the user of<br />
the overall process.<br />
In light of the above considerations we propose a<br />
type of solution that would effectively combine<br />
human judgment capabilities with autonomous<br />
actions by agents. The key motivation here is to<br />
relieve human users from the necessity of being<br />
involved in each and every SLA negotiation<br />
process. Rather, the system should allow the<br />
human decision maker to effectively and efficiently<br />
manage the fleet of negotiating agents to<br />
meet and maintain higher-level business targets.<br />
The field of decision support systems (DSS) is<br />
very much related to the area of negotiations and<br />
negotiation support systems [13]. In the recent<br />
DSS literature there has been important research<br />
streams directed towards building “active” and<br />
agent-based systems that would transform the<br />
original “toolbox” model of DSS into an active<br />
participant in the decision-making process,<br />
which could perform some decision-related tasks<br />
in an autonomous fashion [1, 11, 23, 27].<br />
One such model introduced recently is known as<br />
“situated decision support system”, or “decision<br />
station” [26, 28]. Situated DSS looks to combine<br />
the benefits of agent technologies and those of<br />
decision support systems to facilitate active<br />
problem sensing, decision implementation and<br />
monitoring in addition to pure decision support.<br />
In essence, situated DSS expands the traditional<br />
model from purely “problem-solving/decision<br />
making” frame to situation assessment and action<br />
generation.<br />
Situated DSS is made up from different active<br />
(agent) components in addition to the traditional<br />
“toolbox” of data, models, and knowledge. The<br />
components include: sensors (for information<br />
search and retrieval), effectors (for affecting<br />
current state of affairs), manager (for deciding<br />
how to handle a particular situation), and active<br />
user interfaces (for intelligent interaction with<br />
the user).<br />
The composition and interactions of different<br />
components of the situated DSS model provides<br />
an appealing blueprint for facilitating one-tomany<br />
SLA negotiations with limited autonomous<br />
action and overall control of the process by a<br />
human decision maker. Various negotiationrelated<br />
tasks can be mapped to specific agents in<br />
the model. For example, tasks including monitoring<br />
service levels, sensing potential problems,<br />
generating alerts, and predicting market trends<br />
for SLA could be delegated to the “sensor”<br />
agents. Controlling the fleet of negotiating<br />
agents by monitoring their progress and instructing<br />
them on the adjustments to negotiation<br />
strategies, reservation levels and preference<br />
structures can be delegated to some extent to the<br />
“manager” agent. Human decision maker could<br />
set the limits of authority of this agent and intervene<br />
if necessary. Each negotiation instance<br />
could be delegated to a particular “effector”<br />
agent that conducts a given negotiation and reports<br />
on the progress.<br />
The model for supporting multiple SLA negotiations<br />
is shown in figure 1.The situated DSS<br />
208
model is essentially hierarchical involving three<br />
layers. At the bottom layer, which could be<br />
called “operational” the agents perform negotiations.<br />
They are given the preferences, aspiration<br />
and reservation layers and strategies to follow<br />
and try to negotiate effectively with a given<br />
opponent. The basic cycle of generating an offer<br />
by these agents could be described as: retrieve<br />
preference structure (possibly updated by the<br />
“manager” agent); compare past offer and<br />
counter-offer; generate new offer according to<br />
adopted strategy. Automated negotiations have<br />
been studied extensively in the recent past and<br />
thus we do not focus on the detailed design of<br />
negotiating agents. One possibility is to employ<br />
“smart” strategy by the negotiating agents proposed<br />
in [8].<br />
The second layer contains the manager agent,<br />
which makes use of the traditional DSS components<br />
(data, models and knowledge) to manage<br />
the negotiating agents. The manager monitors<br />
key indicators of the negotiation processes, such<br />
as number of agreements reached, proportion of<br />
failed negotiations, resource consumption and<br />
others and decides whether to intervene or not.<br />
For example, if memory resource becomes<br />
scarce, then in the next ro<strong>und</strong> of SLA negotiations<br />
the manager would instruct the agents to<br />
stress more the importance of memory resource<br />
in the utility calculation. One way to encode the<br />
knowledge of the manager could be through “If-<br />
Then” (possibly fuzzy) rules. The following<br />
simple example from a manager policy in SWRL<br />
[12] is the following rule which expresses that if<br />
memory units sold is somewhat larger than<br />
memory units estimated, the importance of<br />
memory units is slightly increased:<br />
Function(?MemoryUnitsSold, ?MemoryUnitsEstimated,<br />
?MemoryImportance,<br />
?Increment) ←<br />
builtin:greaterThanOrEquals(?MemoryU<br />
nitsSold, ? MemoryUnitsEstimated),<br />
,builtin:add(?MemoryImportance,?Increm<br />
ent,? MemoryImportance)<br />
Such rule, when invoked would change the preference<br />
structure for the negotiator agents, which<br />
then would be willing to give up less memory<br />
units as a trade-off compared to other issues.<br />
This level could be termed “planning” layer.<br />
Figure 1: Situated DSS for managing multiple<br />
SLA negotiations<br />
The manager agent would base its decisions<br />
solely on the information available to it, i.e.<br />
reports received from the sensors and negotiators.<br />
However, in reality, there are many other<br />
sources and types of information that may not be<br />
accessible to the agent. Furthermore, often<br />
judgmental input would be required to set the<br />
limits of authority for the manager agent, set the<br />
objectives and constraints (e.g. to maintain the<br />
ratio of failed vs. successful negotiations at a<br />
certain level) and attune its parameters, e.g. its<br />
risk profile, or the thresholds for sensitivity for<br />
reacting to <strong>und</strong>esirable developments. This is<br />
accomplished by the user through active user<br />
interface. Active user interface facilitates effective<br />
interaction with the user, while learning user<br />
preferences. This layer could be called the<br />
“judgmental” layer. Components of situated DSS<br />
could send various alerts to the user when human<br />
intervention may be desirable.<br />
4. Numerical Experiments<br />
To illustrate the situated DSS approach with the<br />
example and obtain preliminary empirical results<br />
we have conducted simulation experiments described<br />
in this section.<br />
4.1 Data Generation<br />
We have used the scenario where service provider<br />
negotiates with multiple prospective customers<br />
the terms of the SLA. For the simplicity,<br />
we considered only two issues: price of service<br />
and number of CPU hours. Simplifying assumptions<br />
were made, e.g. the negotiation ended the<br />
same day as it began. We did not actually simulate<br />
every negotiation process, as automated<br />
209
negotiations have been studied in the past, and<br />
the focus of the work is on the overall performance<br />
of situated DSS that includes negotiating<br />
agents as components. We generated various<br />
values for reservation levels for the incoming<br />
customers and their preference structures. We<br />
furthermore assumed that if agreement between a<br />
customer and an agent was possible it would be a<br />
Nash solution. Thus, we assumed that the parties<br />
(consumers and negotiating agents) followed<br />
reasonable strategies to achieve mutually beneficial<br />
outcomes. The latter assumption has been<br />
adopted, since in [8] strategies were proposed for<br />
autonomous negotiations that lead to desirable<br />
outcomes for the parties involved.<br />
In simulated settings we had included 20 days of<br />
SLA negotiations with twenty potential service<br />
consumers generated each day. The consumer<br />
reservation levels for price were generated from<br />
normal distribution aro<strong>und</strong> the “market” price,<br />
which was set to $100 per 1000 CPU hours. The<br />
requested number of CPU hours was also distributed<br />
normally. The number of available CPU<br />
hours was set to two millions and this “capacity”<br />
did not change throughout the period. Each service<br />
consumer was assigned a separate negotiator<br />
and the reservation levels were provided by<br />
the “manager” (agent). In addition, we have<br />
incorporated the preferences for the number of<br />
CPU hours vs. unit price by introducing discount<br />
options, which were also controlled by the manager.<br />
In particular, if agents committed more<br />
(less) CPU hours than targeted for a given period,<br />
the manager would decrease (increase) the<br />
discount values for additional CPU hours. This<br />
was accomplished through “If-Then” type of<br />
rules, similar to that described in the previous<br />
section.<br />
On a single day a number of agreements were<br />
made ranging from 0 to 20. Then the outcomes<br />
were analyzed and corrections were made by the<br />
manager to price limits and discounts using simple<br />
rules, e.g.: “if the number of successful negotiations<br />
is smaller than expected by a given margin<br />
then adjust the reservation level for price<br />
downward”. The manager could make such adjustments<br />
only within the limits specified by the<br />
human user. In the extreme case no such adjustments<br />
could be allowed.<br />
Figure 2: Effect of price level adjustments on<br />
number of agreements reached<br />
4.2 Results<br />
Figure 2 shows the number of service level<br />
agreements reached throughout the 20-day period<br />
as it is affected by the manager’s adjustments.<br />
Vertical bars represent increases and<br />
decreases to price levels set by the manager<br />
agent and communicated to the negotiator<br />
agents. For example, in period 8 the manager<br />
drops the reservation levels for negotiating<br />
agents to increase the number of deals made. The<br />
number of agreements reduces to zero by the end<br />
of the period as the capacity limit for CPU hours<br />
is reached. Figure 3 shows the effect of adjustments<br />
to the discount levels on the number of<br />
CPU hours.<br />
Figure 3: Effect of adjustments to discount<br />
levels on the number of CPU hours<br />
As mentioned earlier, the freedom given to the<br />
manager in setting the levels is controlled by the<br />
human decision maker. In setting these levels the<br />
human decision maker should use his or her<br />
knowledge of the market situation and make a<br />
judgment. The settings could also translate into<br />
210
the various levels of risk the decision maker is<br />
willing to take. For example if these are set high<br />
and tight there is a risk that not all CPU hours<br />
would be sold by the end of the period. If the<br />
spread between the upper and lower levels is<br />
large, then manager would start with higher<br />
levels and then gradually decrease them to be<br />
aligned with the market, thus possibly losing<br />
some early opportunities.<br />
The above figures were based on a scenario<br />
when the market price remains stable. In case the<br />
market price is initially much lower than the<br />
price levels set by the manager and then gradually<br />
increasing, the manager would have to make<br />
several adjustments to its reservation level before<br />
it could get the needed number of agreements.<br />
This situation is shown in figure 4.<br />
Figure 4: Adjustments when market price<br />
increases from low to high<br />
Also, in case of a sudden change in market price<br />
the manager would be able to adjust, though it<br />
would take some time and result in the loss of<br />
some possibilities. In such cases making early<br />
interventions as a result of rightly exercised<br />
judgment by the human decision maker would be<br />
beneficial. Such a combination of human judgment<br />
with the automated capabilities could be<br />
beneficial to the successful operation of the business.<br />
Finally, figure 5 compares profits achieved by<br />
the fixed pricing policy (price was fixed equal to<br />
the market price) vs. negotiation based policies<br />
based on the profits averaged over 20 runs.<br />
Figure 5: Comparison of average profits<br />
achieved by fixed pricing vs. situated DSSdriven<br />
negotiation-based approaches<br />
The negotiation-based scenarios divide into two:<br />
the one where manager’s actions were restricted<br />
(SDSS1), and the one where there are little restrictions<br />
(SDSS2). The latter case promises<br />
highest profits, though also least control that<br />
might jeopardize higher-level policies. As one<br />
can see the more adaptive situated DSS-based<br />
solutions perform better than the fixed price<br />
based one.<br />
Table 2 shows the average profits made by using<br />
the fixed pricing and situated DSS approaches<br />
<strong>und</strong>er various price settings. The column headings<br />
reading “price” refer to constant price set in<br />
case of the fixed pricing and to the initial reservation<br />
price in cases of situated DSS.<br />
Method Price = Price = Price =<br />
90 100 110<br />
Fixed 171.8 190.9 163.0<br />
SDSS1 206.9 213.4 191.1<br />
SDSS2 210.6 213.0 207.4<br />
Table 2: Mean Profits earned by situated DSS<br />
vs. fixed pricing (in $1000)<br />
5. Conclusion<br />
This work proposed applying situated decision<br />
support approach to managing automated SLA<br />
negotiations. The framework is based on the<br />
model for situated decision support that effectively<br />
combines human judgment and autonomous<br />
decision making and action by agent components.<br />
The key idea behind the approach lies in<br />
the managing the fleet of negotiating agents by<br />
the use of a “manager” agent and human decision<br />
maker. We have illustrated the approach<br />
through simulation experiments performed <strong>und</strong>er<br />
a set of simplifying assumptions.<br />
Possible future work could be directed towards<br />
extending the situated DSS to workflow cases,<br />
211
where the service consumers engages into concurrent<br />
negotiations with many service providers<br />
such that SLAs will be made with respect to all<br />
constituents of the workflow. In addition, future<br />
work will comprise the implementation of a<br />
user-friendly prototype involving multiple negotiated<br />
issues and empirical testing involving<br />
human subjects.<br />
References<br />
[1] Angehrn, A.A. and S. Dutta, Case-Based<br />
Decision Support. Communications of the<br />
ACM, 1998. 41(5): p. 77-86.<br />
[2] Carr, N., The End of Corporate Computing.<br />
MIT Sloan Management Review, 2005. 46(3):<br />
p. 66-73.<br />
[3] Chen, E., R. Vahidov, and G.E. Kersten,<br />
Agent-Supported Negotiaitons in the e-<br />
Marketplace. Int. J. Electronic Business, 2005.<br />
3(1): p. 28-49.<br />
[4] Czajkowski, K., et al. SNAP: A Protocol for<br />
Negotiating Service Level Agreements and Coordinating<br />
Resource Management in Distributed<br />
Systems. in 8th Workshop on Job Scheduling<br />
Strategies for Parallel Processing. 2002.<br />
Edinburgh, Scotland.<br />
[5] Dang, J. and M.N. Huhns, Concurrent Multiple-Issue<br />
Negotiation for Internet-Based Services.<br />
Internet Computing, 2006. 10(6): p. 42-<br />
49.<br />
[6] Deora, V. and O. Rana, SLA Enforcement in<br />
Grid using Runtime Violation Prediction and<br />
Reaction. Working Paper, 2007.<br />
[7] Eymann, T., et al. On the Design of a Two-<br />
Tiered Grid Market Structure. in Business Applications<br />
of P2P and Grid Computing, MKWI<br />
2006. 2006.<br />
[8] Faratin, P., C. Sierra, and N.R. Jennings,<br />
Using Similarity Criteria to Make Issue Trade-<br />
Offs in Automated Negotiations. Artificial Intelligence,<br />
2002. 142: p. 205-237.<br />
[9] Gimpel, H., et al. PANDA: Specifying Policies<br />
for Automated Negotiations of Service<br />
Contracts. in 1st International Conference on<br />
Service Oriented Computing (ICSOC 03).<br />
2003. Trento, Italy.<br />
[10] Hasselmeyer, P., et al. Towards Autonomous<br />
Brokered SLA Negotiation. in eChallenges<br />
2006. 2006. Barcelona.<br />
[11] Hess, T.J., L.P. Rees, and T.R. Rakes, Using<br />
Autonomous Software Agents to Create Next<br />
Generation of Decision Support Systems. Decision<br />
Sciences, 2000. 31(1): p. 1-31.<br />
[12] Horrocks, I., et al., Swrl: A semantic web<br />
rule language combining owl and ruleml.<br />
Technical report, 2004.<br />
[13] Jarke, M., M.T. Jelassi, and M.F. Shakun,<br />
MEDIATOR: Towards a Negotiation Support<br />
System. European Journal of Operational Research,<br />
1987. 31(3): p. 314-334.<br />
[14] Kersten, G.E. and G. Lo, Aspire: Integration<br />
of Negotiation Support System and Software<br />
Agents for E-Business Negotiation. International<br />
Journal of Internet and Enterprise<br />
Management (IJIEM), 2003. 1(3): p. (in print).<br />
[15] Kuo, D., M. Parkin, and J. Brooke. Negotiating<br />
Contracts on the Grid. in eChallenges.<br />
2006. Barcelona, Spain.<br />
[16] Li, H., S.Y.W. Su, and H. Lam, On Automated<br />
e-Business Negotiations: Goal, Policy,<br />
Strategy, and Plans of Decision and Action.<br />
Journal of Organizational Computing and Electronic<br />
Commerce, 2006. 13: p. 1-29.<br />
[17] Ludwig, H., et al. Template-Based Automated<br />
Service Provisioning - Supporting the<br />
Agreement-Driven Service Life-Cycle. in International<br />
Conference on Service Oriented Computing.<br />
2005.<br />
[18] Nguyen, T.D. and N.R. Jennings. Coordinating<br />
multiple concurrent negotiations. in 3rd<br />
Int. Conf. on Autonomous Agents and Multi-<br />
Agent Systems. 2004. New York, NY.<br />
[19] Ouelhadj, D., et al., A Multi-agent infrastructure<br />
and a Service Level agreement Negotiation<br />
Protocol for Robust Scheduling in Grid<br />
Computing, in Advances in Grid Computing -<br />
EGC 2005, T.P.A.R. Peter M.A. Sloot Alfons<br />
G. Hoekstra, Marian Bubak, Editor. 2005,<br />
Springer: Heidelberg. p. 651-660.<br />
[20] Parkin, M., D. Kuo, and J. Brooke. A<br />
Framework & Negotiation Protocol for Service<br />
Contracts. in Proceedings of the IEEE International<br />
Conference on Services Computing.<br />
2006.<br />
[21] Rahwan, I., R. Kowalczyk, and H.H. Pham,<br />
Intelligent Agents for Automated One-to-Many<br />
e-Commerce Negotiation. Australian Computer<br />
Science Communications, 2002. 24(197-204).<br />
[22] Rana, O., OGF-19 Report. Technical Report,<br />
2006.<br />
[23] Rao, H.R., R. Sridhar, and S. Narain, An<br />
Active Intelligent Decision Support System.<br />
Decision Support Systems, 1994. 12(1): p. 79-<br />
91.<br />
[24] Sahai, A., et al. Specifying and Monitoring<br />
Guarantees in Commercial Grids through SLA.<br />
in Conference of Cluster Computing and Grids<br />
(CCGrid). 2003.<br />
[25] Siddiqui, M., A. Villazon, and T. Fahringer.<br />
Grid Capacity Planning with Negotiationbased<br />
Advance Reservation for Optimized QoS.<br />
212
in Proceedings of the ACM/IEEE Conference<br />
on Supercomputing. 2006. Tampa, USA.<br />
[26] Vahidov, R. Decision station: a Notion for a<br />
Situated DSS. in 35th Hawaii International<br />
Conference on System Sciences. 2002.<br />
[27] Vahidov, R. and B. Fazlollahi, Pluralistic<br />
Multi-Agent Decision Support System: A<br />
Framework and an Empirical Test. Information<br />
& Management, 2004. 41(7): p. 883-398.<br />
[28] Vahidov, R. and G.E. Kersten, Decision<br />
Station: Situating Decision Support Systems.<br />
Decision Support Systems, 2004. 38(2): p. 283-<br />
303.<br />
[29] Van de Walle, B., S. Heitsch, and P. Faratin.<br />
Coping with One-to-Many Multi-criteria Negotiations<br />
in Electronic Markets. in 12th International<br />
Workshop on Database and Expert Systems<br />
Applications. 2001.<br />
213
Using k-pricing for Penalty Calculation in Grid Market<br />
Michael Becker, <strong>Institut</strong>e of Information Systems and Management, University of Karlsruhe 1<br />
Nikolay Borrisov, <strong>Institut</strong>e of Information Systems and Management, University of Karlsruhe 2<br />
Vikas Deora, School of Computer Science, Cardiff University 3<br />
Omer F. Rana, School of Computer Science, Cardiff University 4<br />
Dirk Neumann, <strong>Institut</strong>e of Information Systems and Management, University of Karlsruhe 5<br />
Abstract<br />
To distribute risk in Grid, the design of service level<br />
agreements (SLAs) plays an important role, since<br />
these contracts determine the price for a service at<br />
an agreed quality level as well as the penalties in<br />
case of SLA violation. This paper proposes a price<br />
function over the quality of service (QoS) on the<br />
basis of the agreements negotiated upon price and<br />
quality objective. This function defines fair prices for<br />
every possible quality of a service, which are in line<br />
with the business of the customer and incentivize the<br />
provider to supply welfare-maximizing quality.<br />
Therewith, penalties can be calculated for every<br />
possible quality level as the difference between the<br />
agreed price and the output of the price function for<br />
the effectively met quality. A price function according<br />
to the k-pricing scheme is presented for a single<br />
service scenario and for a scenario with multiple<br />
interdependent services.<br />
1. Introduction<br />
In order to drive sustainable business growth,<br />
organizations are nowadays constantly improving<br />
their information technology systems to produce<br />
business results sooner and make them available to a<br />
wider audience within the organization. To meet<br />
increasing demands on IT services without bearing<br />
enormous costs, IT services are sourced out more and<br />
more. To be able to fulfill high requirements even on<br />
peak demand, services are purchased on demand.<br />
Unfortunately, all organizations purchasing and<br />
selling computational services in a Service Oriented<br />
Computing (SOC) environment such as Grid have to<br />
_____________________________<br />
1<br />
michael.becker1981@web.de<br />
2 borrisov@iism.uni-karlsruhe.de<br />
3 v.deora@cs.cardiff.ac.uk<br />
4 o.f.rana@cs.cardiff.ac.uk<br />
5<br />
neumann@iism.uni-karlsruhe.de<br />
bear risks. Each outsourced service used by a<br />
company could provide high risks since they cannot<br />
be controlled adequate anymore. But also service<br />
providers bear risks, since their reputation will shrink<br />
and a penalty fee will have to be paid, if it fails to<br />
fulfil the Service Level Agreement (SLA).<br />
A SLA defines the responsibilities of the<br />
service provider and the consumer of the relevant<br />
service by describing the service provided and the<br />
quality standards of the service, measurement<br />
criteria, reporting criteria as well as penalties in cause<br />
of SLA violation. A SLA can be seen as contract that<br />
clearly outlines the rights and obligations of the<br />
parties in order to reduce the scope for disagreement<br />
to arise in the course of the parties` business<br />
relationship. The source of the largest number of<br />
disputes is likely to be the gap between the actual<br />
performance by a service and the performance<br />
expected by the service consumer. Thus, wellnegotiated<br />
SLAs that are comprehensive and well<br />
drafted are f<strong>und</strong>amental to minimizing the risk of<br />
disputes. Thereby SLAs should be negotiated in a<br />
way that both, the service provider and consumer will<br />
profit by successful executed SLAs. “The best SLAs<br />
are setup to allow both you and your service provider<br />
to share in the success and failure of an agreement”<br />
[3].<br />
Besides the establishment of transparency, the<br />
creation of SLAs aims to arrange risk sharing in<br />
order to provide a fair market, which equates<br />
financially weak and strong enterprises as well as<br />
service providers and consumers. To establish a fair<br />
agreement it is important that agreed service level<br />
objective, price and penalties are accomplishable and<br />
in line with the business of the contractual partners.<br />
To incentivize providers to fulfill the agreement,<br />
penalty design plays an important role. Thereby,<br />
penalties have to be designed according to the<br />
business value of the service for the customer and in<br />
a way that incentivizes the provider to fulfill the<br />
agreement satisfactory.<br />
214
The main contribution of this paper is to define<br />
balanced penalties that induce the right incentives for<br />
service providers to meet the SLAs. This paper<br />
introduces a price function on the basis of the results<br />
of the negotiation (price and service quality). This<br />
function provides fair prices for every possible<br />
quality of a service, which are in line with the<br />
business of the customer and incentivize the provider<br />
to supply welfare-maximizing quality. On this basis,<br />
penalty fees can easily be calculated for as the<br />
difference between the agreed price (for the agreed<br />
service level objective) and the output of the price<br />
function (for the effectively met quality).<br />
Section 2 discusses related work. Section 3<br />
defines the price function for penalty calculation<br />
referring to a k-pricing scheme for a single service<br />
scenario and for a scenario with multiple<br />
interdependent services. Section 5 concludes with a<br />
summary and an outlook on future research.<br />
2. Related work<br />
In the field of match making and pricing of<br />
services at a certain quality level, considerable<br />
research provides several pricing schemes in the Grid<br />
context (e.g. [2], [8] for auction and commodity<br />
market models and [6] for an overview of state-ofthe-art<br />
bargaining models).<br />
A review of current research of penalty<br />
calculation in the context of Grid Computing showed<br />
that there is no satisfactory comprehensive research<br />
in this area. In their paper, Yeo and Buyya [9]<br />
presented a proportional share allocation technique<br />
called “LibraSLA” that takes into account the utility<br />
of accepting new jobs into the cluster based on their<br />
SLA. Thereby a linear penalty function that reduces<br />
the profit of a job over time after the lapse of its<br />
deadline is introduced. Equally, a linear penalty<br />
function that reduces profit after a jobs run time has<br />
been presented by [1]. Multiple interdependent<br />
services are not considered in related work.<br />
Since linear penalty functions are not sufficient<br />
to determine penalties according to the business<br />
valuations of service provider and consumer in<br />
practice, this work focuses specifically on presenting<br />
a framework for penalty calculation according to the<br />
business value of the agreed services for the<br />
customer, which also incentivizes the provider to<br />
perform at the best. In doing so, the k-pricing<br />
scheme, well-known as a method for pricing in<br />
double-auctions [4] and already applied for pricing in<br />
Grid context [5] is used to come up with a fair and<br />
incentive compatible price function for all possible<br />
qualities of multiple, interdependent services. In the<br />
following we employ the intuition of k-pricing to the<br />
calculation of penalties.<br />
3. Penalty calculation<br />
3.1. SLA violation<br />
This section explains what SLA violation means<br />
in the context of Grid and how SLA violation could<br />
be measured. For the purpose of this paper, following<br />
general definition of SLA violation is sufficient:<br />
“A service level agreement is violated, as soon as it<br />
is impossible to meet the agreed service level<br />
objective(s).”<br />
The determination of the degree of the breach is<br />
important for penalty calculation to enable penalty<br />
design according to the occurring loss of the<br />
customer. For the purpose of this paper, the quality of<br />
service q is arbitrary normalized on a percentage rate<br />
basis, i.e. q ∈ 0,100 , in order to provide<br />
comparability of the degree of SLA breaches. For<br />
example, availability measured in percent could be<br />
considered as normalized service quality.<br />
3.2. Requirements on penalty scheme<br />
The value of the penalty fee of a SLA has deep<br />
impact on the SLA negotiation and the decision of<br />
accepting or rejecting an offer. Along with the price<br />
for a service and the reputation and reliability of the<br />
prospective associate partner the penalty in case of<br />
violation constitute the basis of decision-making<br />
whether to accept an offer or not. Hence following<br />
objectives for penalty calculation can be identified:<br />
Requirement 1: Penalties should be fair. The<br />
penalty agreed in a SLA should be proportionable to<br />
the loss triggered by violation. Penalty calculation<br />
has to be specific to customer`s (and provider`s)<br />
business and should truly establish risk sharing [3].<br />
Requirement 2: Penalties should incentivize<br />
contractual partners to perform best for each<br />
other. The penalty agreed in the SLA should<br />
incentivize the provider to fulfill the agreement<br />
satisfactory to the customer concerning the delivered<br />
Quality of service (QoS). This effect can be achieved<br />
by establishing a penalty proportional to the degree<br />
of SLA violation and by implementing a reward,<br />
which will be paid, if the agreed QoS is<br />
215
outperformed. Although the amount of this penalty<br />
and reward respectively has to be greater than the<br />
cost the provider may save by not performing at its<br />
best. Furthermore, the business value of the provided<br />
service level for the customer has to be taken into<br />
account, since the consumer should not be worse off,<br />
if the provider increases service quality (suppose the<br />
business value of a service will diminish as from a<br />
certain value).<br />
According to this, the business value of the<br />
service for the consumer as well as the production<br />
costs expressed by the reservation price for the<br />
provider have to be taken into account when<br />
calculating the penalty and reward respectively in<br />
SLAs.<br />
3.3. Required information and assumptions<br />
This section describes the needed information<br />
for proper penalty calculation. Thereby a single<br />
service scenario, which is about one single SLA<br />
between one service consumer and one provider that<br />
includes exactly one service, as well as a multiple<br />
service scenario is considered. The multiple service<br />
scenario is about multiple, interdependent SLAs<br />
between one service consumer and one service<br />
provider, whereas every service is represented by<br />
exactly one SLA.<br />
Information 1: Business value of service<br />
qualities for the consumer. The business value BV<br />
of a service for the consumer can be expressed as a<br />
function of the quality q of a service. Thereby, the<br />
business value represents the amount of currency<br />
units CU the service is worth for the consumer at<br />
each possible quality of a service. For example, the<br />
consumer is willing to pay $1.5 for every additional<br />
percentage point of availability of a service. Then its<br />
business value of this service can be expressed as a<br />
straight line BV q = 1.5q [$] in the domain<br />
q ∈ [0,100]. The cardinal definition based on the<br />
measurement in CU is essential for penalty<br />
calculation, since a same base for the functions BV<br />
and the provider`s valuation function R (introduced<br />
below) is indispensible for penalty calculation.<br />
Furthermore, the strength of preferences (slope of<br />
function BV) has magnificent influence on the<br />
penalty calculation. The quality is defined as the<br />
provided quality relative to the determined maximum<br />
quality (e.g. availability expressed in percent). Thus,<br />
the business value of a single service can be<br />
expressed as follows:<br />
for single services and<br />
BV: q → IR + ∀q ∈ [0,100]<br />
BV: q 1 , … , q n → IR +<br />
∀q i ∈ [0,100]<br />
for the multiple service scenario respectively,<br />
whereas q i represents the quality of service i.<br />
Thereby, the business value represents the amount of<br />
currency units CU each possible combination of<br />
qualities of the n services is worth for the consumer.<br />
This graph mirrors the consumer`s preferences about<br />
the quality of service and could vary for every<br />
consumer as well as for every service considered.<br />
To simplify the design of incentive constraints,<br />
it is assumed that the function BV(q i ) is completely<br />
differentiable in every direction in the domain<br />
q i ∈ [0,100] for all possible i = 1, … , n. Since the<br />
business value of a service for the consumer has to be<br />
determined in advance of the negotiation and match<br />
making process, it is assumed that consumers have<br />
already disclosed their valuations to the market<br />
directory when it comes to penalty calculation.<br />
Furthermore it is assumed that consumers truly<br />
disclose business values and do not distort their<br />
preferences in order to manipulate penalty<br />
calculation. This assumption will be discussed in<br />
section 5.<br />
Information 2: Reservation price of the<br />
service provider. Equally, the reservation price R<br />
(measured in CU), which represents the value of the<br />
service for the provider can be expressed as a<br />
function of the quality q of service<br />
R: q → IR + ∀q ∈ [0,100]<br />
in the single service scenario and as following graph<br />
R: q 1 , … , q n → IR +<br />
∀q i ∈ [0,100]<br />
in the multiple service scenario. Thereby the same<br />
assumptions as for the Business Value BV(q i ) apply.<br />
Opportunity costs occurring during service execution<br />
because of arising business alternatives are not taken<br />
into account, since they are very hard to measure and<br />
the service provider will have no incentive to<br />
disclose his true estimation.<br />
216
Information 3: Quality of service according<br />
to SLA. The quality of service q ∗ defined in the<br />
service level objectives (SLO) of the SLA is<br />
necessary to determine whether a penalty has to be<br />
paid by the provider or a reward has to be paid by the<br />
service consumer. For multiple services several SLO<br />
have been negotiated:<br />
BV [CU]<br />
q i ∗ ∈ 0,100 ∀ i = 1, … , n<br />
Information 4: Price of service according to<br />
SLA. In the negotiation process both parties have<br />
agreed upon one certain price p ∗ for the services<br />
provided at the quality q ∗ i . Thereby it is assumed,<br />
that both parties have agreed upon one certain all<br />
ro<strong>und</strong> price p ∗ for all services i provided at the<br />
agreed qualities q ∗ i . This assumption is maintainable,<br />
since all considered services, each represented by one<br />
SLA, usually belong to one contract between the<br />
service provider and consumer. All these services<br />
included in one contract should be negotiated<br />
together, since interdependencies between the<br />
services could have significant influence on the<br />
valuation for both contractual partners. Therefore,<br />
these interdependencies already have been taken into<br />
account in the negotiation process. The result of the<br />
negotiation is a b<strong>und</strong>le of service level objectives<br />
(q ∗ 1 , q ∗ ∗<br />
2 , … , q n−1 , q ∗ n ) and one agreed all ro<strong>und</strong> price<br />
p ∗ ∈ IR + in currency units [CU].<br />
p ∗ ∈ IR + ,<br />
measured in currency units [CU]<br />
This price p ∗ will be basis for penalty and reward<br />
calculation respectively via a price function<br />
described subsequently.<br />
Figure 1. exemplary shows the initial<br />
information in the single service scenario: business<br />
value for the consumer BV (q), reservation price of<br />
the provider R (q), agreed price p ∗ and quality of<br />
service q ∗ according to SLA.<br />
The exemplary function BV may be typical for<br />
IT services: It increases slowly at low qualities,<br />
because very low service quality (e.g. availability)<br />
usually will have no useful effect to the consumer.<br />
From a certain quality level on, BV may increase<br />
progressive. At high quality levels marginal utility<br />
could diminish caused by saturation.<br />
3.4. Incentive and fairness constraints<br />
In the following several constraints for<br />
designing a price function P(q i ) are described, which<br />
secure to meet the objective of establishing fairness<br />
(fairness constraints) as well as incentivizing the<br />
provider to act satisfactory to its customer (incentive<br />
constraints). This function P(q i ) represents the total<br />
amount of currency units (CU) the consumer will<br />
have to pay for the services. The penalty and the<br />
reward respectively can be calculated as the<br />
difference between the agreed price p ∗ and the output<br />
P(q i ) of effectively met qualities q i . P(q i ) can be<br />
defined as follows:<br />
P: q 1 , … , q n → IR +<br />
∀ q i ∈ [0,100]<br />
whereas following constraint has to be fulfilled:<br />
P q 1 ∗ , … , q n<br />
∗<br />
= p ∗<br />
This constraint guarantees the agreed price p ∗ for met<br />
service level objectives q i ∗ , i.e. no penalty or reward<br />
has to be paid, if the agreed qualities are met exactly.<br />
BV(q)<br />
R(q)<br />
p*<br />
q*<br />
0 50 100<br />
q [%]<br />
To implement a SLA pricing design which<br />
maximizes overall welfare, several constraints have<br />
to be added. In this context, the welfare which has to<br />
be distributed among the contractual partners is<br />
defined by the difference W q = BV q – R(q) for<br />
every possible service quality q. To clarify the<br />
explanations of the following constraints, only one<br />
service is regarded and a case differentiation is<br />
introduced in order to reduce complexity:<br />
Figure 1. Initial situation for single service<br />
217
a. ∀q ∈ q<br />
∂R(q)<br />
∂q<br />
∂R q<br />
∂q<br />
≤<br />
∂BV q<br />
∂q<br />
≤ ∂P(q)<br />
∂p<br />
, R q ≤ BV(q)}:<br />
≤ ∂BV(q)<br />
∂q<br />
R q ≤ P q ≤ BV q<br />
(IC)<br />
FC<br />
Case a. represents the situation that the given<br />
quality of service is rated higher by the consumer<br />
BV q than by the provider R q . Furthermore the<br />
slope of function BV(q) is greater than the slope of<br />
reservation price R(q). In this case welfare W(q) is<br />
positive and increases with rising service quality q.<br />
To incentivize the service provider to deliver higher<br />
quality in order to maximize overall welfare, the<br />
price P(q) has to increase faster than the reservation<br />
price R(q). Although, the price P(q) has to increase<br />
slower than the business value BV q to secure that<br />
the consumer will not be worse off, if the provider<br />
delivers higher quality. These requirements are<br />
expressed by the incentive constraint (IC). The<br />
fairness constraint (FC) guarantees that no party will<br />
have to bear a loss as long as overall welfare is<br />
positive.<br />
∂R q<br />
b. ∀q ∈ {q|<br />
∂q<br />
∂R(q)<br />
∂q<br />
≤<br />
∂BV q<br />
∂q<br />
≤ ∂P(q)<br />
∂p<br />
, R q ≥ BV q }:<br />
≤ ∂BV(q)<br />
∂q<br />
BV q ≤ P q ≤ R q<br />
(IC)<br />
FC<br />
On the other hand case b. represents the<br />
situation that reservation prices of the provider are<br />
higher than the monetary utility for the consumer<br />
R q > BV q . Furthermore the slope of function<br />
BV(q) is greater than the slope of the reservation<br />
price function R(q). This means that overall welfare<br />
W(q) is negative but increasing with rising service<br />
quality q. The incentive constraint (IC) implements<br />
the incentive for the service provider to supply higher<br />
quality in order to maximize overall welfare without<br />
making the consumer being worse off, i.e. the price<br />
P(q) has to increase faster than its reservation price<br />
R(q) and slower than BV q . The fairness constraint<br />
(FC) guarantees that no party will make a profit as<br />
long as overall welfare is negative.<br />
c. ∀q ∈ q<br />
∂R q<br />
∂q<br />
∂BV(q)<br />
∂q<br />
><br />
∂BV q<br />
∂q<br />
≤ ∂P(q)<br />
∂q<br />
, R q ≤ BV q :<br />
≤ ∂R(q)<br />
∂q<br />
(IC)<br />
R q ≤ P q ≤ BV q<br />
FC<br />
Case c. represents the situation of positive<br />
overall welfare W(q), which is decreasing with<br />
rising service quality q. To incentivize the service<br />
provider to deliver lower quality in order to<br />
maximize overall welfare, the price P(q) has to<br />
increase slower than R(q), but faster than the<br />
business value BV q to secure that the consumer<br />
will not be worse off if the provider decreases<br />
quality. Again, the fairness constraint secures that no<br />
party will have to bear a loss as long as overall<br />
welfare is positive.<br />
d. ∀q ∈ q<br />
∂R q<br />
∂q<br />
∂BV(q)<br />
∂q<br />
><br />
∂BV q<br />
∂q<br />
≤ ∂P(q)<br />
∂q<br />
, R q ≥ BV q :<br />
≤ ∂R(q)<br />
∂q<br />
BV q ≤ P q ≤ R q<br />
FC<br />
(IC)<br />
Finally, in case d. overall welfare W(q) is<br />
negative and decreasing with rising service quality q.<br />
To incentivize the service provider to deliver lower<br />
quality in order to maximize overall welfare, the<br />
price P(q) has to increase slower than the reservation<br />
price R(q) of the provider but faster as the business<br />
value to the consumer again. The fairness constraint<br />
guarantees that no contractual partner will make a<br />
profit as long as overall welfare is negative.<br />
Note that following assumption forms the basis<br />
of these constraints: If the marginal reservation price<br />
of the service provider equals the marginal revenue<br />
of the service provider, it will act according the<br />
preferences of the service consumer.<br />
In conclusion, these constraints can be extended<br />
for the multiple service scenario as follows:<br />
s.t.<br />
P: q 1 , … , q n → IR +<br />
∀q i ∈ [0,100]<br />
P q 1 ∗ , … , q n<br />
∗<br />
= p ∗<br />
and the incentive constraint IC :<br />
min ∂R q i<br />
∂q i<br />
, ∂BV q i<br />
∂q i<br />
≤ ∂P q i<br />
∂q i<br />
≤ max{ ∂R(q i)<br />
, ∂BV(q i)<br />
} ∀q<br />
∂q i ∂q i ∈ [0,100]<br />
i<br />
218
and the fairness constraint (FC):<br />
min R q i , BV q i ≤ P q i<br />
≤ max{R q i , BV(q i )}<br />
∀i = 1, … , n<br />
Every function P(q i ) which fulfills these<br />
constraints incentivizes the service provider to act in<br />
a way that maximizes overall welfare and guarantees<br />
that no party will bear a loss, if overall welfare is<br />
positive (and no party will make a profit, if overall<br />
welfare is negative respectively), i.e. the objective<br />
“service provider and consumer win and lose<br />
together” is implemented.<br />
Even for not completely differentiable valuation<br />
functions BV(q i ) and R(q i ), a price function P(q i ),<br />
which fulfils the requirements can be designed by<br />
introducing steps according the considerations made<br />
above.<br />
The introduced constraints do not determine the<br />
distribution of welfare to service provider and<br />
customer. This profit distribution is depending on the<br />
design of the price function P q i according to these<br />
constraints. An exemplary price function which<br />
guarantees fair distribution of the overall welfare is<br />
introduced in the following section.<br />
3.5. Using k-pricing for penalty calculation<br />
for single services<br />
This section aims to come-up with a price<br />
function that fulfils the incentive constraints<br />
introduced above for a single service scenario, i.e.<br />
only one SLA between one service provider and one<br />
consumer is considered. The resulting penalty of a<br />
SLA can easily be computed as the difference of the<br />
agreed price p ∗ for met service level objective and<br />
the output of the price function for effectively met<br />
service quality q:<br />
Penalty(q) = p ∗ − P(q)<br />
Thereby a negative penalty is called a reward, since it<br />
has to be paid from the consumer to the provider for<br />
outperforming services, i.e. if the effectively met<br />
service quality is higher than the agreed service level<br />
objective.<br />
Sattherthwaite and Williams [4] presented a k-<br />
pricing scheme to distribute the welfare of trades in<br />
double auction markets. The <strong>und</strong>erlying idea of the k-<br />
pricing scheme is to determine prices for a buyer and<br />
a seller on the basis of the difference between their<br />
valuations. For instance, suppose that a buyer wants<br />
to purchase a computation service for $5 and a seller<br />
wants to sell for at least $4. The difference between<br />
these bids is W = 5 − 4 = 1, where W is the surplus<br />
of this transaction that can be distributed among the<br />
participants. For a single service exchange, the k-<br />
pricing scheme can be formalized as follows [5].<br />
According to chapter 0, let BV q = bv be the<br />
valuation of the consumer and R q = r be the<br />
valuation of the provider of the service at a given<br />
quality level q. The price for the service of quality q<br />
can be calculated by P q = kbv + 1 − k r with<br />
k ∈ [0,1]. In regard to the above-mentioned<br />
valuations for the service with the quality q, the<br />
resulting price is P q = 0.5 ∗ 4 + 1 − 0.5 ∗ 5 =<br />
4.5 [$] for k = 0.5.<br />
Sattherthwaite and Williams assumed that<br />
BV q ≥ R(q), i.e. the consumer has a valuation for<br />
the commodity which is at least as high as the seller`s<br />
reservation price. Otherwise no trade would occur.<br />
This assumption is not applying in the context of<br />
penalty calculation, since (unwished) situations could<br />
occur in which the business value for the consumer is<br />
lower than the reservation price of the provider. E.g.,<br />
suppose the provider has served 100% availability<br />
until a certain point of time and suddenly it is unable<br />
to provide the service due to an electrical power<br />
outage until the agreed time is past. Then the met<br />
quality of service could be unexpected low and the<br />
situation mentioned above could occur.<br />
The k-pricing scheme can also be applied to<br />
penalty calculation in SLAs in order to provide a<br />
price function that guarantees fair distribution of the<br />
welfare according to the constraints introduced in the<br />
previous chapter for every possible valuation<br />
functions BV q and R q :<br />
P q ≔ k ∗ BV q + 1 − k ∗ R q<br />
∀q ∈ [0,100]<br />
whereas following equation determines parameter k:<br />
k ∶= p∗ − R(q ∗ )<br />
BV q ∗ − R(q ∗ )<br />
This equation guarantees the fulfilment of the<br />
constraint P q ∗ = p ∗ and secures the distribution of<br />
positive and negative overall welfare according to the<br />
welfare distribution agreed in the SLA negotiation<br />
process (for the price p ∗ of the service at the quality<br />
219
level q ∗ ). Figure 2. shows the price function P(q) for<br />
the exemplary initial situation showed in Figure1.<br />
BV [CU]<br />
P(q)<br />
BV(q)<br />
R(q)<br />
p*<br />
q*<br />
0 50 100<br />
q [%]<br />
Figure 2. Price function calculation using k-<br />
pricing (k=0.8)<br />
3.6. Using k-pricing for penalty calculation<br />
for multiple services<br />
Provided that only one overall price has been<br />
negotiated (see section 3.3) to meet exactly one<br />
service level objective for every service, k-pricing<br />
could be used to calculate fair, incentive compatible<br />
penalties for multiple services as follows:<br />
with<br />
P q 1 , … , q n ≔<br />
k ∗ BV q 1 , … , q n + 1 − k ∗ R q 1 , … , q n<br />
k ∶=<br />
∀q i ∈ 0,100 ; i = 1, … , n<br />
p ∗ − R(q 1 ∗ , … , q n ∗ )<br />
BV q 1 ∗ , … , q n<br />
∗<br />
− R(q 1 ∗ , … , q n ∗ )<br />
Again, this equation guarantees the fulfilment<br />
of the constraint P(q 1 ∗ , … , q n ∗ ) = p ∗ and secures the<br />
distribution of positive and negative overall welfare<br />
according to the welfare distribution agreed in<br />
negotiation process. The incentive constraints<br />
determined in section 3.4 are met by this price<br />
function.<br />
3.7. Application in Grid Markets<br />
Since the mathematical formulation of valuation<br />
functions, especially for n-dimensional valuations in<br />
the multiple service scenario, would be too complex<br />
in practice, a simplified framework with tiered<br />
valuation functions is introduced in this section.<br />
To ease the formulation of valuation functions,<br />
the quality range of all services s i is divided into a<br />
certain number j i of intervals. The relevant service<br />
qualities have to be valuated by the provider and the<br />
consumer with one value for each interval in the<br />
single service scenario and with one value for each<br />
possible combination of quality domains in the<br />
multiple service scenario (since services are<br />
interdependent). Thereby, the number of intervals j i<br />
does not necessarily have to be equal for all services<br />
and the length of intervals does not necessarily be the<br />
same within one service. These parameters have to be<br />
set according to the usual structure of the valuation<br />
functions for every service. The parameter setting<br />
stands for a trade-off between convenience and<br />
accuracy, i.e. a high number of short valuation<br />
intervals will provide better incentives, but will also<br />
raise complexity. It must be pointed out, that<br />
complexity increases very fast with the number of<br />
relevant services. With two services and five quality<br />
intervals for each service, 5 2 = 25 combinations<br />
have to be valuated by both contractual partners.<br />
With three services and the same classification,<br />
already 5 3 = 75 combinations have to be valuated.<br />
This classification of quality levels leads to step<br />
functions according to the business value and the<br />
reservation price of the services, which could be<br />
discontinuous at the interval bo<strong>und</strong>s. In the intervals<br />
the slope of the functions is zero. According to this,<br />
the valuation functions generally can be defined as<br />
follows:<br />
Information 1 (simplified): Business value<br />
BV(q i ) of service qualities for the consumer for n<br />
services:<br />
BV:<br />
n<br />
i=1<br />
Q i<br />
→ IR + ∀i = 1, … , n<br />
whereas BV is a graph spanned over the Cartesian<br />
product<br />
with<br />
n<br />
i=1<br />
Q i<br />
= Q 1 × … × Q n ∶=<br />
q 1 , … , q n |q i ∈ Q i ; i = 1, … , n<br />
Q i : = (q 1,1 , q 1,2 , … , q 1,j1 −1, q 1,j1 ).<br />
Thereby Q i represents the set of quality intervals<br />
q i,1 , q i,2 , … , q i,j1 −1, q i,j1 of service i, whereas j i is the<br />
number of quality intervals of service i. In other<br />
220
words, all possible combinations of quality intervals<br />
of the different services have to be valuated by the<br />
consumer by allocation of one positive, monetary<br />
value.<br />
Information 2 (simplified): Reservation price<br />
R(q i ) of the service consumer for n services.<br />
Equally, the reservation price of the provider can be<br />
expressed as a graph:<br />
R:<br />
n<br />
i=1<br />
Q i<br />
→ IR + ∀i = 1, … , n<br />
whereas R is spanned over the Cartesian product<br />
n<br />
i=1 Q i again.<br />
With information about the results of the<br />
negotiation process, as a b<strong>und</strong>le of service level<br />
objectives (q ∗ 1 , q ∗ ∗<br />
2 , … , q n−1 , q ∗ n ) and one agreed all<br />
ro<strong>und</strong> price p ∗ for all the services, the price function<br />
applying k-pricing can be written as follows:<br />
with<br />
P q 1 , … , q n ≔<br />
k ∗ BV q 1 , … , q n + 1 − k ∗ R q 1 , … , q n<br />
k ∶=<br />
∀q i ∈ 0,100 ; i = 1, … , n<br />
p ∗ − R(q 1 ∗ , … , q n ∗ )<br />
BV q 1 ∗ , … , q n<br />
∗<br />
− R(q 1 ∗ , … , q n ∗ )<br />
This equation guarantees the fulfilment of the<br />
constraint P(q 1 ∗ , … , q n ∗ ) = p ∗ and secures the<br />
distribution of positive and negative overall welfare<br />
according to the welfare distribution agreed in<br />
negotiation process. Of course, fulfilling the<br />
incentive compatibility conditions presented in<br />
section 3.4 is not feasible with tiered functions. Since<br />
these step functions could be considered as<br />
approximation and pass into real valuation functions<br />
for sufficiently small intervals, incentive<br />
compatibility can be achieved in some degree<br />
anyway.<br />
Figure 3. visualises k-pricing in case of the<br />
simplified scenario by using step functions for<br />
service valuation for a single service.<br />
BV [CU]<br />
P(q)<br />
BV(q)<br />
R(q)<br />
p*<br />
q*<br />
0 50 100<br />
q[%]<br />
Figure 3. Simplified step functions BV(q),<br />
R(q) and P(q) using k-pricing (k=0.8)<br />
In the following a numerical example with two<br />
interdependent services is drawn to clarify the<br />
(simplified) k-pricing scheme in multiple services<br />
scenario. For the sake of clarity a graphic account has<br />
been left out.<br />
In this example, two interdependent services are<br />
considered. These services are supposed to be<br />
complements for the consumer, i.e. these services<br />
should be consumed together. In economics, a good 1<br />
is a complement to good 2, if the demand for good 1<br />
goes down when the price of good 2 goes up [7]. A<br />
famous example of complement goods would be<br />
hamburgers and hamburger buns. In a Grid market,<br />
CPU and storage could be considered as complement<br />
goods, since there may be only very limited use for<br />
an application to purchase computing power without<br />
having the appropriate memory capacity. The same<br />
services could be of substitute character to the<br />
service provider, for example if the provision of these<br />
services is independent from each other. Then the<br />
reservation price for both services will simply be the<br />
sum of the reservation prices of the two single<br />
services. Following tables show exemplary<br />
valuations of two services, which are of complement<br />
character for the consumer and of substitute character<br />
for the provider respectively.<br />
Table 1. Exemplary business value BV(q 1 , q 2 )<br />
of complements for the consumer<br />
q 2,1 q 2,2 q 2,3 q 2,4<br />
∈ 0,25 ∈ [25,50) ∈ [50,75) ∈ [75,100]<br />
q 1,1 ∈ 0,25 0 5 10 20<br />
q 1,2 ∈ [25,50) 5 50 60 70<br />
q 1,3 ∈ [50,75) 10 60 100 110<br />
q 1,4 ∈ [75,100] 20 70 110 150<br />
221
Table 2. Exemplary provider`s reservation<br />
price R(q 1 , q 2 ) of substitutes<br />
q 2,1 q 2,2 q 2,3 q 2,4<br />
∈ 0,25 ∈ [25,50) ∈ [50,75) ∈ [75,100]<br />
q 1,1 ∈ 0,25 10 20 30 40<br />
q 1,2 ∈ [25,50) 20 40 50 60<br />
q 1,3 ∈ [50,75) 30 50 60 70<br />
q 1,4 ∈ [75,100] 40 60 70 80<br />
Table 4. Penalties and rewards for different<br />
service levels<br />
q 2,1 q 2,2 q ∗ 2 = q 2,3 q 2,4<br />
∈ 0,25 ∈ [25,50)] ∈ [50,75) ∈ [75,100]<br />
q 1,1 ∈ 0,25 72.5 63.75 55 45<br />
q 1,2 ∈ [25,50) 63.75 37.5 27.5 17.5<br />
∗<br />
q 1,3 ∈ [50,75) 55 27.5 10 0<br />
q 1,4 ∈ [75,100] 45 17.5 0 -17.5<br />
Suppose the service level objective is to meet a<br />
quality q 1 ∈ [75,100] for service s 1 and quality<br />
q 2 ∈ [50,75) for service s 2 for an all ro<strong>und</strong> price<br />
p ∗ = 80 [CU], which has been determined for met<br />
service level objectives in the negotiation process. So<br />
factor k can be calculated as<br />
k =<br />
p ∗ − R q 1 ∗ , q 2<br />
∗<br />
BV q 1 ∗ , q 2<br />
∗<br />
− R q 1 ∗ , q 2<br />
∗<br />
= 0.25<br />
=<br />
80 − 70<br />
110 − 70<br />
With the equation presented above the price can be<br />
calculated for every possible combination of services.<br />
P q 1,a , q 2,b =<br />
k ∗ BV q 1,a , q 2,b + 1 − k ∗ R q 1,a , q 2,b<br />
∀ a = 1, … ,4 and ∀ b = 1, … ,4<br />
Table 3. shows the results of a price calculation.<br />
Table 3. Prices P(q 1 , q 2 ) calculated via k-<br />
pricing (k = 0, 5)<br />
q 2,1 q 2,2 q ∗ 2 = q 2,3 q 2,4<br />
∈ 0,25 ∈ [25,50) ∈ [50,75) ∈ [75,100]<br />
q 1,1 ∈ 0,25 7.5 16.25 25 35<br />
q 1,2 ∈ [25,50) 16.25 42.5 52.5 62.5<br />
∗<br />
q 1,3 ∈ [50,75) 25 52.5 70 80<br />
q 1,4 ∈ [75,100] 35 62.5 80 = p ∗ 97.5<br />
The calculated prices are always between the<br />
valuations of the relevant service partners of the<br />
contractual partners. For the agreed service level<br />
objective q 1 ∗ ∈ [75,100] and q 2 ∗ ∈ [50,75) the<br />
calculated price P(q 1 ∗ , q 2 ∗ ) equals the agreed price p ∗ .<br />
From these prices again the penalty and reward<br />
respectively can be calculated as<br />
Penalty(q 1 , q 2 ) = p ∗ − P(q 1 , q 2 )<br />
whereas q i is the actually met quality of service i.<br />
For the agreed service level objective<br />
q 1 ∗ ∈ [75,100] and q 2 ∗ ∈ [50,75) the penalty is zero,<br />
since the calculated price P(q 1 ∗ , q 2 ∗ ) equals the agreed<br />
price p ∗ . The penalty for q 1 ∈ [50,75) and q 2 ∈<br />
[75,100] is also zero, because of the symmetry of the<br />
valuations in this example If the SLO is<br />
outperformed, i.e. q 1 ∈ [75,100] and q 2 ∈<br />
[75,100], the consumer will pay a reward (negative<br />
penalty) to the service provider.<br />
Table 5. Benefit of service provider<br />
P(q 1 , q 2 ) − R(q 1 , q 2 )<br />
q 2,1 q 2,2 q ∗ 2 = q 2,3 q 2,4<br />
∈ 0,25 ∈ [25,50) ∈ [50,75) ∈ [75,100]<br />
q 1,1 ∈ 0,25 -2.5 -3.75 -5 -5<br />
q 1,2 ∈ [25,50) -3.75 2.5 2.5 2.5<br />
∗<br />
q 1,3 ∈ [50,75) -5 2.5 10 10<br />
q 1,4 ∈ [75,100] -5 2.5 10 17.5<br />
Table 5. shows the benefit of the service<br />
provider at each quality level combination calculated<br />
as the difference P(q 1, q 2 ) − R(q 1, q 2 ). Table 6.<br />
presents the associated benefits for the consumer,<br />
calculated equivalent as the difference between its<br />
valuation and the determined price BV(q 1, q 2 ) −<br />
P(q 1, q 2 ).<br />
Table 6. Benefit of service consumer<br />
BV(q 1 , q 2 ) − P(q 1 , q 2 )<br />
q 2,1 q 2,2 q ∗ 2 = q 2,3 q 2,4<br />
∈ 0,25 ∈ [25,50) ∈ [50,75) ∈ [75,100]<br />
q 1,1 ∈ 0,25 -7.5 -11.25 -15 -15<br />
q 1,2 ∈ [25,50) -11.25 7.5 7.5 7.5<br />
∗<br />
q 1,3 ∈ [50,75) -15 7.5 30 30<br />
q 1,4 ∈ [75,100] -15 7.5 30 52.5<br />
As can be seen from these tables, both parties<br />
will bear a loss, if at least one service is delivered at<br />
the lowest quality. As soon as both services are<br />
provided in a quality greater or equal than 25%, both<br />
parties will make a (positive) profit, which is<br />
222
increasing with rising quality of both services. For<br />
every possible change of service qualities making the<br />
provider being better off, the service consumer will<br />
also be better off. Hence, the provider is incentivized<br />
to provide both services in best possible quality to<br />
maximize its profit. Due to the fixed ratio of overall<br />
welfare distribution provided by the k-pricing<br />
scheme, this will maximize the benefit of the<br />
consumer too. In this case the SLO is outperformed<br />
by the service provider and an overall welfare even<br />
greater than the welfare in the originally agreed<br />
situation will be distributed fairly.<br />
4. Conclusion<br />
Penalty calculation is crucial for true risk<br />
sharing in SLAs, since it determines the distribution<br />
of the associated risk of the SLA among the<br />
contractual partners. The k-pricing scheme provides<br />
fair and economical reasonable prices for every<br />
quality level of a single service. Thereby the overall<br />
welfare is distributed at a fixed ratio according to the<br />
result of a negotiation process for all possible met<br />
service qualities. Thus, the party with the bigger<br />
share of positive welfare will also bear a higher share<br />
of the loss, if the SLA is violated, i.e. the party which<br />
gets a bigger share of the positive welfare also bears<br />
a higher risk. Due to these characteristics and the<br />
fact, that k-pricing fulfils the incentive constraints<br />
presented in section 3.4 (proof trivial) and provides<br />
fair welfare distribution, k-pricing is the ideal scheme<br />
for penalty calculation in automated markets.<br />
However, the problem of using k-pricing for penalty<br />
calculation is that it provides incentives for market<br />
participants to distort their true valuations. Suppose a<br />
service consumer is interested in a service at one<br />
specific level of quality. Then he is incentivized to<br />
<strong>und</strong>erstate its valuation for all other service levels in<br />
order to maximize penalties in case of violated SLA<br />
and to minimize rewards in case of outperformed<br />
SLO. This incentive problem cannot be solved in<br />
context with k-pricing, since the price function is<br />
always directly associated with the valuations and<br />
thus directly influence able. Thus, the incentive to<br />
report true valuations has to be provided by other<br />
components in a Grid Market, maybe the Market<br />
Mechanism, which matches supply and demand. Just<br />
as well the creditability of the contractual partners<br />
regarding the disclosed valuations could indirectly be<br />
gauged by observing valuations for similar services<br />
in the past. But surely the incentive to distort<br />
valuations will remain as a challenge to be solved.<br />
References<br />
[1] Irwin D.E., Grit L.E., Chase J.S. (2004): Balancing<br />
Risk and Reward in a Market-based Task Service. In<br />
Proceedings of the 13th International Symposium on High<br />
Performance Distributed Computing (HPDC13), Honolulu,<br />
Hawaii, June 2004, pp. 160-169.<br />
[2] Joita L., Rana O.F., Gray W., Miles J.(2004): A Double<br />
Auction Economic Model for Grid Services. Proceedings<br />
of Euro-Par 2004 Parallel Processing - 10th International<br />
Euro-Par Conference, Pisa, Italy, 31 August - 3 September<br />
2004, ISBN 3-540-22924-8 Springer Berlin Heidelberg<br />
New York,<br />
http://www.springerlink.com/index/XLADBT1AW3HPXC<br />
8L .<br />
[3] Quick A. (2007): Service Level Agreements (SLA)<br />
Boot Camp. http://helpdesk.wyopub.com/2005/07/servicelevel-agreement-sla-boot-camp.html<br />
, recalled May 14<br />
2007.<br />
[4] Sattherthwaite, M. A., Williams, S. R., (1993). The<br />
Bayesian Theory of the k-Double Auction. In: Friedman,<br />
D., Rust J. (Eds.), The Double Auction Market -<br />
<strong>Institut</strong>ions, Theories, and Evidence. Addison-Wesley, Ch.<br />
4, pp. 99-123.<br />
[5] Schnizler B., Neumann D., D. Veit, C. Weinhardt<br />
(2006): Trading Grid Services – A Multi-attribute<br />
Combinatorial Approach. European Journal of Operations<br />
Research. Forthcoming, Download from Science Direct.<br />
[6] Sim K. M. (2006): A Survey of Bargaining Models for<br />
Grid Resource Allocation. ACM SIGECOM: E-commerce<br />
Exchange, Vol. 5, No. 5, January, 2006.<br />
[7] Varian H. R. (2003): Intermediate microeconomics: a<br />
modern approach. 6 th ed, Norton, New York, 2003.<br />
[8] Wolski R., Brevik J., Plank J., Bryan T. (2003): Grid<br />
Resource Allocation and Control Using Computational<br />
Economies. In F. Berman, A. Hey, and G. Fox (eds.): Grid<br />
Computing – Making the Global Infrastructure a Reality. J.<br />
Wiley, New York, 2003.<br />
[9] Yeo C.S., Buyya R. (2005): Service Level Agreement<br />
based Allocation of Cluster Resources: Handling Penalty to<br />
Enhance Utility. Proceedings of the 7th IEEE International<br />
Conference on Cluster Computing (Cluster 2005, IEEE CS<br />
Press, Los Alamitos, CA, USA), Sept. 27 - 30, 2005,<br />
Boston, Massachusetts, USA.<br />
223
20 th Bled eConference<br />
eMergence:<br />
Merging and Emerging Technologies, Processes, and <strong>Institut</strong>ions<br />
June 4 - 6, 2007; Bled, Slovenia<br />
Rightsizing of Incentives for collaborative e-Science<br />
Grid Applications with TES<br />
Arun Anandasivam, Dirk Neumann, Christof Weinhardt<br />
<strong>Institut</strong>e for Information Systems and Management,<br />
University of Karlsruhe, Germany<br />
{name}@iism.uni-karlsruhe.de<br />
Abstract<br />
The particle physics community is sharing its resources over a Grid network.<br />
Currently resources are used on availability. But there is no fair allocation<br />
considering the importance or the private value of a job. Researchers are using<br />
Grid resources regardless of whether others need them more urgently.<br />
Furthermore, they do not even provide their own resources as they do not have an<br />
incentive for sharing them. Researchers are demanding a Grid infrastructure<br />
where an incentive mechanism is implemented. This will support a fair allocation<br />
of resources. Incentives can be provided by using money. But one constraint is<br />
that no real money should be involved for billing the resources in the science<br />
community. In this paper, a mechanism called Token Exchange System (TES) is<br />
proposed. Considering the requirements from the particle physics community this<br />
mechanism combines the advantages of reputation and payment mechanism in<br />
one coherent system to realize a fair allocation. It enables to build up a Grid<br />
infrastructure with an incentive mechanism which is scalable, incentivecompatible<br />
and does not require a face-to-face agreement like the current system.<br />
Keywords: Token Exchange System, incentive mechanism, Grid network<br />
1 Introduction<br />
In the e-Science community, Grid is an important infrastructure for the<br />
collaboration of scientific researchers to run calculations in a reasonable period of<br />
time. Combining distributed computing resources together and sharing these<br />
resources with others is demanded by research institutes. Especially researchers<br />
from particle physics are interested in sharing resources. They have a great<br />
resource demand for analyzing the data collected from the particle detector. The<br />
large amount of data comprising 4 million Gigabytes a year that is generated by<br />
the Large Hadron Collider experiment cannot be stored and processed by a single<br />
research institute (Berlich et al. 2005). Currently the GSI institute in Darmstadt,<br />
Germany, with its focus on particle and ionic physics produces about 1 Gigabyte<br />
data per second with its detectors. The various institutes take an interest in<br />
cooperative data analysis for saving precious time. Any institute can contribute his<br />
224
esources to the e-Science community and can in turn use the resources of others.<br />
The technical infrastructure allowing such coordinated resource sharing is<br />
provided by the Grid environment so as to realize an e-Collaboration among the<br />
researchers.<br />
This resource sharing approach, however, is problematic due to free-riding as<br />
known from Peer-to-Peer coupled with the prevailing competition in the research<br />
field (Adar, Huberman 2000). It turns out that even in Academia researchers are<br />
quite not willing to share resources or knowledge with other institutes without<br />
reciprocity. Furthermore, in the particle physics the resources are not utilized<br />
efficiently. Users tend to send too many jobs on others’ computing nodes. To<br />
avoid overconsumption, the sharing of resources has to be compensated to enable<br />
a fair transaction, which is not trivial in this context. This is not a “tragedy of the<br />
commons” problem (Hardin 1968, Ostrom 1990), since the resources belong to<br />
institutes or research projects, who are the owner. The need for more resources<br />
from other institutes is due to analysis of the large amount of data.<br />
In industry, compensation for shared resources is typically made by transferring<br />
money. Money is used to pay the counterpart a compensation for having access to<br />
the resources like applications (Application Service Providing), information or<br />
services. In principle, this payment system is also applicable to the scientific<br />
communities. However, the community faces major problems by implementing<br />
this system: researchers deny the usage of money for sharing resources for two<br />
main reasons. Firstly, researchers are not allowed to spend money without<br />
presenting valid reasons for their expenses to the sponsor. They always take care<br />
of the investment they make. An investment can lead to revenue. In some<br />
countries it is prohibited by law for institutes to make profit (e.g. Germany).<br />
Secondly, it is regarded unethical that those institutes with a deeper pocket are in a<br />
position to do better research than others. Thus, it is not surprising that researchers<br />
tend to prefer reputation mechanisms rather than money as incentive mechanisms.<br />
In reputation mechanisms the researchers do not rate the value of resources, but<br />
the behavior of the user. A disadvantage of reputation mechanism is the lack of<br />
incentive for motivating researchers to offer scarce resources. The participants do<br />
only the bare minimum to retain an acceptable reputation level that allows<br />
participating in the exchange process without restrictions. This means that there is<br />
no clear incentive to share scarce resources (i.e. high computational power).<br />
The main contribution of this paper is to give an overview of the requirements of a<br />
mechanism in the scientific community, compare former research work regarding<br />
the requirements and present a tailored mechanism for the e-Science Grid<br />
community. The incentive mechanism we propose is called Token Exchange<br />
System (TES). TES promises to achieve sharing resources in e-Science without<br />
money while retaining the beneficial coordination effect of money. TES is based<br />
on a reputation mechanism and a payment mechanism merged into one system.<br />
We use tokens instead of money for compensation. These tokens have a certain<br />
value. In contrast to real money or e-Cash, the tokens can be issued and<br />
distributed by the participants themselves. The value of the tokens is derived from<br />
the reputation of a participant and the number of issued tokens. Several<br />
restrictions are realized through a function calculating the token value to restrain<br />
flooding the whole system with their own tokens to manipulate the system. This<br />
approach is novel, as it combines existing payment and reputation mechanism into<br />
one coherent system. This way, transaction between institutes can be<br />
accomplished without burdening their financial endowment.<br />
2<br />
225
System analysis<br />
This paper is structured as follows: in section 2 requirements and the related work<br />
will be outlined. The model of TES is described in section 3 and the simulation<br />
results are shown in section 4. Section 5 concludes the paper.<br />
2 System analysis<br />
In a Grid network agents can have different behavioral characteristics. They can<br />
be altruistic, selfish or malicious (Yu, Singh 2003). These characteristics are not<br />
fixed per se. The agents can change the way they behave. For instance, Liebau et<br />
al (2005) define a normal agent who behaves selfishly, but to a certain extent<br />
altruistic. This type of agent only shares a few files for a certain period of time.<br />
One goal in a network with different agents is to maximize the overall welfare to<br />
enhance trust in the network. The good agents expect to be treated fairly, if<br />
resources are allocated in the Grid. Otherwise, they will not participate in the<br />
network any longer (in the long run). The idea behind TES is to provide a new<br />
incentive which cannot be fulfilled by current payment and reputation<br />
mechanisms. This will make the agent behave more cooperative and lead to an<br />
enhanced overall outcome. The incentive is set by combining both mechanisms<br />
into one coherent system. The payment mechanism is used to evaluate a<br />
commodity (goods or services). The reputation mechanism is necessary to<br />
evaluate the behavior of a user in a network. In TES the reputation has an impact<br />
on the budget of the users.<br />
2.1 Requirements<br />
The Grid usage for particle physics is one example from a variety of possible<br />
scenarios. Although TES is not limited to the application in particle physics Grid,<br />
we use it as a sample scenario. Different scenarios are defined by different<br />
requirements. The requirements describe the upcoming issues for exchanging<br />
resources in Grid. The most important requirement is the absence of real money in<br />
the Grid (requirement 1). In scientific environment the Grid computing resources<br />
are financed by research projects. These projects are led and controlled by<br />
national governments. One restriction by the national governments is not to earn<br />
money with the government-f<strong>und</strong>ed resources. Implementation of a non-monetary<br />
system is, therefore, essential. Instead of money, tokens are used which have other<br />
properties.<br />
Another property is to provide a compensation payment (requirement 2). Grid<br />
resources have to be consumed immediately. A reservation without consumption<br />
is a loss for the provider and other consumers who are waiting to complete their<br />
jobs. Without a payment mechanism (neither real money nor tokens), the provider<br />
will not receive compensation. As real money is not applicable, tokens can be<br />
used for paying compensation to the provider.<br />
Tokens as a payment mechanism can give agents an incentive to share their<br />
scarce resources with others (requirement 3). The particle physics Grid has<br />
several institutes in its projects. They have to share their resources with other<br />
project members, which is not always done consistently. An economic incentive<br />
can elicit the researcher or his institute to contribute high computational power or<br />
machines with specific configuration for certain jobs. These incentives have to be<br />
set by the economic model based on the token payment system. The model must<br />
226<br />
3
e designed in such a way that it enforces the participants to behave in a fair<br />
manner.<br />
Enforcements are implemented on both at the technical and at economical level.<br />
The implementation at both levels has to be complementary. The goal of<br />
enforcements is to achieve fairness and robustness for the participants using the<br />
accounting system (requirement 4). An exchange in the Grid is fair, if the<br />
requester gets the desired service and the provider receives adequate payment for<br />
the service. In case even if one of the rules is not obeyed, the accounting system<br />
has to be robust enough to identify the failure and punish the offending party<br />
(Dasgupta 2000; Papaioannou, Stamoulis 2005).<br />
A Grid network must be capable of accommodating the expanding number of<br />
users. Scalability is, therefore, necessary for technical and volume throughput<br />
expansion (requirement 5). A scalable economic model can be implemented into a<br />
technically well-defined system to guarantee a compo<strong>und</strong> dynamic and flexible<br />
system as emphasized by Forster (Foster et al. 2006).<br />
2.2 Related work<br />
In TES, every agent has his own tokens. For scalability reasons, the <strong>und</strong>erlying<br />
technical infrastructure is supposed to be a decentralized network as known from<br />
P2P. There has been a lot of research work done on implementing payment<br />
mechanisms for P2P and Grid systems (Barmouta, Buyya 2003; Dutta et al. 2003;<br />
Irwin et al. 2005). One example is Karma (Vishnumurthy, Chandrakumar, Sirer<br />
2003). Karma uses a uniform currency called Karma for payment. Every user in<br />
the network can earn and pay with Karma. The problem is the inflation and<br />
deflation of the currency. Furthermore, a reputation mechanism is not considered<br />
in Karma.<br />
A potential combination of payment and reputation was used in PeerTrust system<br />
(Xiong, Liu 2004). They introduced a Transaction Context Factor in their<br />
reputation mechanism to give a big transaction more weight than a small<br />
transaction. The aim is to avoid malicious attacks, where small transactions are<br />
done truthfully and fairly for building up a reputation and then to be dishonest on<br />
a big transaction to earn more profit while not substantially decreasing their own<br />
reputation. Hence, the amount of a transaction has an impact on the reputation.<br />
But the reputation has no impact on the user’s budget.<br />
Liebau et al. (2004) propose a mechanism called Token-based Accounting System<br />
(TbAS) for P2P systems. Tokens serve as a receipt for a transaction between two<br />
peers. The peer who receives a file hands over a token to the file provider. The<br />
provider exchanges the received token into one of his own. The exchange of the<br />
token is done via a public aggregation function, which is not defined. Moreover, a<br />
reputation mechanism is not integrated in the accounting system. In TbAS the<br />
token itself does not have a value. They assume that one token is fixed to a trading<br />
parameter like one token for one Megabyte or for one minute of service. Thus, the<br />
“price” will not vary for one Megabyte, i.e. depending on quality of service or<br />
reputation of provider.<br />
The solution to avoid double-spending in TbAS is done by using a serial number<br />
for each token. However, the double-spending check is done after the transaction.<br />
The problem here is that once the tokens have been exchanged and the reputation<br />
evaluated, there is no possibility for the provider to revoke the process or to get<br />
compensation. In TES the check is always done during the transaction before the<br />
4<br />
227
The model<br />
services are exchanged. There is no way that once spent tokens can be re-offered<br />
and distributed successfully.<br />
Moreton and Twigg (2003) presented a token-based protocol, where the token has<br />
a value and it has an impact on the exchange of resources and payment. The<br />
reputation is represented by the token-value. However, they did not examine how<br />
their calculation of the token value can set incentives for behaving truthfully. The<br />
calculation function is a one way track. Once the peer behaves badly and gets a<br />
bad evaluation, he cannot “clean” his reputation again. On one side it so<strong>und</strong>s as a<br />
good incentive to behave well. On the other side, if it is an attack where several<br />
peers rate him incorrectly, he has no chance to show his true face later on.<br />
3 The model<br />
In TbAS, each agent in the network is assigned to a group of trusted peers (Liebau<br />
et al. 2004). The tokens are issued by the trusted peers who are assumed to be<br />
trustworthy. These trusted peers manage the accounting information of the agent.<br />
An agent has a certain number of tokens. To consume resources, the agent has to<br />
send a request for the emission of the tokens for payment. The trusted peers check<br />
the amount of available tokens with the amount of requested tokens to avoid<br />
unauthorized token distribution and emit the tokens to the agent. With the<br />
received tokens the agent can consume resources from other agents. For every<br />
utilized service he has to hand over tokens to the service provider. The service<br />
provider redeems the tokens at his trusted peers. This will raise his “budget”. The<br />
foreign tokens from the consumer agent will be changed into the tokens of the<br />
service provider. The exchange rate is a publicly available function. The provider<br />
can use his earned tokens to acquire resources for himself.<br />
3.1 Exchanging the tokens in TES<br />
The function for calculating the exchange rate was not defined by Liebau et al.<br />
From the economic perspective, this is essential for an incentive-based resource<br />
exchange. Tokens can be changed at a predefined ratio, for instance 1:1. In our<br />
approach, the exchange rate is not based on a public function. Every price offer<br />
from the provider side is based on the number of offered tokens and is open to<br />
bargain. For instance, a provider can ask 5 tokens for 1 CPU hour. The requester<br />
can offer only 4 tokens as payment.<br />
The aim of TES is to set an incentive for an unselfish and a non-malicious<br />
behavior. Tokens are deployed to be used as a payment system. The participants<br />
in a network can pay another participant with the tokens. One characteristic of<br />
these tokens is individuality. Every participant has his own tokens, which can be<br />
created and distributed by himself. The agent is the token issuer. For instance,<br />
user A wants to receive a Grid service from user B. A offers his tokens to B. If B<br />
accepts the offer he will be “paid” in A’s Tokens. B as the owner of a certain<br />
number of tokens from A can decide whether he wants to redeem these tokens at<br />
A or to offer them to another participant C. After several transactions a<br />
participants possesses tokens from different users. The incentive of possessing<br />
others’ tokens is to increase the liquidity. Every token from agent A has a token<br />
value v A which can change over time. The token value is influenced by the<br />
reputation. A bad behavior of A has an impact on his token value. If B has tokens<br />
228<br />
5
from A, the value of these tokens will decrease and therefore his budget will<br />
decrease, too. In future, B will think twice about accepting tokens belonging to A.<br />
In the long run the tokens of users with a bad token value (and thus a bad<br />
reputation) will not be accepted widely. The bad users can still earn money by<br />
providing services. However, their token value is further an indicator of their<br />
reputation. Thus, a service provider with a bad reputation has an issue with<br />
canvassing customers. Both ways he has to increase his token value by behaving<br />
well. Three constraints have to be considered in providing this token system.<br />
Axiom 1: Since everybody has his own tokens, these tokens have to be<br />
limited in number (Moreton, Twigg 2003). Otherwise the users can<br />
mint their tokens arbitrarily and have infinite money (like minting<br />
Axiom 2:<br />
your own Cent-Coins).<br />
The reputation has an impact on the token value. An incentive for<br />
emitting tokens is necessary. Otherwise the agents will not<br />
distribute their tokens and their token value will not have an effect<br />
on others’ budget.<br />
Axiom 3: The number of distributed tokens has to be bo<strong>und</strong>ed (Axiom 1).<br />
However, a hard constraint will make it less flexible for the user.<br />
Any agent should be able to obtain a credit for an emergency. In<br />
this case, the agent has to pay some kind of interest.<br />
3.2 Calculating the token value<br />
The determination of the token value has to consider these axioms. To fulfill these<br />
axioms the token value function v x of an agent X is calculated as follows<br />
6<br />
v ( m<br />
x<br />
x<br />
, ncr<br />
x<br />
mx<br />
0 x<br />
x<br />
k<br />
l<br />
2<br />
) = max{0, ub − ncr − j * ( e * m − ) } (1)<br />
with v x ∈ [0, …, ub 0 - ncr x ], m x ∈ IN 0 . ub 0 is the upper bo<strong>und</strong> value which has to<br />
be determined for a system. The influence of the reputation value on the token<br />
value is expressed by the non-credibility rate ncr x with 0 as the best reputation<br />
value (similar to Papaioannou, Stamoulis 2005). The higher the value of ncr x the<br />
worse is the reputation. The calculation of the reputation value is only an example.<br />
Other reputation mechanisms can be included by calculating the ncr x value. For<br />
instance, by including a reputation mechanism with a valuation rep x ∈ [0, 1] the<br />
function does not need an upper bo<strong>und</strong>, as it is limited by the reputation value:<br />
v ( m<br />
x<br />
x<br />
, rep<br />
x<br />
m<br />
l<br />
2<br />
) = max{0, rep − j * ( e * m − k)<br />
}<br />
(2)<br />
In this paper we will consider the first function (1). Every distributed token has<br />
the current value v x which is adjusted dynamically based on the two variables m x<br />
and ncr x . The number of distributed tokens m x is necessary to define the liquidity<br />
of an agent X. By spending his own tokens (increasing m x ), the agent has less<br />
tokens for further transaction. He can earn foreign tokens or wait until others<br />
redeem his tokens to increase his budget. But the distribution of his tokens will<br />
raise his token value until a predefined maximum m* is reached.<br />
Furthermore, with the three parameters j, k and l the value function can be<br />
adjusted regarding the initial token value for m x = 0, the maximum token value m*<br />
and the gradient of the decrease of the token value (Figure 1). The gradient is<br />
229<br />
x<br />
x<br />
x
Evaluation<br />
crucial to determine how hard an agent is punished when distributing more than<br />
m* tokens. The option to provide more tokens than the maximum reflects<br />
axiom 3. The agent can provide more tokens, but he has to take his reputation loss<br />
into account (as payment of interest).<br />
The token value cannot be increased infinitely by distributing unlimited number of<br />
tokens (axiom 1). The limitation of the distributed tokens is not a hard constraint,<br />
but it reduces the reputation and the acceptance of the tokens and the transaction<br />
with a negatively rated agent will decrease:<br />
lim<br />
m →∞<br />
x<br />
v ( m<br />
x<br />
x<br />
, ncr<br />
x<br />
) =<br />
lim<br />
m →∞<br />
x<br />
max{0,<br />
l<br />
2<br />
ub0 − ncr − j * ( e * m − k)<br />
} = 0<br />
x<br />
m<br />
x<br />
x<br />
(2)<br />
The fulfillment of axiom 2 is presented <strong>und</strong>er the terms of a simple utility function<br />
of u x (v x ) = v x . For m x,1 , m x,2 ∈ [0, …, m*] with m x,1 < m x,2 the token value is as<br />
follows: v x (m x,1 , ncr x ) < v x (m x,2 , ncr x ). The agent X is always better off by<br />
increasing his token value until m x = m*.<br />
Figure 1 illustrates how the axioms are fulfilled by the token value function.<br />
Figure 1: Token value function<br />
4 Evaluation<br />
4.1 Simulation setup<br />
In this simulation we want to show the effect of reputation on the budget of the<br />
agents. There are two types of agents we name Pastors and Mavericks. An agent’s<br />
behavior can be good (providing full service or transferring payment) or bad<br />
(providing bad service or not transferring payment). As an agent will not always<br />
behave well or badly, a probability parameter decides how well or badly an agent<br />
might act. Thus, the Pastors might behave well with a probability of 90% and a<br />
Maverick might behave well with a probability of 10%. All agents of one type are<br />
symmetric. Different agents do not have different utility functions.<br />
The parameters are defined as follows: ub 0 = 1000; ncr x = 0; j = 0.05; k = 70;<br />
l = 110. The reputation mechanism is based on the credibility mechanism<br />
(Papaioannou, Stamoulis 2005). However, the punishment is not implemented<br />
since any utility function and learning algorithms are not considered yet. The<br />
7<br />
230
eputation mechanism is based on an asymmetric approach, where a bad rating<br />
increases the ncr-value by 10 and a good rating decrease the ncr-value by 2 with 0<br />
as the best value and with no upper bo<strong>und</strong> value.<br />
For the transaction in each case two randomly selected agents will be allocated<br />
together (25 allocations for 50 agents in one ro<strong>und</strong>). After the allocation the<br />
transaction begins and is divided into 4 phases. In phase 1 it has to be decided, if<br />
the agent behaves well or badly based on the probability. The second phase is<br />
about how the buyer and the seller are thinking about the reputation of the<br />
counterpart and if they want to settle a transaction with the partner. Pastors will<br />
not deal with agents who have a token value below a threshold of v 0 (·) = 500. In<br />
Phase 3 the offered price and the payment will be determined. The process of<br />
finding the right tokens (token selection process) is divided into three subsections.<br />
The succeeding subsection is only executed, if the preceding subsection was not<br />
successful:<br />
1. At first, the buyer tries to maximize his own reputation. Thus, he<br />
scrutinizes, whether he can distribute his tokens without decreasing his<br />
reputation.<br />
2. If he cannot distribute his own tokens (m x = m*), he will check, if he<br />
possesses enough tokens from the seller.<br />
3. In case he does not have any tokens from the seller, he tries to offer tokens<br />
from other agents.<br />
After this token selection process there can be an offer (in case there are enough<br />
tokens for payment) or an offer is not possible due to lack of enough tokens and<br />
the token value respectively. The price of an offer will be calculated by summing<br />
up the current token value of each token. Now the provider can decide whether he<br />
accepts the offer or not. The price varies randomly between 0 and 10,000. In<br />
Phase 4 the transfer of the tokens and the evaluation of the provider and consumer<br />
are settled. A well behaving agent will be rated well whereas a bad agent will be<br />
rated poorly. The <strong>und</strong>erlying assumption is that the agent will rate each other<br />
truthfully and there are no opportunistic or false ratings.<br />
4.2 Simulation scenarios and results<br />
We analyzed two scenarios for a better <strong>und</strong>erstanding of the effects of reputation<br />
on the budget. In the first scenario the Mavericks did not take their individual<br />
token value into account whereas it was considered in the second scenario. If their<br />
token value falls below the threshold value v 0 (·), the Mavericks will change their<br />
strategy and behave well until their token value is over the threshold value again.<br />
The results are compared to the combination of a payment and a reputation<br />
mechanism. The identical reputation mechanism was implemented in both<br />
systems. In the payment system the agents have an initial budget of 41488<br />
currency units. For comparison, this value is derived from the area below the<br />
curve of TES (assuming the agents have the best reputation ncr x = 0):<br />
m*<br />
46<br />
∫ =<br />
vx<br />
0<br />
( m<br />
x<br />
) ∂m<br />
x<br />
=<br />
46<br />
∫<br />
0<br />
1000 − 0.05 * ( e<br />
mx<br />
110<br />
* m<br />
x<br />
2<br />
− 70) ∂m<br />
x<br />
= 41488<br />
(3)<br />
The simulation was run over 20 generations with each 5000 ro<strong>und</strong>s for the two<br />
scenarios. Nevertheless, in the figures only the relevant first 1000 ro<strong>und</strong>s are<br />
visible. In the simple scenario the Mavericks do not care about their own<br />
8<br />
231
Evaluation<br />
reputation. The budget of the Pastors and Mavericks was used as a metric to<br />
compare both systems. As illustrated in Figure 2 the Mavericks earn a lot of<br />
tokens at the beginning where the Pastors trust the Mavericks. After 100 ro<strong>und</strong>s<br />
there are no more transactions between the two groups, because the reputation of<br />
the Mavericks is below a predefined threshold value. The Pastors do not trust the<br />
Mavericks. But Pastors still trust other Pastors. The Mavericks do not have the<br />
chance to trust Pastors, because there is no transaction due to their token value. In<br />
the monetary system, liquidity of the Mavericks is not decreasing whereas in TES<br />
the Mavericks can still “lose money”. This is the major incentive not to behave<br />
selfishly in TES. In the first scenario, the Mavericks only have a small loss,<br />
because their tokens are distributed among all agents. We assumed the worst case,<br />
where the Mavericks wait to behave selfishly until 90% of their tokens are<br />
emitted. It has to be considered that not all tokens of the Mavericks are distributed<br />
to Pastors. The Mavericks do not identify who the Pastors are. Thus, Mavericks<br />
can earn the tokens of Mavericks as well. After 450 ro<strong>und</strong>s the reputation of the<br />
Mavericks cannot get worse anymore. Thus, the budget of each group in TES<br />
remains almost stable. Because the Mavericks behave well and the Pastors behave<br />
poorly with a probability of 10%, the reputation affects the budget marginally.<br />
(a)<br />
Figure 2: Simulation results for payment mechanism (a) and TES (b) in<br />
scenario one (average budget of an agent)<br />
If the Mavericks are concerned about their reputation as in the second scenario,<br />
the difference between both systems is even more significant (Figure 3). Even<br />
though the Mavericks can gain nearly all the money in the system with time, in<br />
TES their budget is worse than the budget of the Pastors. In the first case they<br />
behave well to gain more money from others, if their reputation is below the<br />
threshold value. In TES, the Pastors can recover from the selfishness of the<br />
Mavericks and regain their budget. The reason is a constraint of the acceptance of<br />
the tokens. A Pastor who already has a good token value does not accept tokens<br />
with a low token value. Thus, in a transaction between two agents of each group<br />
the Maverick will accept the low valued token for payment and pay with high<br />
valued tokens, because they need resources. They have no alternative. In the long<br />
run the budget of the Mavericks will get even worse, since they only receive the<br />
worse tokens. The Mavericks are forced to behave well over a long period of time.<br />
In this simple simulation setup, we could present that the Mavericks in TES<br />
perform worse in both scenarios than in a common payment system.<br />
(b)<br />
232<br />
9
(a)<br />
Figure 3: Simulation results for payment mechanism (a) and TES (b) in<br />
scenario two (average budget of an agent)<br />
(b)<br />
5 Conclusion<br />
In our approach, the tokens represent the reputation value and the liquidity of an<br />
agent. A payment is done by exchanging tokens. As everybody has his individual<br />
tokens, his reputation has a direct effect on the token value and thus the liquidity.<br />
With a better performance and a good evaluation of others on his service and<br />
payment the agent can increase his reputation.<br />
TES enables a resource exchange in scientific Grids. Tokens are used for<br />
payments and do not represent real money (requirement 1). The payment via<br />
tokens allows the agents to pay compensation (requirement 2), if the resource of<br />
the provider was not utilized or misused by the consumer. Furthermore, TES<br />
provides an incentive to share scarce resources as the provider can earn more<br />
tokens, if he offers the resources that are in greater demand (requirement 3). From<br />
the economic perspective, the price for the resource (here the number of tokens<br />
with a high value offered) will rise. Fairness and robustness have to be considered<br />
in the reputation mechanism. An appropriate mechanism has to be chosen, which<br />
affords a fair evaluation of the agent’s reputation and an attack-resistant reputation<br />
system (requirement 4). Moreover, the scalability is given by the economically<br />
decentralized token system. A P2P-System like TbAS is a complementary<br />
technically decentralized system (requirement 5).<br />
The simulation proved that TES has a better performance than a common payment<br />
system combined with a reputation mechanism. Up to now in the simulation the<br />
payment in TES does not offer a combination of tokens from different agents<br />
within one transaction. This will lead to more transactions in a ro<strong>und</strong>. A<br />
sensitivity analysis with the initial settings of the parameters is required to adapt<br />
the function to different scenarios. Additionally, untruthful feedback of lying<br />
agents can worsen the system.<br />
Another aspect is to introduce an expiration date for the tokens. Firstly, this will<br />
prevent the hoarding of tokens by one user as the tokens will get invalid.<br />
Secondly, if agents join and leave the network the tokens should be taken out of<br />
the system to avoid inflation. An expiration date will automate this process.<br />
All in all, TES is an approach to set an incentive for good behavior and<br />
cooperation the scientific Grid network. Our token based system is tailored to the<br />
needs of the scientific community and is thus a promising starting point to<br />
overcome incentive problems in e-Science.<br />
10<br />
233
Conclusion<br />
References<br />
Adar, E.; Huberman, B. (2000): Free riding on Gnutella. First Monday, Vol.5<br />
No.10.<br />
Barmouta, A.; Buyya, R.(2003): Gridbank: A grid accounting services<br />
architecture (gasa) for distributed systems sharing and integration. In 17th<br />
IEEE International Parallel & Distributed Processing Symposium,<br />
Workshop on Internet Computing and E-Commerce, Nice, France.<br />
Berlich, R.; Kunze, M.; Schwarz, K. (2005): Grid computing in Europe: from<br />
research to deployment. In Proceedings of the 2005 Australasian workshop<br />
on Grid computing and e-research, Vol. 44.<br />
Dasgupta, P. (2000): Trust as a commodity. In Gambetta, D. (ed.): Trust: Making<br />
and Breaking Cooperative Relations, electronicedition.<br />
Dutta, D.; Goel, A.; Govindan, R.; Zhang, H. (2003): The design of a distributed<br />
rating scheme for peer-to-peer systems. In Proceedings of the first workshop<br />
on economics of Peer-to-Peer Systems, Berkeley (USA).<br />
Foster, I.; Kishimoto, H.; Savva, A.; Berry, D.; Grimshaw, A.; Horn, B.; Maciel,<br />
F.; Siebenlist, F.; Subramaniam, R.; Treadwell, J.; Von Reich, J.(2006): The<br />
Open Grid Services Architecture, Version 1.5.<br />
http://www.ggf.org/documents/GFD.80.pdf (accessed on 19Oct 2006).<br />
Hardin, G. (1968): The Tragedy of the Commons. In Science Magazine, Vol. 162<br />
No. 3859, pp. 1243-1248.<br />
Irwin, D.; Chase, J.; Grit, L.; Yumerefendi, A. (2005): Self-recharging virtual<br />
currency. In Proceeding of the 2005 ACM SIGCOMM, pp. 93-98.<br />
Liebau, N.; Darlagiannis, V.; Mauthe, A.; Steinmetz, R. (2005): Token-based Accounting<br />
for P2P-Systems. In: Proceeding of Kommunikation in Verteilten Systemen KiVS,<br />
pp.16-28.<br />
Moreton, T.; Twigg, A. (2003): Trading in Trust, Tokens and Stamps. In<br />
Proceedings of the first workshop on economics of Peer-to-Peer Systems,<br />
Berkeley (USA).<br />
Ostrom, E. (1990): Governing the Commons: The Evolution of <strong>Institut</strong>ions for<br />
Collective Action. Cambridge University Press.<br />
Papaioannou, T.G.; Stamoulis, G. (2005): An incentives' mechanism promoting truthful<br />
feedback. In peer-to-peer systems, Cluster Computing and the Grid, CCGrid 2005.<br />
IEEE International Symposium on, Vol.1, pp. 275-283.<br />
Vishnumurthy, V.; Chandrakumar, S.; Sirer, E. (2003): KARMA: A Secure<br />
Economic Framework for Peer-to-Peer Resource Sharing. In 1st Workshop<br />
on Economics of Peer-to-Peer Systems.<br />
Xiong, L.; Liu, L. (2004): PeerTrust: supporting reputation-based trust for peer-to-peer<br />
electronic communities. In IEEE Transactions on Knowledge and Data<br />
Engineering, Vol. 16 No. 7, pp. 843-857.<br />
Yu, B.; Singh, M.(2003): Incentive Mechanisms for Agent-Based Peer-to-Peer Systems.<br />
In Agents and Peer-to-Peer Computing, Second International Workshop, pp. 77-<br />
88.<br />
234<br />
11
WI – Schwerpunktaufsatz<br />
Distributed Ascending Proxy Auction<br />
A Cryptographic Approach<br />
The Authors<br />
Daniel Rolli<br />
Michael Conrad<br />
Dirk Neumann<br />
Christoph Sorge<br />
Daniel Rolli<br />
Dirk Neumann<br />
<strong>Institut</strong> für <strong>Informationswirtschaft</strong><br />
<strong>und</strong> -<strong>management</strong><br />
Universität Karlsruhe (TH)<br />
Englerstr. 14<br />
76131 Karlsruhe<br />
{rolli | neumann}@iw.uka.de<br />
Michael Conrad<br />
Christoph Sorge<br />
<strong>Institut</strong> für Telematik<br />
Universität Karlsruhe (TH)<br />
Zirkel 2<br />
76131 Karlsruhe<br />
{conrad | sorge}@tm.uka.de<br />
The Internet revolution has given rise to a<br />
widespread use of online auctions as a price<br />
discovery mechanism for selling different<br />
items (e.g. eBay). When referring to auctions,<br />
one traditionally has central or brokered<br />
auctions in mind with a designated<br />
entity that collects all the bids, determines<br />
the resulting allocation, and corresponding<br />
prices. Central auction platforms are typically<br />
of high reliability, where reliability refers<br />
to (1) correctness with respect to compliance<br />
to the auction rules, (2) continuous<br />
availability, and (3) security to a certain extent.<br />
Since the designated broker offers<br />
auctions to several groups repeatedly and<br />
enforces agreements, it can build up reputation<br />
that eventually may form trust in the<br />
platform.<br />
The central approach, however, also has<br />
several disadvantages. Since a designated<br />
central entity conducts all brokering services,<br />
it is vulnerable against Denial-of-Service<br />
(DoS) attacks and may suffer <strong>und</strong>er<br />
overload. Hence, scalability may be problematic,<br />
as the broker can only handle a<br />
certain number of participants at a time.<br />
Furthermore, the central broker typically<br />
charges fees from the participants in order<br />
to recover the investments in infrastructure<br />
and its maintenance.<br />
Peer-to-Peer (P2P) technologies have<br />
emerged that remove or at least alleviate<br />
these technical disadvantages of client-server<br />
architectures. In general, P2P networks<br />
distinguish themselves by providing scalable<br />
and robust applications. Combining<br />
P2P technologies with auctions is expected<br />
to bring forth auction systems that are scalable<br />
and robust on a technical level and retain<br />
the excellent allocation properties of<br />
auctions on an economic level.<br />
In so-called decentralized auctions, peers<br />
can be brokers and bidders at a time. When<br />
red<strong>und</strong>ant broker peers supersede the central<br />
broker, decentralized auctions are less<br />
vulnerable to DoS attacks. Scalability properties<br />
can be widely improved in comparison<br />
to the central approach, since every<br />
bidder contributes his share of additional<br />
infrastructure. Since peers collectively<br />
make for the total infrastructure, holding a<br />
decentralized auction can be free of (monetary)<br />
charge. Regarding the massive infrastructure<br />
of major auction providers like<br />
eBay, scalability may be less of an issue.<br />
But a need for decentralized auctions is<br />
further driven by the demand of P2P based<br />
user communities, who strive for disintermediation.<br />
Executive Summary<br />
1 Introduction<br />
The paper presents a distributed ascending proxy auction protocol. The protocol substitutes<br />
trust in a central (auctioneer) entity by ensuring correct conduct and privacy preservation. It<br />
finally enables the robust application of decentralized auctions in open environments like<br />
Peer-to-Peer systems.<br />
&<br />
&<br />
&<br />
Initial registration of bidders is not required; they can enter and bid spontaneously.<br />
Winner and second-price determination are robust to bidder and auctioneer failure or<br />
disconnection, as well as malicious peers.<br />
The privacy of the winning bidder’s highest bid is preserved – with a minor exception<br />
theoretically possible.<br />
Keywords: Internet Economics, Distributed Auctions, Privacy, Trust, Security Protocols, Peerto-Peer<br />
Systems<br />
235<br />
WIRTSCHAFTSINFORMATIK 48 (2006) 1, S. 7–15
8 Daniel Rolli, Michael Conrad, Dirk Neumann, Christoph Sorge<br />
In recent times, the benefits of decentralized<br />
auction platforms have been recognized<br />
in research [McAf00; HaSt05a;<br />
HaSt05b] as an alternative to central auction<br />
servers. Nonetheless, there are several<br />
obstacles to overcome before decentralized<br />
auctions will become an important pillar of<br />
e-business solutions: Due to the lack of a<br />
centralized platform, trust becomes an issue.<br />
Since any peer can set up an auction,<br />
there is no persistent instance, which assures<br />
reliability of the auction process. To<br />
introduce a threshold level of trust into decentralized<br />
auctions, appropriate security<br />
mechanisms become indispensable.<br />
In the following, we present the distributed<br />
ascending proxy auction as an example<br />
for an iterative decentralized auction. It<br />
is coupled with proxy agents that increase<br />
the price on behalf of the corresponding<br />
participants to beat the preceding highest<br />
price, as long as the bid that is given to the<br />
proxy agent is at least an increment higher<br />
than the current price. This allows bidders<br />
to have their bid value revealed step by step<br />
– and no earlier than necessary – without<br />
the need of constantly being online. This<br />
auction corresponds to the original eBay<br />
auction, while the latter is centralized. In<br />
the present paper, this decentralized auction<br />
is extended to a secure, decentralized<br />
mechanism, which can be easily conducted<br />
by peers over the internet.<br />
The remainder of this paper is structured<br />
as follows. The requirements of an ascending<br />
proxy auction are pointed out in section<br />
2. In section 3, related approaches in<br />
literature are reviewed with respect to these<br />
requirements. Subsequently, the protocol<br />
for the auction is presented in section 4.<br />
Section 5 evaluates the protocol in terms of<br />
its security and robustness. Lastly, in section<br />
6, the paper concludes with a summary<br />
and future work.<br />
2 Requirements<br />
For a secure asynchronous ascending<br />
proxy auction, the following main requirements<br />
have been identified. Since the auction<br />
is envisioned to be decentralized, its<br />
protocol is no longer managed by a central<br />
auctioneer but by multiple collaborating<br />
auctioneer nodes. Principally, it is possible<br />
that those auctioneer nodes are recruited<br />
from the bidders.<br />
In the context of ascending proxy auctions<br />
there arise four main requirements<br />
pertaining to the role of those multiple auctioneers<br />
(A1–A4).<br />
&<br />
&<br />
&<br />
&<br />
Distributed Result Determination (A1):<br />
In absence of a central entity, which can<br />
keep track of the bidding process, proxy<br />
bidding is difficult to realize. Since we<br />
want to avoid another trusted third<br />
party, we rely on several auctioneers and<br />
require that they jointly determine the<br />
price (which equals in our case the second<br />
highest bid) and the winner.<br />
Preservation of Secret Highest Bid (A2):<br />
The idea of a proxy auction is that bidders<br />
may submit their valuation truthfully.<br />
A bidder can be reluctant to do so,<br />
as he wants to keep his true valuation secret<br />
and may not trust the auction process.<br />
Thus, ensuring the preservation of<br />
the winning bidder’s highest bid within<br />
the distributed protocol is an imperative.<br />
Resistance to Bidder Exclusion (A3): The<br />
ascending proxy auction is intended as<br />
an open auction allowing all peers in the<br />
network access. Neither a single auctioneer<br />
nor “smaller” coalitions of auctioneers<br />
are able to exclude bidders by<br />
illegitimately dropping bids.<br />
Robustness to Paralysis Attacks (A4):<br />
For a smooth, error-free conduct of the<br />
auction, it is required that neither a single<br />
auctioneer nor “smaller” coalitions<br />
of auctioneers are able to block the protocol.<br />
From the auction format of an ascending<br />
proxy auction five additional requirements<br />
arise referring to the bidding process<br />
(B1–B5).<br />
&<br />
&<br />
&<br />
&<br />
Spontaneous Entry and Exit (B1): To account<br />
for the dynamic environment of<br />
the Internet, any bidder must be able to<br />
submit bids without prior registration at<br />
any time. This means, a bidder can spontaneously<br />
join and leave the auction.<br />
Asynchronous Bid Submission (B2): Bidders<br />
can independently submit bids at<br />
any time during the bidding phase. Only<br />
when submitting a bid, a bidder must be<br />
online. Under no circumstances are all<br />
bidders required to be online at a time.<br />
This requirement focuses on the monitoring<br />
costs, which should be as low as<br />
possible.<br />
Iterative Nature (B3): Bids can be submitted<br />
iteratively with the standing<br />
highest price revealed as information<br />
feedback.<br />
Single Individual Independent Key-Pair<br />
(B4): For participation in a distributed<br />
ascending proxy auction, each bidder<br />
needs only a single individual key-pair<br />
for encryption, which are not dependent<br />
on any other peer’s key-pair. These keys<br />
can be (re-)used for an arbitrary number<br />
236<br />
of auctions. Generating keys for any<br />
new auction creates high transaction<br />
costs. This requirement is closely related<br />
to B1 and emphasized, because many<br />
existing secure auctions are particularly<br />
costly in this respect.<br />
& Non-repudiation (B5): As with any<br />
other auction, the winning bidder may<br />
not deny the submission of the winning<br />
bid.<br />
An auction protocol that achieves those requirements<br />
is secure in a sense that there is<br />
no single point of failure, assured bid submission,<br />
protected bid acceptance, and a<br />
certain degree of robustness against paralysis<br />
attacks.<br />
3 Related Work<br />
In recent years, several cryptographic protocols<br />
have been proposed mainly for secure<br />
sealed-bid auctions. Among the first<br />
was the auction protocol developed by<br />
Franklin and Reiter [FrRe96] that was later<br />
criticized due to its inadequate privacy preservation<br />
after the auction is concluded.<br />
Privacy preservation means that neither<br />
bids nor bidder identities are revealed (A2).<br />
Since privacy preservation is a crucial requirement<br />
for sealed-bid auctions, and in<br />
particular for second-price sealed bid auctions<br />
– the so-called Vickrey auction – various<br />
new protocols have been suggested to<br />
remedy this shortcoming [Bran02].<br />
Among those newly suggested approaches<br />
only few can cope with auction<br />
formats that both comply with the privacy<br />
preservation requirement and embody<br />
other pricing rules than pay-your-bid. The<br />
auction protocols that do satisfy privacy<br />
preservation and, nonetheless, support<br />
more sophisticated pricing schemes, can be<br />
classified into two classes.<br />
& Distributed computation among multiple<br />
auctioneers: Typically, the agents submit<br />
shares of the bids to different auctioneers.<br />
The auctioneers jointly compute<br />
the auction price by using the techniques<br />
of secure multiparty function evaluation.<br />
The gist of those techniques is<br />
that the auctioneers can compute the<br />
auction price without knowing any bid<br />
&<br />
[KHAN00; Kiku01].<br />
Partially trusted third party: The second<br />
class of protocols requires the introduction<br />
of a third party, which controls the<br />
auctioneer. Crucial for those protocols<br />
is that the third party needs not be fully<br />
trusted but can draw on a weaker form<br />
of trust [Cach99; BaSt01].<br />
WIRTSCHAFTSINFORMATIK 48 (2006) 1, S. 7–15
Distributed Ascending Proxy Auction 9<br />
Both classes inherently impose several<br />
challenges and problems. In fact, the first<br />
class requires a threshold number of obedient<br />
auctioneers, i.e. auctioneers that correctly<br />
follow the protocol. Otherwise, collusion<br />
among malicious auctioneers might<br />
compromise the auction process. The second<br />
class of protocols requires some sort<br />
of third party that is trustworthy to a certain<br />
degree. Brandt introduced a third way<br />
to secure sealed-bid auctions. In fact, his algorithm<br />
requires all bidders in conjunction<br />
with the seller to resolve the auction. Because<br />
this approach preserves privacy completely,<br />
as only the winner and the corresponding<br />
bid is resolved, it is deemed<br />
superior to the other approaches for supporting<br />
sealed-bid auctions [Bran03].<br />
Iterative auctions (B3), however, have<br />
hardly been considered explicitly. In traditional<br />
iterative auction formats, the privacy<br />
requirements are not as demanding as in<br />
sealed formats. As bids are by nature open,<br />
there is no further need to secure them.<br />
For preventing malicious behavior (A3 &<br />
A4), it is often recommended to use the secure<br />
sealed-bid auction iteratively. The lessons<br />
learned from sealed-bid auctions<br />
would suggest to refer to the approach of<br />
Brandt [Bran03] for iterative auctions, as it<br />
allows anonymity and correctness (B5)<br />
along the bidding process.<br />
This recommendation is, however, premature<br />
and even faulty. Staging ro<strong>und</strong>-wise<br />
bidder-resolved auctions does not support<br />
a continuous bidding process (B2). This<br />
shortcoming could be excused, but, even<br />
worse, Brandt’s algorithm requires all participating<br />
bidders staying in the auction<br />
until it is resolved (B1<br />
B4). Once a bidder leaves the auction during<br />
one ro<strong>und</strong>, the auction cannot be resolved.<br />
For a dynamic network such as the<br />
Internet or any P2P network this requirement<br />
is way too strong. Another argument<br />
that challenges Brandt’s approach is concerned<br />
with the active involvement of the<br />
seller in the bidding process. Since the seller<br />
receives the auction outcome first and,<br />
subsequently, determines who has submitted<br />
the winning bid, he can leave the<br />
auction at will, before it is settled. In a oneshot<br />
auction with no outside option, this is<br />
rather unproblematic, as the seller can always<br />
impose a reservation price (e.g., the<br />
seller can also submit a bid specifying the<br />
reservation price) and all bids above this<br />
price contribute to positive utility. In reality,<br />
this explanation is not adequate, as it<br />
excludes alternative ways to sell the item.<br />
For instance, the seller could use the auction<br />
to get an idea about the bidders’ valuations,<br />
which can be exploited in subsequent<br />
re-negotiations with parts of the bidders.<br />
From a strategic point of view, bidders will<br />
presumably take this behavior into account<br />
and may bid less aggressively.<br />
Thus, Brandt’s approach cannot be used<br />
for supporting iterative auctions in more<br />
realistic settings – despite its appealing theoretical<br />
conduct. Other existing alternative<br />
cryptographic approaches hold even severer<br />
problems. Existing protocols using<br />
multiple auctioneers typically require all<br />
auctioneers to jointly determine the winning<br />
bid. As such, all auctioneers are<br />
employed at a time to resolve the auction,<br />
violating spontaneous exit and entry (B1)<br />
and asynchronous bidding (B2) [Kiku01;<br />
AbSu02]. Protocols using partially trusted<br />
third parties can principally represent continuous<br />
bidding processes with spontaneous<br />
exit and entry of bidders (as long as<br />
the third party is online). However, third<br />
party protocols typically have problems<br />
with non-repudiation and privacy preservation<br />
[BaSt01; LiAN02].<br />
Table 1 wraps up the current approaches<br />
proposed in literature. Note that most protocols<br />
are originally intended for sealed<br />
auctions. Nonetheless, the authors claim<br />
that these sealed protocols can be applied<br />
for iterative auctions as well. This paper examines<br />
the appropriateness of those protocols<br />
for iterative auctions only, not questioning<br />
their merits for sealed auctions.<br />
As Table 1 illustrates, in most protocols,<br />
possible bids (prices) have to be determined<br />
before the auction starts. The computation<br />
complexity usually depends on<br />
the number k of possible bids. With Asynchrony,<br />
we refer to the possibility of asynchronous<br />
bid submission; additionally, a<br />
protocol with more than one auctioneer<br />
should not require all auctioneers to be online<br />
at the same time. Verifiability means<br />
that it is possible to verify the correctness<br />
of the auction outcome.<br />
This paper attempts to provide a reasonable<br />
protocol for supporting secure iterative<br />
auctions. Drawing on asymmetric encryption<br />
the protocol can satisfy correctness<br />
in a way that winning bidder and<br />
corresponding price are accurately and<br />
transparently determined. Accordingly,<br />
bids can never be repudiated by any agent<br />
and the possibility for successful collusion<br />
among malicious auctioneers is extremely<br />
low. Furthermore, bidders can join and<br />
leave the auction spontaneously – there is<br />
no need for connecting to the auction other<br />
than submitting a bid.<br />
4 Protocol Description<br />
For the distributed ascending proxy auction,<br />
in essence we use multiple auctioneers<br />
in different groups to prevent at any time<br />
one single auctioneer from obtaining complete<br />
control over the auction process.<br />
As in most iterative auctions, bidders<br />
send their bids at arbitrary points in time<br />
to the auction mechanism. Any bidder can<br />
address an arbitrary auctioneer node in the<br />
collection of auctioneers. Since one of our<br />
main goals is the protection of the highest<br />
bid amount, we produce for any bid a bid<br />
chain BC, i.e. a sequence of strictly monotonously<br />
increasing bid steps B i where i<br />
denotes the order in the bid chain. The idea<br />
of the protocol is that only one bid step at<br />
a time can be revealed in the order the bid-<br />
Table 1<br />
Related Protocols (adapted from [Bran05])<br />
Protocol Price Auctioneers Number of ro<strong>und</strong>s Overall Computation Verifiability Non-Repudiation Asynchrony Spontaneous<br />
entry and exit<br />
[Bran03] ðM þ 1Þ st z þ 1 Oð1Þ OðkÞ Yes Yes No No<br />
[LiAN02] ðM þ 1Þ st 2 Oð1Þ OðkÞ Yes No No Yes<br />
[AbSu02] ðM þ 1Þ st 2 Oðlog kÞ OðkÞ Yes Yes No No<br />
[BaSt01] 1 st 1 Oð1Þ Oðzðlog kÞ z 1 Þ No No No Yes<br />
[Kiku01] ðM þ 1Þ st M > k Oðlog kÞ Oðz log kÞ Yes Yes No No<br />
WIRTSCHAFTSINFORMATIK 48 (2006) 1, S. 7–15<br />
237
10 Daniel Rolli, Michael Conrad, Dirk Neumann, Christoph Sorge<br />
der intends. To ensure this sequential revelation,<br />
the bidder encrypts the steps of the<br />
bid chain in cascades. Avoiding that one<br />
auctioneer reveals subsequent bid steps<br />
every step is encrypted with the private<br />
keys of different auctioneer groups. We use<br />
two public keys of different auctioneer<br />
groups to encrypt one bid step in order to<br />
increase privacy preservation. The additional<br />
auctioneer group of the second key<br />
also needs to be compromised for illegitimately<br />
revealing one bid step. The number<br />
of involved groups per bid step can be arbitrarily<br />
adjusted starting from one. Whenever<br />
an auctioneer group receives a bid<br />
chain during the auction process, one or<br />
several randomly chosen members of the<br />
group decrypt the next encrypted bid step,<br />
if their private group key is applicable and<br />
the preceding bid step in the chain is not<br />
higher than the standing highest bid<br />
(SHB). In case their key is not applicable,<br />
the group passes the bid chain on to the<br />
corresponding auctioneer group. A newly<br />
decrypted bid step higher than the SHB is<br />
broadcast to all other auctioneer groups as<br />
the new SHB. Each auctioneer group holding<br />
a partially decrypted bid chain only<br />
passes it on to the next group in the chain,<br />
if the bid step they decrypted does not beat<br />
the SHB. As a consequence, only the bid<br />
chain containing the SHB is typically kept<br />
partially encrypted and held by one auctioneer<br />
group, while all other bid chains<br />
are fully decrypted and discarded.<br />
At the end of the auction, the winner<br />
with the highest bid chain pays the price of<br />
the highest bid step in the second-highest<br />
and fully revealed bid chain.<br />
The description of the crucial details in<br />
the auction protocol is structured along<br />
three distinct parts: The initial setup describes<br />
the selection of the auctioneer<br />
groups and their order of appearance in the<br />
decryption process, the bid placing specifies<br />
the way bid chains are structured and<br />
submitted, and finally the bid processing<br />
describes how the auctioneers deal with<br />
bid chains for outcome determination.<br />
4.1 Initial Auction Setup<br />
At the outset of the auction, seller S has to<br />
create a document D describing the auction.<br />
This document includes the description<br />
of the items for sale and the fully specified<br />
auction procedure with all relevant<br />
parameters such as starting and ending rule<br />
or minimum price. Document D is used to<br />
determine the auctioneers and their groups<br />
that share the same public/private key pair.<br />
To ensure authenticity of document D,<br />
seller S has to digitally sign D using the private<br />
key of his public/private key pair.<br />
Starting from a desirably large number<br />
A total of auctioneers responsible for the<br />
protocol, the number of groups G number<br />
and their group size G size can be determined.<br />
The number of groups should grow<br />
faster, with an increasing A total , than the<br />
size of each group. As a design choice for<br />
our particular auction protocol, we use the<br />
following concave function for G size and<br />
the convex counterpart for G number :<br />
<br />
G size ¼ log x ðA p<br />
totalÞþ ffiffiffiffiffiffiffiffiffiffi <br />
A total<br />
<br />
2<br />
G number ¼<br />
A <br />
total<br />
G size<br />
In the formula for G size , we use the average<br />
of the base x logarithm and the square root<br />
of the total number of auctioneers A total to<br />
determine the size of any auctioneer group<br />
and take the integer part of the result. x is<br />
common knowledge throughout the auction<br />
and in our examples set to 2. For the<br />
number of groups G number we divide the<br />
total number of auctioneers A total by the<br />
group size G size and take the integer part.<br />
This approach provides many auctioneer<br />
groups relatively small in size. The particular<br />
choice of this group generating function<br />
is discussed in section 5.1.<br />
After having determined G size and<br />
G number the auctioneers themselves and<br />
their membership in groups have to be<br />
identified. This identification must be deterministic<br />
and traceable, but hard to anticipate.<br />
In particular, the seller himself<br />
must not be capable of influencing the selection<br />
process. Otherwise, he could select<br />
collaborating or malicious nodes in the set<br />
of auctioneers.<br />
Hence, we employ the description document<br />
D for the selection. One approach is<br />
the application of a cryptographic hash<br />
function. Such a function, which can be efficiently<br />
computed, maps arbitrary input<br />
onto a result of fixed length. No inverse<br />
can be efficiently calculated, nor can two<br />
different inputs leading to the same result<br />
be efficiently determined. Typical hash<br />
functions generate a 128-bit or 160-bit output<br />
value.<br />
Starting with the hash value of D, a chain<br />
of hash values can be calculated for the<br />
auctioneers.<br />
H r ðDÞ ¼HðHðHð...HðDÞÞÞÞ<br />
|fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}<br />
r þ 1 times<br />
Given an addressing mechanism which is<br />
capable of mapping such hash values to<br />
238<br />
distinct nodes, a chain of auctioneer nodes<br />
can be derived from the chain of hash values.<br />
Such mechanisms are commonly<br />
available in structured P2P networks like<br />
CAN [RFHK01] or Chord [SMKK01].<br />
The configuration of auctioneer addresses<br />
in groups is computed by using the following<br />
equation, where A ij is the j-th auctioneer<br />
in the i-th group.<br />
Address ðA ij Þ¼H i Gsize þ jðDÞ<br />
where 0 i < G number<br />
and 0 j < G size<br />
Having defined groups, the members of<br />
each group have to obtain a common<br />
group public/private key pair. For simplicity,<br />
we have one auctioneer per group<br />
generate a public/private key pair and distribute<br />
it to all other auctioneers in the<br />
group using the individual public key of<br />
each group member.<br />
After a successful conduct of this auction<br />
setup, which includes determination of the<br />
number G number and size G size of groups,<br />
selection of group members and groupbased<br />
key generation, we receive the following<br />
entities:<br />
G i ¼ group i of auctioneers,<br />
where 0 i < G number<br />
K i ¼ public key of group G i<br />
A ij ¼ auctioneer j of group i<br />
with public key K i ;<br />
where 0 j < G size<br />
4.2 Bid Placement<br />
Having set up the auction, potential buyers<br />
can join by placing bid chains. According<br />
to requirement A2 (refer to section 2), a<br />
bidder wants to hide his highest bid<br />
amount as long as possible, which may reflect<br />
his true valuation.<br />
If a bidder has a valuation v, he typically<br />
offers successively increasing bids in the<br />
form of several bid chains up to v. Our<br />
procedure makes it possible, but not necessary,<br />
for the bidder to observe the auction<br />
during its whole runtime in order to receive<br />
information feedback.<br />
The approach at hand gives the bidder<br />
the possibility to place with any bid chain a<br />
series of bid steps and ensures that these<br />
steps are only revealed if necessary for<br />
moving closer to or surpassing competing<br />
bidders. When a bidder joins an auction,<br />
he can retrace G number and G size by using<br />
the same algorithm as the seller, which is<br />
described in the previous section.<br />
WIRTSCHAFTSINFORMATIK 48 (2006) 1, S. 7–15
Distributed Ascending Proxy Auction 11<br />
According to his bidding strategy, a bidder<br />
typically submits a bid chain BC with a<br />
price p ¼ v, where p is the maximum value<br />
that is submitted with BC. For breaking<br />
down BC, he formulates n strictly monotonously<br />
increasing bid steps starting from<br />
a value p start < p (i.e. the SHB of the auction)<br />
ranging to p with certain increments.<br />
Each bid step in the chain is digitally<br />
signed by the bidder to ensure non-repudiation.<br />
The bidder can reproduce A ij , the auctioneer<br />
addresses with group-association,<br />
applying the same calculations as the seller.<br />
From this he arbitrarily chooses a hopping<br />
sequence (HS) of auctioneer groups and receives<br />
the respective public keys. Each single<br />
bid step is then encrypted using two<br />
different keys of the HS.<br />
HS i ¼ j<br />
where<br />
0 j < G number ^ HS 2k 6¼ HS 2k þ 1 j k 2 N 0<br />
The idea is that the first bid step, which is<br />
decrypted, contains all subsequent encrypted<br />
bid steps in cascades. So, the construction<br />
of any bid chain starts from the<br />
highest to the lowest bid step in reverse order<br />
to the decryption process.<br />
For asymmetric cryptographic operations<br />
described in the following paragraphs,<br />
S A ðxÞ denotes x, signed by A; K i is<br />
the public key of the i-th auctioneer group.<br />
The bidder starts by signing the highest<br />
bid step plus its index number, which results<br />
in S L ðB n , nÞ, and encrypts this with<br />
the public key K HS2n of the last auctioneer<br />
group in the HS. As shown in the following<br />
formula (1), this encrypted statement is<br />
denoted as I 1 . With the second last public<br />
key K HS2n 1<br />
in the HS the bidder generates<br />
I 2 – formula (2) – by encrypting the following:<br />
& The index HS 2n of the auctioneer group,<br />
which is responsible for decrypting I 1<br />
With this information the subsequent<br />
auctioneer group in the hopping sequence<br />
can be addressed (c.f. 4.1) in the<br />
decryption process.<br />
& The encrypted statement I 1 for achieving<br />
encryption in cascades.<br />
& The signed hash value S L ðHðB n Þ, nÞ of<br />
the previously encrypted bid step and its<br />
index number. As described in 4.3 this<br />
hash value is used by auctioneer group<br />
HS 2n to verify the correctness of their<br />
decryption operation.<br />
With the third public key K HS2n 2<br />
of the HS,<br />
the bidder incorporates the index of the responsible<br />
auctioneer group HS 2n 1 , the encrypted<br />
statement I 2 and the signed bid<br />
step plus index number S L ðB n 1 , n 1Þ.<br />
The encryption analogously continues<br />
until all bid steps are encrypted. Those<br />
statements with uneven indices contain a<br />
signed bid step with its index number<br />
S L ðB i , iÞ, whereas the ones with an even index<br />
contain a signed hash value of the bid<br />
step and its index number S L ðHðB i Þ, iÞ.<br />
To the very last statement, which is<br />
firstly generated to the abovementioned<br />
rule for even indices, the bidder adds HS 1 ,<br />
which determines the auctioneer group<br />
capable of decrypting the statement, and<br />
signs this.<br />
The following formulas illustrate the encryption<br />
of the bid chain from the highest<br />
to the lowest bid step. In the formulas, HS i<br />
stands for the i-th key of the hopping sequence,<br />
E i denotes the encryption with key<br />
K i , S L the signing by bidder L, B j the j-th<br />
bid step of the bid chain, HðB j Þ the hash<br />
value of bid step B j , and I k for the k-th encrypted<br />
statement of the bid chain BC,<br />
where k ¼ 2n i þ 1.<br />
E HS2n ðS L ðB n , nÞÞ ¼ I 1 ð1Þ<br />
E HS2n 1<br />
ðHS 2n , I 1 , S L ðHðB n Þ, nÞÞ ¼ I 2 ð2Þ<br />
E HS2n 2<br />
ðHS 2n 1 , I 2 , S L ðB n 1 , n 1ÞÞ ¼ I 3 ð3Þ<br />
E HS2n 3<br />
ðHS 2n 2 , I 3 , S L ðHðB n 1 Þ, n 1ÞÞ ¼ I 4 ð4Þ<br />
...:<br />
E HS4 ðHS 5 , I 2n 4 , S L ðB 2 ,2ÞÞ ¼ I 2n 3 ð5Þ<br />
E HS3 ðHS 4 , I 2n 3 , S L ðHðB 2 Þ,2ÞÞ ¼ I 2n 2 ð6Þ<br />
E HS2 ðHS 3 , I 2n 2 , S L ðB 1 ,1ÞÞ ¼ I 2n 1 ð7Þ<br />
S L ðHS 1 , E HS1 ðHS 2 , I 2n 1 , S L ðHðB 1 Þ,1ÞÞÞ ¼ I 2n ð8Þ<br />
Having constructed the encrypted bid chain,<br />
the bidder transmits I 2n to any auctioneer.<br />
If each bid chain consists of a fixed number<br />
of bid steps and the value p is reached<br />
after a random number of steps, which is<br />
significantly larger than 1, other bidders or<br />
auctioneers are not able to predict the value<br />
of p from the starting value and/or other<br />
parameters like the increments or number<br />
of bid steps. Accordingly, the number of<br />
bid steps has to be chosen sufficiently large.<br />
4.3 Bid Processing<br />
The decision tree in Figure 1 illustrates the<br />
exact steps each auctioneer group has to<br />
follow according to our ascending proxy<br />
auction protocol.<br />
So, according to Figure 1, an auctioneer<br />
receiving a newly submitted bid chain BC<br />
follows the decision tree in the following<br />
239<br />
way: he firstly sends BC in its initial form<br />
of I 2n to HS 1 – even if it happens that he<br />
belongs to this group. The auctioneer<br />
group HS 1 then checks, if HS 1 in I 2n addresses<br />
them. If it does address them and<br />
there is no decrypted lower bid step, a subset<br />
(randomly chosen) of all group members<br />
of HS 1 directly starts the first decryption<br />
step using key K HS1 ; otherwise they<br />
would pass BC on to the actual HS 1 . After<br />
the first successful decryption step, they<br />
check, if the revealed S L ðHðB 1 Þ,1Þ contains<br />
the index number 1. If it does not,<br />
they discard the bid chain. If it does, however,<br />
contain the right index number 1, the<br />
auctioneer group sends a bid confirmation<br />
for BC back to the bidder L. The group<br />
also conveys I 2n 1 ¼ E HS2 ð...Þ and<br />
S L ðHðB 1 Þ,1Þ to the auctioneer group with<br />
index HS 2 to continue decryption. The<br />
members of HS 2 check the index number 1<br />
in S L ðHðB 1 Þ,1Þ that they receive and presume<br />
that no bid step of BC has been decrypted,<br />
yet. Hence, a subset (randomly<br />
chosen) of all group members of HS 2 decrypts<br />
I 2n 1 to HS 3 , I 2n 2 , and S L ðB 1 ,1Þ.<br />
The decrypting members then propagate<br />
the signed bid step S L ðB 1 ,1Þ to all other<br />
group members. This enables all group<br />
members firstly to check if the received index<br />
number 1 in S L ðHðB 1 Þ,1Þ equals the<br />
index number in the newly revealed<br />
S L ðB 1 ,1Þ. If not, there is a decryption or<br />
integrity error. Secondly, they can verify<br />
valid decryption by hashing B 1 and comparing<br />
the result to HðB 1 Þ of S L ðHðB 1 Þ,1Þ.<br />
If those do not match, another auctioneer<br />
subset of the respective group has to repeat<br />
this whole process until validity is established<br />
or an integrity error must be presumed.<br />
Having successfully checked validity of<br />
the decrypted bid step B 1 , and B 1 beating<br />
the SHB, the auctioneer group broadcasts<br />
S L ðB 1 ,1Þ that includes B 1 as the SHB to all<br />
auctioneers.<br />
WIRTSCHAFTSINFORMATIK 48 (2006) 1, S. 7–15
12 Daniel Rolli, Michael Conrad, Dirk Neumann, Christoph Sorge<br />
5.2 Security Analysis<br />
a<br />
YES<br />
b) (no decrypted bid step B i-1 )<br />
b<br />
or (B i-1 < SHB)?<br />
YES NO<br />
2) decrypt bid step B i 2<br />
0<br />
1) pass on bid chain BC 1<br />
e) B i >B i-1 ?<br />
f) newly decrypted B i >SHB?<br />
3) publish B i as new SHB<br />
Figure 1<br />
Decision Tree<br />
If bid step B 1 does not beat the leading<br />
bid, the group transfers I 2n 2 , and S L ðB 1 ,1Þ<br />
to the next auctioneer group HS 3 in the<br />
hopping sequence. After successfully validating<br />
the index numbers and B 1 as well as<br />
processing of the statement I 2n 2 , the next<br />
auctioneer group HS 3 passes on the newly<br />
decrypted I 2n 3 and S L ðHðB 2 Þ,2Þ plus<br />
S L ðB 1 ,1Þ to the next auctioneer group HS 4 .<br />
After validating B 1 < SHB, HS 4 decrypts<br />
I 2n 3 and firstly checks, if the index<br />
number of their newly decrypted S L ðB 2 ,2Þ<br />
is exactly one higher than the index number<br />
they received in S L ðB 1 ,1Þ from HS 2 via<br />
HS 3 . If so, they check if B 2 is higher than<br />
B 1 . If not, they discard BC. IfB 2 > B 1 applies,<br />
they also check if B 2 is higher than<br />
SHB and, in case this applies, they publish<br />
S L ðB 2 ,2Þ as the new SHB. If B 2 > SHB<br />
does not apply, they pass on I 2n 4 and<br />
S L ðB 2 ,2Þ to HS 5 .<br />
This process is continued until the auction<br />
terminates. Obviously, all bid chains –<br />
except for the winning bidder’s – end up<br />
fully decrypted.<br />
5 Evaluation<br />
YES<br />
YES<br />
YES<br />
This section analyzes the protocol taking<br />
into account the requirements presented in<br />
section 2. After a discussion of the group<br />
generating functions, the security and<br />
threat analysis is performed, and the complexity<br />
analysis closes the evaluation.<br />
5.1 Group Generating Function<br />
In section 4.1 we presented a pair of group<br />
generating functions to define the number<br />
and size of auctioneer groups. The proposed<br />
auction protocol does not depend on<br />
a special function, as long as the applied<br />
3<br />
1<br />
YES<br />
f<br />
e<br />
NO<br />
1<br />
c<br />
NO<br />
d<br />
NO<br />
4<br />
NO<br />
0<br />
NO<br />
1<br />
a) bid chain intended for this group?<br />
1) pass on bid chain BC 1<br />
0) throw integrity error<br />
c) more encryption on B i ?<br />
d) i of B i =(jofreceivedB j )+1?<br />
0) throw integrity error<br />
4) discard BC 1<br />
1) pass on bid chain BC 1<br />
group generating function is common<br />
knowledge. The intuition, however, behind<br />
the chosen pair of functions is as follows:<br />
The size of auctioneer groups increases robustness<br />
and security as bigger groups can<br />
tolerate more (in absolute terms) disabled<br />
or even corrupted members. The number of<br />
auctioneer groups, however, is associated<br />
with a higher level of security, as more<br />
available auctioneer groups entail a higher<br />
number of encryption keys leaving any bidder<br />
with more options to generate a hopping<br />
sequence. Hence, we prefer the group<br />
number to increase faster than the size of<br />
groups. We regard log-based and squareroot-based<br />
functions for G size as stylized<br />
functions to motivate the trade-off between<br />
robustness and communication overhead.<br />
In essence, the log as well as the square root<br />
both have certain disadvantages.<br />
– Using the log-based function entails a rapid<br />
growth of G number but a slow growth<br />
of G size . For large auction setups with<br />
many auctioneers, many different groups<br />
of very small size are generated. This can<br />
be exploited by an attacker, who may relatively<br />
easily overtake a small group<br />
empowering him to partially block the<br />
auction.<br />
– In contrast, the square-root-based function<br />
generates almost identical results for<br />
G number and G size . Compared to the logbased<br />
function, it attains a slower growth<br />
of group numbers, but faster growth of<br />
group sizes. Hence, given a large number<br />
of auctioneers, groups with more group<br />
members are generated. As a side effect<br />
this creates higher communication and<br />
<strong>management</strong> costs within the groups,<br />
which may become a bottleneck.<br />
The proposed ad-hoc function combines<br />
both functions to a compromise, where the<br />
number of groups grows modestly faster<br />
than the group size.<br />
240<br />
In the security analysis, we refer to security<br />
risks within the auction protocol only.<br />
Those risks related to standard cryptographic<br />
algorithms like SHA1 or RSA as<br />
well as common threats (e.g. data flooding<br />
or virus infection) are beyond our scope.<br />
Two major risks associated with the distributed<br />
ascending proxy auction are<br />
1. premature revelation of the highest bid<br />
step(s) and<br />
2. exclusion of a targeted bidder from the<br />
auction.<br />
For each threat we provide an attack tree<br />
[Schn99], exhibiting the crucial attack possibilities<br />
and giving a short explanation of<br />
how such attacks are countered in the protocol.<br />
Figure 3 shows the attack tree of the premature<br />
revelation attack (refer to requirement<br />
A2).<br />
As Figure 2 shows, there are two approaches<br />
to illegitimately revealing all bid<br />
steps up to the highest one. Firstly, the attacker<br />
infiltrates the auctioneer groups to<br />
reveal the highest bid step. To do so, he<br />
firstly needs to infiltrate the auctioneer<br />
group holding the bid chain and forward it<br />
to the next auctioneer group in the hopping<br />
sequence. The attacker then has to infiltrate<br />
this auctioneer group to decrypt the next<br />
bid step. If this bid step is not the highest in<br />
the bid chain, the attacker has to repeat this<br />
process until the highest bid step is revealed.<br />
Secondly, the attacker attempts to retrieve<br />
the remaining bid chain from the<br />
auctioneer group currently holding it.<br />
Having succeeded, the attacker tries to decrypt<br />
the remaining bid steps by using illegitimately<br />
obtained private keys of the involved<br />
auctioneer groups.<br />
Accordingly, both attacks require the<br />
identification of the auctioneer group holding<br />
the bid chain and all subsequent groups<br />
in the hopping sequence. The addresses of<br />
the involved auctioneer groups are encoded<br />
within the bid chain and can only sequentially<br />
be obtained. The attacker can do<br />
nothing but guess the current auctioneer<br />
group with the bid chain and learn each<br />
following auctioneer group stepwise.<br />
Note that one possibility of revealing the<br />
overall highest bid step is immanent to the<br />
proposed protocol. If the highest bid step<br />
of the second-highest bid chain is higher<br />
than or equal to the second-highest bid<br />
step of the highest bid chain, the overall<br />
highest bid step is revealed at least to one<br />
auctioneer group.<br />
The second attack tree addresses excluding<br />
potential bidders from an auction.<br />
WIRTSCHAFTSINFORMATIK 48 (2006) 1, S. 7–15
Distributed Ascending Proxy Auction 13<br />
There are three independent possible attacks,<br />
which are shown in Figure 3.<br />
The first attack possibility is to suppress<br />
the submission of the bid chain. Hence, the<br />
attacker can lock out the bidder by flooding<br />
its peer and thus disconnecting it from<br />
the network. Such an attack can be easily<br />
detected.<br />
The second possible attack is forging the<br />
auction description. In this case, the attacker<br />
could provide wrong information about<br />
the auction, so that the bidder cannot join<br />
the auction and place a valid bid chain.<br />
However, this attack can easily be detected<br />
by any bidder. At first the signature of the<br />
auction description document generated by<br />
the seller becomes invalid. Furthermore the<br />
bidder does not retrieve a positive bid confirmation<br />
after placing an invalid bid.<br />
The third, more complex attack possibility<br />
is the dedicated prevention of processing<br />
bid steps of a specific bidder. To reach<br />
this, the attacker has to overtake at least<br />
one auctioneer group in every HS of the<br />
bidder as whole. This attack requires the<br />
knowledge of any HS of a bidder, and the<br />
bidder can even react by adapting the HS<br />
of a repeated bid chain. Accordingly, the<br />
attacker must either employ a lot of effort<br />
for maliciously obtaining and stepwise decrypting<br />
all bid chains of one bidder, or<br />
simply guess to take over several auctioneer<br />
groups randomly, hoping to compromise<br />
the right one by chance.<br />
Most of the attacks base on the presumption<br />
that the attacker succeeds in guessing<br />
the HS of a bidder. Figure 4 shows the<br />
probability of an attacker unfolding one<br />
next bid step of the standing highest bid<br />
chain depending on the number of infiltrated<br />
auctioneer groups in relation to the<br />
total group number. To reveal the next bid<br />
step an attacker has to compromise at minimum<br />
three auctioneers in different groups.<br />
The first group is needed to maliciously<br />
transfer the encrypted bid step to the next<br />
group, without a higher SHB present; the<br />
other two for actual decryption, since we<br />
assume that each bid step is encrypted with<br />
two different keys of the HS.<br />
We analyze auctions starting from 16 up<br />
to 1024 different auctioneers. Using our<br />
group generating functions described in<br />
section 4.1 results in 4 auctioneer groups<br />
for 16 auctioneers up to 48 auctioneer<br />
groups for 1024 auctioneers. We assume<br />
the attacker is able to compromise between<br />
3 and 15 auctioneers.<br />
Figure 4 and its findings imply that the<br />
probability of unfolding the next bid step<br />
and the exclusion of a certain bidder by an<br />
attacker decrease significantly with an ascending<br />
number of auctioneer groups.<br />
Even in case the attacker is able to compromise<br />
almost one third (15 of 48) of all auctioneer<br />
groups (from a total of 1024 auctioneers)<br />
he reaches a probability lower<br />
than 3 percent of unfolding even the next<br />
bid step of a specific bid chain. All values<br />
are calculated using a hypergeometric distribution:<br />
<br />
<br />
M N M<br />
x n x<br />
PðX ¼ xÞ ¼<br />
Forward bid chain<br />
to responsible<br />
auctioneer group<br />
Infiltrate auctioneer<br />
group holding<br />
remaining bid chain<br />
Know hopping<br />
sequence of<br />
dedicated bidder<br />
Figure 2<br />
Overtake first<br />
auctioneer group<br />
in bid chain<br />
Know hopping<br />
sequence of<br />
dedicated bidder<br />
Figure 3<br />
Reveal remaining<br />
bidding step(s) by<br />
malicious auctioneers<br />
<br />
N<br />
n<br />
<br />
AND<br />
Decrypt next bidding<br />
step of bid chain<br />
Infiltrate auctioneer<br />
group decrypting<br />
next bidding step<br />
Reveal highest<br />
bidding step(s)<br />
Attack Tree – Highest Bid Step Revelation<br />
Prevent submission of<br />
bid chain<br />
Lock out bidder<br />
from network<br />
Attack Tree – Bidder Exclusion<br />
241<br />
Exclude a<br />
dedicated bidder<br />
Forge auction<br />
description<br />
Infiltrate auctioneer<br />
group holding<br />
remaining bid chain<br />
Know hopping<br />
sequence of<br />
dedicated bidder<br />
Infiltrate next<br />
auctioneer group(s)<br />
in bid chain<br />
Know hopping<br />
sequence of<br />
dedicated bidder<br />
Reveal remaining<br />
bidding step(s) by<br />
attacker<br />
AND<br />
Prevent processing of<br />
next bidding step<br />
Variable N stands for the total number of<br />
auctioneer groups; n is the number of compromised<br />
groups. The variables M and x<br />
by assumption are 3, because an attacker<br />
needs all of the three involved auctioneer<br />
groups to reveal the next bidding step. The<br />
value PðX ¼ xÞ gives the probability of unfolding<br />
the next bid step of a specific bid<br />
chain.<br />
5.3 Complexity Analysis<br />
Decrypt remaining<br />
bidding step(s)<br />
Know private keys<br />
of auctioneer groups<br />
used in bid chain<br />
Know hopping<br />
sequence of<br />
dedicated bidder<br />
Overtake current<br />
auctioneer group<br />
in bid chain<br />
Know hopping<br />
sequence of<br />
dedicated bidder<br />
This section covers a basic complexity analysis<br />
of the presented auction protocol concerning<br />
calculation and communication<br />
complexity.<br />
For the analysis, let z be the number of<br />
bidders, A total the number of auctioneers, n<br />
the (average) number of bid steps with new<br />
information in a bid chain, e the number of<br />
groups whose keys are used to encrypt each<br />
bid step (two in our example), and b the<br />
average number of bid chains per bidder.<br />
Each bidder has to generate b n bid<br />
steps. As each bid step is encrypted e times,<br />
b n e encrypt operations have to be performed<br />
by a bidder, leading to a (computation)<br />
complexity in Oðb n eÞ for a bid-<br />
WIRTSCHAFTSINFORMATIK 48 (2006) 1, S. 7–15
14 Daniel Rolli, Michael Conrad, Dirk Neumann, Christoph Sorge<br />
Revealing Probability<br />
1<br />
0.8<br />
0.6<br />
0.4<br />
0.2<br />
Figure 4<br />
Decrypt Operations<br />
Probability of revealing the next Bidding Step of the Winner’s Bid Chain<br />
P (3 compromised auctioneers) (log+sqrt)<br />
P (6 compromised auctioneers) (log+sqrt)<br />
P (9 compromised auctioneers) (log+sqrt)<br />
P (12 compromised auctioneers) (log+sqrt)<br />
P (15 compromised auctioneers) (log+sqrt)<br />
0<br />
4 6 9 14 21 34 48<br />
Number of Auctioneer Groups<br />
400<br />
350<br />
300<br />
250<br />
200<br />
150<br />
100<br />
50<br />
Figure 5<br />
Probability of Maliciously Revealing Next Bid Step<br />
Decrypt Operations per Auctioneer<br />
(10 bidders with 3 bid chains each per auction, 100 steps per bid chain)<br />
Bid Chains per Auctioneer (one group per bidding step) (log)<br />
Bid Chains per Auctioneer (two groups per bidding step) (log)<br />
Bid Chains per Auctioneer (one group per bidding step) (sqrt)<br />
Bid Chains per Auctioneer (two groups per bidding step) (sqrt)<br />
Bid Chains per Auctioneer (one group per bidding step) (log+sqrt)<br />
Bid Chains per Auctioneer (two groups per bidding step) (log+sqrt)<br />
0<br />
16 32 64 128 256 512 1024<br />
Total Number of Auctioneers<br />
Decrypt Operations per Auctioneer<br />
der. On the other hand, all bid chains but<br />
one are completely decrypted, and those<br />
members of an auctioneer group who have<br />
not decrypted a bid step must check the<br />
proper decryption. As the latter only requires<br />
the use of a cryptographic hash<br />
function, the computation time for this<br />
check can be neglected in comparison to<br />
the asymmetric decryption operations necessary.<br />
All in all, this leads to a computation<br />
complexity in Oðz b n eÞ for the<br />
entirety of bidders and auctioneers<br />
throughout the whole auction process. The<br />
complexity for one auctioneer depends on<br />
the<br />
<br />
number of auctioneers and is in<br />
O<br />
z b n e <br />
.<br />
A total<br />
Communication complexity turns out<br />
similar, as each bid step is sent to all auctioneers<br />
of a group to allow for decryption<br />
and its verification. This results in<br />
an overall communication complexity in<br />
Oðz b n e G size Þ.<br />
Note that b and e are typically small<br />
constants and the suggested group generation<br />
functions also keep G size small.<br />
Figure 5 illustrates the number of decrypt<br />
operations for a single auctioneer.<br />
For this figure, we assume that on average<br />
10 bidders are involved in an auction<br />
(z ¼ 10). Furthermore, we set b ¼ 3 and<br />
n ¼ 100. Encryption with two group keys<br />
for each bid step (e ¼ 2) is compared to a<br />
single key (e ¼ 1). These parameters result<br />
in 3000 (10 3 100) bid steps with 3000 or<br />
6000 decrypt operations that must be processed<br />
by the auctioneers. As can easily be<br />
seen, complexity of the protocol is independent<br />
of the group generating functions:<br />
in all three cases the auctioneers have to<br />
overall decrypt the same number of bid<br />
steps. Only the number of encryptions per<br />
bid step impacts the complexity as two decryption<br />
steps per bid step double the<br />
costs. Generally, Figure 5 shows that the<br />
computation complexity of each auctioneer<br />
is significantly reduced with a growing<br />
number of auctioneers. With 256 auctioneers,<br />
e.g., each auctioneer has to perform<br />
less than 50 decrypt operations during the<br />
whole auction.<br />
Table 2<br />
Proposed Protocol<br />
Protocol Price Auctioneers Number of ro<strong>und</strong>s Overall Computation Verifiability Non-Repudiation Asynchrony Spontaneous<br />
entry and exit<br />
Proposed 2 nd M not applicable Oðz b n eÞ Yes Yes Yes Yes<br />
z – number of Bidders, M – constant, k – possible bids, b – bid chains per bidder (1 for the non-iterative auction protocols in table 1), n – bid steps per bid chain,<br />
e – number of encryption operations per bid step<br />
242<br />
WIRTSCHAFTSINFORMATIK 48 (2006) 1, S. 7–15
Distributed Ascending Proxy Auction 15<br />
6 Conclusion<br />
We present a secure mechanism for distributed<br />
ascending proxy auctions. It fosters<br />
privacy and overall correct conduct to a degree<br />
that enables the trusted application of<br />
an asynchronous, iterative, second-price<br />
auction in systems as open as P2P networks.<br />
The approach produces several desirable<br />
properties for the auction process: It eliminates<br />
the dependency on one single auctioneer,<br />
and the winning bidder can hide his true<br />
valuation, respectively his highest bid.<br />
Using an encrypted bid chain for bidding<br />
ensures that only a fraction of information<br />
is revealed to each auctioneer. With any new<br />
bid chain each bidder can freely decide,<br />
which auctioneer groups to trust. Robustness<br />
is achieved by forming groups of auctioneers,<br />
where even one group member<br />
suffices to decrypt a bid step. All requirements<br />
stated in section 2 are fulfilled.<br />
In contrast to previous distributed second-price<br />
auction mechanisms, our approach<br />
is suitable for iterative open-cry<br />
auctions. It is never necessary for all participants<br />
to be online at the same time, which<br />
is what makes other approaches highly vulnerable,<br />
as it allows the blocking of an auction<br />
fairly by attacking any participating<br />
node [cf. Bran03]. In our protocol, all a<br />
bidder needs to do is to convey his bid<br />
chains, whereas the auctioneer groups must<br />
be accessible during the whole auction process.<br />
However, one single obedient auction<br />
group member being online at a time suffices<br />
to conduct the auction with a sufficient<br />
standard of security.<br />
Table 2 summarizes all the properties of<br />
our ascending proxy auction. Obviously,<br />
the proposed protocol can remedy some of<br />
the shortcomings of the protocols presented<br />
in Table 1.<br />
The overall computation complexity<br />
does not depend on the number of auctioneers;<br />
hence, the computation per auctioneer<br />
decreases linearly in the number of auctioneers.<br />
The possible bids need not be<br />
specified in advance, nor does computation<br />
complexity depend on them. The values b<br />
and e, on which our protocol’s complexity<br />
depends, are small; n can be chosen arbitrarily.<br />
Note that the proposed auction protocol<br />
can only support 2 nd -price auctions.<br />
As a next step, a transfer of the presented<br />
notions to multi-unit and possibly combinatorial<br />
auctions is envisioned. For these<br />
challenging tasks, distributed algorithmic<br />
mechanism design and cryptography have<br />
to be combined to further realize the potential<br />
of decentralized auctions.<br />
Acknowledgement<br />
This research was partially f<strong>und</strong>ed by the<br />
German Federal Ministry of Education<br />
and Research (BMBF) in the SESAM-project<br />
as part of the research program Internet<br />
Economics. The authors are responsible<br />
for the content of this publication.<br />
References<br />
[AbSu02] Abe, M.; Suzuki, K.: Mþ1-st price auction<br />
using homomorphic encryption. 5th International<br />
Conference on Public Key Cryptography,<br />
2002.<br />
[BaSt01] Baudron, O.; Stern, J.: Non-interactive<br />
private auctions. 5th Annual Conference on Financial<br />
Cryptography, 2001.<br />
[Bran02] Brandt, F.: A verifiable, bidder-resolved<br />
auction protocol. AAMAS Workshop on Deception,<br />
Fraud and Trust in Agent Societies, 2002.<br />
[Bran03] Brandt, F.: Fully private auctions in a<br />
constant number of ro<strong>und</strong>s. 7th International<br />
Conference on Financial Cryptography (FC),<br />
2003.<br />
[Bran05] Brandt, F.: How To Obtain Full Privacy<br />
in Auctions. Preliminary Draft, February 9,<br />
2005.<br />
[Cach99] Cachin, C.: Efficient private bidding and<br />
auctions with an oblivious third party. 6th ACM<br />
Conference on Computer and Communications<br />
Security, 1999.<br />
[FoIM05] Fontoura, M.; Ionescu, M.; Minsky, N.:<br />
Decentralized Peer-to-Peer Auctions. In: Electronic<br />
Commerce Research Journal 5 (2005) 1, S.<br />
7–24.<br />
Abstract<br />
[FrRe96] Franklin, M. K.; Reiter, M. K.: The design<br />
and implementation of a secure auction service.<br />
In: IEEE Transactions on Software Engineering<br />
22 (1996), S. 302–312.<br />
[HaSt05a] Hausheer, D.; Stiller, B.: Decentralized<br />
Auction-based Pricing with PeerMart. 9th IFIP/<br />
IEEE International Symposium on Integrated<br />
Network Management (IM 2005), Nice, France,<br />
2005.<br />
[HaSt05b] Hausheer, D.; Stiller, B.: PeerMart: The<br />
Technology for a Distributed Auction-based<br />
Market for Peer-to-Peer Services. 40th IEEE International<br />
Conference on Communications<br />
(ICC 2005), Seoul, Korea, 2005.<br />
[Kiku01] Kikuchi, H.: (Mþ1)st-price auction protocol.<br />
5th International Conference on Financial<br />
Cryptography (FC), 2001.<br />
[KHAN00] Kikuchi, H.; Hotta, S.; Abe, K.; Nakanishi,<br />
S.: Resolving winner and winning bid<br />
without revealing privacy of bids. International<br />
Workshop on Next Generation Internet, 2000.<br />
[LiAN02] Lipmaa, H.; Asokan, N.; Niemi, V.: Secure<br />
Vickrey auctions without threshold trust.<br />
6th International Conference on Financial Cryptography<br />
(FC), 2002.<br />
[McAf00] McAfee, A.: The Napsterization of B2B.<br />
In: Harvard Business Review 78 (2000) 6, S. 18–<br />
9.<br />
[RFHK01] Ratnasamy, S.; Francis, P.; Handley, M.;<br />
Karp, R.; Shenker, S.: A Scalable Content-Addressable<br />
Network. ACM SIGCOMM, San Diego,<br />
CA, 2001.<br />
[Schn99] Schneier, B.: Attack Trees: Modeling Security<br />
Threats. In: Dr. Dobb’s Journal. December<br />
1999.<br />
[SMKK01] Stoica, I.; Morris, R.; Karger, D.; Kaashoek,<br />
M. F.; Balakrishnan, H.: Chord: A scalable<br />
peer-to-peer lookup service for internet applications.<br />
ACM SIGCOMM, San Diego, CA,<br />
2001.<br />
Distributed Ascending Proxy Auction – A Cryptographic Approach<br />
In recent years, auctions have become a very popular price discovery mechanism in the<br />
Internet. The common auction formats are typically centralized in nature. The peer-to-peer<br />
paradigm demands gearing up auctions for decentralized infrastructures. In this context, this<br />
paper proposes a distributed mechanism for ascending second-price auctions that relies on<br />
standard cryptographic algorithms. In essence, the auction protocol has the capability of<br />
preserving the privacy of the winning bidder’s true valuation.<br />
The auction protocol makes use of a high number of auctioneers divided into several<br />
groups. A bidder creates an encrypted chain of monotonously increasing bidding steps,<br />
where each bidding step can be decrypted by a different auctioneer group. This considerably<br />
reduces the attack and manipulation possibilities of malicious auctioneers. In addition,<br />
this secure approach does not require bidders to be online unless they are submitting their<br />
bid chain to the auctioneers.<br />
Keywords: Internet Economics, Distributed Auctions, Privacy, Trust, Security Protocols, Peerto-Peer<br />
Systems<br />
243<br />
WIRTSCHAFTSINFORMATIK 48 (2006) 1, S. 7–15
Technology Assessment and Comparison:<br />
The Case of Auction and E‐negotiation Systems<br />
Gregory E. Kersten 1 , Eva Chen 1 , Dirk Neumann 2 , Rustam Vahidov 1<br />
and Christof Weinhardt 2<br />
1 InterNeg Research Centre, Concordia University, Canada<br />
2 <strong>IISM</strong>, Universität Karlsruhe, Karlsruhe, Germany<br />
An e‐market system is a concrete implementation of a market institution; it embeds one or<br />
more exchange mechanisms. The mechanisms are—from the economic point of view—<br />
disembodied objects (models and procedures) which control access to and regulate execution<br />
of transactions. E‐market systems are also information systems which are information and<br />
communication technology artifacts. They interact with their users; have different features<br />
and tools for searching, processing and displaying information. This work puts forward an argument<br />
that the study of e‐markets must incorporate both the behavioural economic as well<br />
as the information systems perspectives. To this end the paper proposes a conceptual framework<br />
that integrates the two. This framework is used to formulate models, which incorporate<br />
the essential features of exchange mechanisms, as well as their implementations as IS artefacts.<br />
The focus of attention is on two classes of mechanisms, namely auctions and negotiations.<br />
They both may serve the same purpose and their various types have been embedded in<br />
many e‐market systems.<br />
244
2<br />
1. Introduction<br />
E‐markets are an important component of e‐business that bring demand and supply for goods<br />
and services into balance. They are the meeting places for buyers and sellers who use exchange<br />
mechanisms to conduct transactions. Exchange mechanisms are market institutions<br />
providing sets of rules, which specify the functioning of the market and permissible behaviour<br />
of its participants. Mechanisms include catalogues, where requests and offers are posted, negotiations,<br />
where the participants bargain over the conditions of an exchange, and auctions,<br />
where one side automates the process during which participants from the other side compete<br />
against each other.<br />
The design of exchange mechanisms is the concern of economists. The economists’ interest in<br />
market institutions or mechanisms emphasizes problems of social decision making of who<br />
shall get what resources. In their influential articles Hayek (1945) and Hurwicz (1973), formulated<br />
the resource allocation problem as an information aggregation problem. Because the<br />
valuation of different kinds of resources by individuals and organizations is typically private<br />
information, it is difficult for a planner to define the efficient allocation that secures the best<br />
use of resources by the society. To solve this social decision making problem, the economic<br />
planner needs to extract this dispersed knowledge of the individuals by setting the right incentives.<br />
Economists became involved with the engineering of market institutions, which involves the<br />
specification of rules to control the exchanges and trade execution, since the 1990’s (Roth<br />
2002). The field of market design provides a mathematical framework with which the outcomes<br />
of particular mechanisms can be computed as equilibrium, and the mechanisms could<br />
be designed to induce socially optimal outcomes (Maskin and Sjöström 2002).<br />
In recent years, the economists have adopted laboratory experiments to confront theory with<br />
empirical observations in their “toolbox” (Roth 1995; Smith 2003). The test‐beds used for these<br />
experiments abstract from the real world situations in order to isolate specific human decision‐making<br />
behaviours (Friedman and S<strong>und</strong>er 1994). Isolation is achieved by inducing the<br />
pre‐specified characteristics in participants of the experiments via a specially designed incentive<br />
system. This incentive‐based control entails that the participants’ innate characteristics<br />
can be assumed away (Smith 1994) and the market institutions can be implemented in a<br />
highly simplified form.<br />
An exchange mechanism can be used only if it is implemented in some and kind of an information<br />
system (IS). Behavioural economists make a deliberate effort to devoid the system of<br />
any features, which may contribute to its ease or difficulty of use or its appeal. This allows<br />
studying the relationships between different incentive structures and mechanism outcome,<br />
the selection of a revenue‐maximizing mechanism and separate market participants’ deviations<br />
from perfectly rational behaviour.<br />
Viewed from a different perspective, e‐markets are ISs deployed on the networks, which incorporate<br />
(among others) exchange mechanisms. As such, they are of interest to<br />
IS researchers, who study behaviours of users, with a particular interest in explaining perceived<br />
usefulness, fit with the task, intention to use, as well as other important IS adoption‐<br />
245
3<br />
related factors (e.g., Goodhue 1995; e.g., DeLone and McLean 2003; Iivari 2005). Beginning<br />
with the technology acceptance model (TAM) proposed by Davis (1989) to the unified theory<br />
of acceptance and use of technology (Venkatesh et al. 2003), to TAM/user satisfaction integrated<br />
model (Wixom and Todd 2005), users’ attitude and behavioural intention has been the<br />
predominant concern of IS researchers.<br />
In IS research four views of technology have been proposed: tool, perception, model and project<br />
(Orlikowski and Lacono 2001). In some of these views certain facets of a system are ignored<br />
or various components (and <strong>und</strong>erlying models) are made <strong>und</strong>istinguishable. Depending<br />
on the chosen perspective the studies of the technology use and performance focus on its<br />
selected characteristics or actions. Arguably, this occasionally vague or partial consideration<br />
of technology can be contrasted with a key purpose of IS studies which is technology performance<br />
leading to its individual and organizational success (DeLone and McLean 2003; Garrity et<br />
al. 2005) for which there have been “nearly as many measures as there are studies” (DeLone<br />
and McLean 1992, p. 61). Knowledge of many rejected, <strong>und</strong>erperforming and misaligned systems<br />
led IS researchers to seek answers to such questions as: Will this system be adopted? Do<br />
users have positive attitude about using the system? Do they intend to use it?<br />
Many studies of technology use, adoption and performance consider the overall perception of<br />
the experience with hardware, software and rules of their use in performing a particular task<br />
(DeSanctis et al. 1994; Goodhue et al. 1995; Zigurs et al. 1998). If—as it is typically the case—<br />
several technologies are b<strong>und</strong>led together in order to <strong>und</strong>ertake the task, then such an approach<br />
may not give useful results for the developers of a particular technology and/or the<br />
technology adopting organization. Similarly, practical value of the assessment of software performance<br />
that produces similar results but uses different embedded mechanisms is questionable<br />
if these mechanisms are not explicitly distinguished.<br />
Over the last 30 years ISs and other information and communication technologies (ICTs) <strong>und</strong>erwent<br />
dramatic changes. While the old systems were single‐purposed, the new systems allow<br />
to dynamically binding different low‐level services into a service requested by the users<br />
(Turner et al. 2003). The old ISs were difficult to modify; the new ISs have components with<br />
partial and provisional connections which can be easily changed in order to adapt the system<br />
to the user’s preferences and the task at‐hand (Orlikowski and Lacono 2001). The old ISs were<br />
difficult to use; the new ones share common graphical interface features and require little<br />
training. These and other changes led to ICTs being increasingly ubiquitous and entering all<br />
dimensions of work and leisure. Often it is no more a question of an IS being used but of the<br />
selection of the system or the compositions of ITC components and services so that the usersystem<br />
interaction yields better results than if another choice was made. We therefore argue<br />
that we need to concentrate on seeking comprehensive answers to the following question as:<br />
What systems features, components and services are particularly useful for the given user, task,<br />
and context?<br />
In order to study these issues we choose e‐market systems because of their exemplary ubiquity,<br />
ease of use and customization. They are well‐known information systems which embed<br />
different mechanism and services. Some of them may be complementary (e.g., enhancing<br />
shoppers experience), others may be competitive giving gro<strong>und</strong>s to the question formulated<br />
in the preceding paragraph.<br />
246
4<br />
The purpose of this work is to propose a research framework and associated models that allow<br />
studying the performance of ISs. For this purpose we define IS technology as a system which is<br />
designed in order to provide services to its users so that they can achieve their goals by interacting<br />
with the system. Note that the system may be a single software program or several programs;<br />
in the later case the system may be designed at the run time. Note also that we take a<br />
narrow view of the system as being designed with a given purpose so that we do not consider<br />
servers, modems and PCs as an e‐market system if they are also used for other reasons.<br />
Typically, an IS has several ICT components which comprise a complex technological solution<br />
and the services they provide. These components include user and software interfaces, data<br />
storage and access, configuration and <strong>management</strong>, and computational models. What components<br />
are used and in what way they are configured determines the system’s services.<br />
Therefore, to assess the system usefulness and performance we need to be able to assess its<br />
components or artefacts of a technology separately rather than jointly as a black‐box. To obtain<br />
a comprehensive assessment we also need to consider all four views of technology (tool,<br />
perception, model and project). In the case of e‐market systems this leads us to draw from behavioural<br />
economics and IS research so that:<br />
1. Performance is measured at level of individual user, at the group level (all users involved<br />
in a single transaction), role‐level (e.g., buyers and sellers), and at the marketlevel;<br />
and<br />
2. Both behavioural (beliefs and attitudes) and economic (costs, returns, profits) measures<br />
of process inputs and outputs are included.<br />
This work proposes a conceptual framework that integrates the behavioural economic and the<br />
information systems approaches to studying socio‐technical processes. This framework is used<br />
to formulate a model, which incorporates the essential features of exchange mechanisms, as<br />
well as their implementations as IS artefacts. The focus of attention is on two classes of<br />
mechanisms: auctions and negotiations because they both may serve the same purpose and<br />
their various types have been embedded in very similar and in very distinct e‐market systems<br />
(Neumann et al. 2003). In addition to mechanism and system we also include three additional<br />
constructs in the proposed research model: task, individual, and environment. The interaction<br />
of these constructs results in the behavioural and in economic outcomes. Outcomes are used<br />
to describe the transaction process and its results.<br />
The interactions of the individuals and systems and mechanisms is the process, which is not<br />
explicitly modelled in IS studies that typically use either variance or process models (Seddon<br />
et al. 1999; Iivari 2005; Kim et al. 2006). One exception is the DeLone‐MceLean IS success<br />
model (DeLone and McLean 1992)) in which system use is one of the modelled constructs.<br />
The framework proposed in this work, called TIMES proposes to integrate variance and process<br />
models, that is, explicitly and in‐dept consider process as it is a category of concepts which<br />
explain causal relationships between input (independent variables) and output (dependent<br />
variables). “One significant way to improve the robustness of process explanations in variance<br />
theories is to explicitly observe the process argument that is assumed to explain why an independent<br />
variable causes a dependent variable. To do so requires opening the proverbial 'black<br />
box' between inputs and outcomes, and to directly observe process.” (van de Ven 1992, p. 170).<br />
247
5<br />
The TIMES framework provides a basis for the construction of models that allow studying information<br />
systems from several perspectives, including:<br />
1. Analysis of the use of and assessment of the efficacy of components and exchange<br />
mechanisms;<br />
2. Assessment of the impact of IS services and economic mechanisms on human interactions,<br />
their behaviours and behavioural and economic assessments;<br />
3. Study of the behaviours of the individuals, groups of individuals participating in a single<br />
transaction, and individuals participating in the separate roles (e.g., buyers and<br />
sellers); and<br />
4. Development of prescriptions for advocating exchange mechanisms depending on the<br />
participants and their objectives, context and the exchanged object(s).<br />
The remainder of the paper is structured as follows. In Section 2 two exchange mechanisms:<br />
auctions and negotiations are discussed and their hybrid forms are introduced. Sections 3 and<br />
4 review the two perspectives on e‐market design, namely those of economics and IS. Section 5<br />
proposes an integrated framework for assessing e‐market systems that incorporates the two<br />
perspectives. Section 6 utilizes the framework to describe a concrete research model, which is<br />
tested through the comparisons of mechanism and system in a laboratory experiment. Section<br />
7 concludes with a summary and an outlook on future work.<br />
2. Auctions and negotiations<br />
A market mechanism is a set of rules governing the transaction process; it defines how the<br />
market participants reach their agreements. These rules can be <strong>und</strong>erstood as mechanisms,<br />
because they make behaviour more orderly and thus more predictable. The space of potential<br />
market mechanism is large, since there are many ways to define the rules which can be imposed<br />
on the exchange process. While the commonly used catalogue‐based exchanges is one<br />
example of an institution, of greater interest to the researchers are classes of mechanisms,<br />
which permit richer dynamics and more complex behaviour on part of the market participants,<br />
such as auctions and negotiations (Wolfstetter 2000).<br />
2.1 Auctions<br />
Auctions are defined by an explicit set of rules which determine resource allocation and prices<br />
on the basis of the bids made by the market participants (McAfee and McMillan 1987). The<br />
four following characteristics differentiate auctions from other exchange mechanisms:<br />
5. Auction rules are explicit and known to bidder prior to the auction. Therefore, rules<br />
cannot be modified during the auction.<br />
6. The rules completely describe the mechanisms allowing for the determination one or<br />
more winners solely based on the bids. Auctioneers or any other party have no discretion<br />
in winner choice.<br />
7. Rules typically include: (a) bidding rules stating how bids can be formulated and when<br />
they can be submitted; (b) allocation rules describing who gets what on the basis of<br />
248
6<br />
submitted bids; and (c) pricing rules stating the corresponding prices the bidders have<br />
to pay (Reiter 1977); and<br />
8. Auction rules allow mapping of bids on a single dimension – the price. They focus on<br />
prices to achieve either an efficient allocation or revenue maximization.<br />
The adequateness of utilizing the price mechanism has been conjectured by Hayek (1945) and<br />
has been demonstrated <strong>und</strong>er certain assumptions by Hurwicz (1973). If those assumptions<br />
are violated (e.g. there are externalities in preferences), then richer message spaces, e.g. inclusion<br />
of delivery schedules and quality in addition to prices, may become necessary (Mount<br />
and Reiter 1974).<br />
Auctions can be either single‐sided, where one seller auctioneers off goods to a number of<br />
bidders, or double‐sided, where competition is employed on both sides of a market. Auctions<br />
have been widely used in practice to allocate products as varied as securities, offshore mineral<br />
rights and emission certificates (Wolfstetter 1995; Lucking‐Reiley 2000).<br />
Modern auction theory has adapted to these needs of richer message spaces by including<br />
other than price attributes (Bichler 2000). Multi‐attribute auctions and combinatorial auctions<br />
have been designed but have not been used because of their complexity in formulating<br />
strategies, eliciting valuations for combinatorial goods, solving the winner determination<br />
problem and communicating bids (Parkes 2001). Most auction formats still rely on a single<br />
price dimension.<br />
Auction mechanisms dominate the economic view. In fact, the field of market design is almost<br />
exclusively focussed on the theoretical and applied auction theory. This focus has a major<br />
ramification upon practical market design: the decision whether to use auctions or other exchange<br />
mechanisms favours auctions.<br />
2.2 Negotiations<br />
Negotiation is a rich and ill‐defined family of processes, used for exchanging goods or services<br />
among buyers and sellers, and resolving inter‐personal and inter‐organizational conflicts. Negotiation<br />
is an iterative communication and decision making process between two or more<br />
participants who cannot achieve their objectives through unilateral actions. In addition, negotiation<br />
involves exchange of information comprising offers, counter‐offers and arguments<br />
with the purpose of finding a consensus (Bichler et al. 2003).<br />
This notion of negotiations includes negotiation as an exchange mechanism, mediation, voting<br />
and other forms of conflict resolution. Restricting the attention to trade negotiations only,<br />
in which the participating agents engage in order to reach an agreement about an allocation<br />
(which goods are exchanged to whom at what compensation), simplifies negotiations to bilateral<br />
and multi‐bilateral encounters in which two roles are distinguished: buyers and sellers.<br />
The peculiarity of negotiation as an exchange mechanism is its degree of freedom in the level<br />
of structuring the process and the scope of exchanged information. Negotiations differ in the<br />
degree of their structuredness, possibility of modification, and participation rules (Bichler,<br />
Kersten et al. 2003; Ströbel and Weinhardt 2003). The process of interactions may be not prescribed<br />
a priori as it is the case with many face‐to‐face negotiations; and the rules may be<br />
249
7<br />
known only implicitly, for example, based on tradition. Negotiations typically allow for the<br />
modification of the protocol, but this characteristic may be limited in the case of e‐<br />
negotiations. Also, the participation may be limited or determined by one part allowing another<br />
to participate or not.<br />
The above indicates the degree of flexibility in the design of negotiation mechanisms and this<br />
is the key characteristic that differentiates them from auctions. A particular instance of multibilateral<br />
negotiation can be indistinguishable from an auction, with offers having the same<br />
format as bids and forbidden exchange of arguments and other messages. The difference is<br />
that every negotiation allows for a change of its protocol while this is not allowed in auctions;<br />
instead one auction would need to be cancelled and a new one initiated.<br />
Negotiations mechanisms have been studied in economics and game theory. For example, bilateral<br />
bargaining models were used to investigate how the surplus of a transaction is allocated<br />
between two parties (Nash 1954). Recently, in response to the evolution of e‐<br />
marketplaces, two streams of bargaining models have been studied in the field of economics<br />
in order to explain market behaviour. In the first stream of studies several bilateral bargaining<br />
processes between buyers and sellers have been investigated in order to determine the properties<br />
of the aggregate market, which results from the bilateral relationships (De Fraja and<br />
Sakovics 2001; Satterthwaite and Shneyerov 2003). One extension of this mechanism is a double‐multi‐bilateral<br />
negotiation in which one party (e.g. a buyer) negotiates with several sellers<br />
and each seller negotiates with several buyers. Such negotiations are similar to double‐side<br />
auctions, but buyers are engaged in bilateral negotiations with many sellers and sellers are<br />
negotiating with many buyers (Satterthwaite and Shneyerov 2006). The second stream of empirical<br />
research extends bilateral bargaining models to multilateral bargaining (Thomas and<br />
Wilson 2002). The latter are similar to single‐sided auctions in that they incorporate explicit<br />
competition among all participants representing one side.<br />
2.3 Hybrid forms<br />
Auctions and negotiations are considered here as two distinct classes of market institutions.<br />
The popularity of on‐line auctions led some researchers to state that internet negotiations are<br />
auctions (Kumar and S.I.F. 1999), thus indicating that there is no need for negotiation mechanisms,<br />
while others stated that e‐negotiations should be replaced with on‐line auctions (Segev<br />
and Beam 1999). These statements, contrasted with beliefs that negotiations differ from auctions<br />
in essence, may have been behind efforts to design mechanisms, which have certain<br />
properties derived from both classes of mechanisms.<br />
Some of the combined or hybrid mechanisms are well known, albeit little studied. For instance,<br />
the mechanism used in hiring normally involves first a mechanism closely resembling<br />
sealed bid auction with candidates submitting their resumes and other supporting documents.<br />
In the second stage, a few of the “auction winners” are invited for interviews involving<br />
negotiations over salary and benefits. For e‐markets, Teich, Wallenius et al. (2001) propose<br />
NegotiAuction, a mechanism generalizing the process of auction followed by negotiation<br />
(Leskela et al. 2006). A reverse process, in which multilateral negotiations are followed—if the<br />
parties agree to do so—by an auction is suggested by Shakun (2005). These exchange mechanisms<br />
are said to have a hybrid format.<br />
250
8<br />
Other types of hybrid formats occur in practice. For example, e‐Bay auctions allow for inclusion<br />
of a “Buy‐it now” option that is typical for a catalogue, while Alibaba (http://alibaba.com)<br />
combines catalogues with free‐text communication between the buyers and sellers.<br />
3. Experimental and field studies<br />
We are interested in the assessment and comparison of e‐market systems which includes both<br />
economic and IS perspectives. Economic studies focus on market mechanisms, their functioning<br />
and efficiency. The focus of IS studies is on the systems in which the mechanisms are embedded<br />
with—in most cases—little concern for the mechanisms themselves. This indicates<br />
that the two areas of research are complementary as we attempt to show through the discussion<br />
of the theoretical and experimental studies.<br />
3.1 Auctions and negotiations in economics<br />
Research by the economists into auction mechanisms can be fo<strong>und</strong> in several theoretic studies<br />
on single‐ and multi‐attribute auctions. Interestingly, the empirical research provides no<br />
clear answer as to the superiority of one mechanism with respect to another. These results are<br />
one of the motivations for the model we present in Section 5, which could be used in more<br />
comprehensive comparative studies of both auctions and negotiations.<br />
3.1.1 Theoretical comparisons<br />
Bulow and Klemperer (1996) have shown in one of the first formal comparative studies that<br />
simple English auction with N + 1 participating bidders always yielded higher revenue than a<br />
negotiation with N participants. In essence, Bulow and Klemperer did not analyze a specific<br />
negotiation protocol, instead they referred to an “optimal” mechanism. Such mechanism denotes<br />
an abstract selling scheme, which is designed so as to maximize revenue of the seller.<br />
The implication is that a seller should “devote resources to expanding the market than to collecting<br />
the information and making the calculations required to figure out the best mechanism.”<br />
(op. cit., p. 180).<br />
This result does, however, hold only <strong>und</strong>er fairly strong assumptions, e.g. attribute preference<br />
independence. For affiliated preferences for example, the Bulow‐Klemperer (1996) result also<br />
holds when the seller’s choice of the negotiation mechanism is restricted. If other assumptions<br />
are relaxed (regularity, symmetry, etc.) the result may not be valid any more.<br />
Kirkegaard (2004) showed that a seller‐offer bargaining game is more advantageous than an<br />
English auction when demand is discrete, i.e., it has finite space, and the buyers are patient. In<br />
those cases, sellers prefer a bargaining protocol over an English auction. Furthermore, Kirkegaard<br />
(op. cit.) showed that when demand is continuous, an English auction can be improved<br />
by some kind of pre‐negotiation.<br />
The models proposed by Bulow and Klemperer (1996) and Kirkegaard (2004) focus on revenue<br />
as the sole comparison criterion. If other objectives (i.e. allocative efficiency) are considered,<br />
then these models do not provide valuable insights. But even in terms of revenue, the decision<br />
whether to use auctions or negotiations crucially depends on the assumptions.<br />
251
9<br />
Both papers only refer to single‐issue mechanisms, a comparison between auctions and negotiations<br />
that are capable of submitting several attributes beyond the price has not yet been<br />
<strong>und</strong>ertaken. The reason for this can be in the fact that studies of auctions that consider more<br />
attributes have only been published recently.<br />
Che (1993) and Branco (1997) initiated studies on the buyer’s payoffs in the two‐attribute (i.e.,<br />
price and quality) auctions. The private information of buyers determining the utility can be<br />
represented in one dimension; this shortcut allows applying the auction design apparatus to<br />
these problems. More recently, Beil and Wein (2003) analyzed the problem of designing the<br />
multi‐attribute auction. They were in particular concerned with finding a scoring rule to<br />
maximize buyer’s utility.<br />
3.1.2 Laboratory and field experiments<br />
Thomas and Wilson (2002; 2005) conducted two experimental studies, in which reverse auctions<br />
were compared with multi‐bilateral negotiations. In their most recent laboratory experiments,<br />
they noted that general superiority of auctions predicted by Bulow and Klemperer’s<br />
(1996) was not supported by the empirical data. In multi‐bilateral negotiations, a buyer<br />
solicits price offers from multiple sellers and then the buyer requests more favourable offers<br />
form the sellers who need to compete against each other. In their first experiment Thomas<br />
and Wilson (2002) compared multi‐bilateral negotiations with first‐price sealed‐bid auction.<br />
In the second experiment they replaced the first‐price with the second‐price (Vickrey) sealedbid<br />
auction (2005).<br />
Thomas and Wilson (2002) observed that for the inexperienced buyers and sellers multibilateral<br />
negotiations with two sellers led to significantly higher prices than first‐price sealed<br />
bid auctions. In the experiment with four sellers both mechanisms were fo<strong>und</strong> outcomeequivalent.<br />
In their second study, Thomas and Wilson (2005) observed that prices in second<br />
price sealed bid auctions exceed the prices generated in multi‐bilateral negotiations, suggesting<br />
that this auction mechanisms is inefficient in the given experimental setting.<br />
These two studies compared price‐only auctions and negotiations. Therefore, it is surprising<br />
that in some experimental settings negotiations were fo<strong>und</strong> to be more efficient than auctions.<br />
Many commercial and non‐commercial transactions concern objects characterized by multiple<br />
attributes. Few are now supported electronically with multi‐attribute auction systems.<br />
Ongoing and future work in auctions and negotiation suggests an increase in their use in<br />
multi‐attribute transactions. At present time, we are not aware of any comparative studies of<br />
multi‐attribute auctions and negotiations. Therefore, we restrict our discussion to selected<br />
experiments with multi‐attribute auctions followed by discussion of negotiation experiments.<br />
The highly stylized information exchange in auctions makes it impossible for the participants<br />
(e.g., sellers) to learn the preferences (needs, limitations) of the auction owners (e.g., buyers).<br />
Therefore, much effort in multi‐attribute auctions experiments has been devoted to the role<br />
and scope of preference revelation schemes. Experiments in which the bidders (sellers) were<br />
given the utility (value) function of the buyer show that multi‐attribute auctions do not provide<br />
substantial benefits over comparable single‐attribute auctions (Bichler 2000). In other<br />
252
10<br />
words, even with fully‐revealed utilities the additional complexity outweighs the theoretical<br />
gains.<br />
The impact of information availability on the mechanism efficiency was studied by Koppius<br />
and van Heck (2002). The information availability specifies the type of information that is<br />
given to whom, or when and how it becomes available to whom during the auction. They considered<br />
two types of multi‐attribute English auctions: (1) with unrestricted information availability,<br />
in which suppliers are provided with the standing highest bid and the corresponding<br />
bidder as well as score or bid ranking of the most currently loosing bids; and (2) restricted information<br />
availability, in which the bidders are only informed about the standing highest bid<br />
and bidder. The experiments indicated that auctions with unrestricted information availability<br />
yield higher efficiency than auctions with restricted availability.<br />
Strecker (2004) analyzed the impact of preference revelation schemes on the efficiency of<br />
multi‐attribute English and Vickrey auctions. He concluded that English auctions with revealed<br />
preference structure of the buyer are more efficient than Vickrey auctions, and English<br />
auctions with hidden preferences. Chen‐Ritzo, Harrison at al. (2005) introduced a multiattribute<br />
English auction, where only partial information about the buyer’s utility function<br />
was revealed. They showed that this variant performs better in terms of efficiency than a single<br />
attribute (price‐only) auction. This outperforming of the multi‐attribute over the single<br />
attribute auctions holds even though the bids in the multi‐attribute auction were far away<br />
from those predicted by the solution predicted by theory. Notably, complexity in the auction<br />
mechanism consumes a portion of the efficiency gains over price‐only auctions. This observation<br />
however, contradicts with the findings reported by Bichler (2000).<br />
An empirical analysis of auctions and negotiations in the construction industry revealed the<br />
use of the exchange mechanism depends on the knowledge and complexity of the context and<br />
task (Bajari et al. (2002). Negotiations have advantages if the specifications of the product to<br />
be traded are not well‐defined a priori, which is often the case in this industry. Negotiations,<br />
unlike auctions allow for the discussion and clarification of the specifications. Not surprisingly,<br />
their empirical analysis also reveals that auctions perform poorly in terms of efficiency<br />
when changes in the product design need to be made after the transaction took place.<br />
3.2 Lessons learned form economic research<br />
The apparatus of economics has shed some light into the selection decision whether to employ<br />
negotiations or auctions. However, while the theoretical results are unequivocal, the results<br />
from experiments are inconclusive. The strength of economic approach to study market<br />
mechanisms lies in its formal approach to experiment design and conduct. The power of economic<br />
approach to markets is in its capability of abstracting the functions and mechanisms,<br />
removing interfering events and processes and focusing on transactions. This strength coupled<br />
with the focus on the mechanism design and efficiency may also be seen as a weakness,<br />
in particular if the mechanisms implementation and functioning is an issue. In particular, we<br />
consider here four issues of importance for our discussion.<br />
1. Economics is interested in mechanisms and their economic properties rather than in<br />
their users. The concerns are to determine if a mechanism functions according to its<br />
design, what is its efficiency and what outcomes it produces given an assumed envi‐<br />
253
11<br />
ronment (e.g. preferences), which leaves out many relevant peculiarities of the users<br />
(e.g. beliefs, emotions).<br />
2. One of the most persistent issues within behavioural economics is the representation<br />
of human behaviour. Traditionally economics begins with “the notion of homo<br />
economicus acting in a world with full information, independent decision making,<br />
polypolistic competition, transitivity, and fixed preferences” (Beckert 2003). It relaxes<br />
some of the assumptions in order to study the violations of perfect rationality, but<br />
does not consider the real decision‐makers “messiness” (e.g., deviation from<br />
(bo<strong>und</strong>ed) rationality and adherence to various rationalities (Simon 1986)) making its<br />
results difficult to implement in e‐markets.<br />
3. The impact of the system in which an exchange mechanism is embedded is often ignored.<br />
ICT imposes additional rules on human behaviour and it changes the behaviour<br />
of the participating humans. The devotion of economics to analyze highly abstract<br />
mechanisms “with little or no concerns for practical application” (Palfrey 2001) diminishes<br />
the value of economic theory for assessing real‐world negotiation and auction<br />
systems.<br />
4. Economics largely focuses on price‐only mechanisms. Only a few papers address negotiations<br />
or auctions that use multiple attributes in allocation. Neglecting multiattribute<br />
problems diminishes the applicability of economics in procurement and<br />
other settings, where the resources for sale are not specified from the outset of procurement<br />
process. Although advancements have been made in the area of multiattribute<br />
auctions but a comprehensive comparison with multi‐attribute negotiations<br />
is still missing.<br />
The few shortcomings mentioned here do not take away significant contributions of economics<br />
to market research. The discipline of information systems with its concern for peoples’<br />
perceptions, attitudes and feelings, may help introducing ICT to behavioural economics. IS<br />
may allow for a more comprehensive e‐market framework which can be used for the assessment<br />
of auctions and negotiation systems, in addition to the assessment of the mechanisms<br />
embedded in them.<br />
3.3 Information systems research<br />
Information systems research has been concerned with the success of systems; their roles in<br />
organizations and their impact on individual and organizational performance. Its purpose is<br />
to “further knowledge that aids in the productive application of information technology to<br />
human organizations and their <strong>management</strong>” (ISR 2002). Arguably, one of the reasons of IS<br />
researchers focus on success and productivity is because of the “IT productivity debate” of the<br />
1980s and 1990s and the fact that past empirical research generally did not significant productivity<br />
improvements associated with IT investments (see for review of 150 articles Brynjolfsson<br />
and Yang 1996). Economic studies at both micro and macro levels led to a “sobering conclusion:<br />
our <strong>und</strong>erstanding of how information technology affects productivity either at the level<br />
of the firm or for the economy as a whole is extremely limited” (Wilson 1995).<br />
Glass et al. (2004) review of over 1,500 papers from IS, computer science and software engi‐<br />
254
12<br />
neering shows that the IS papers are predominantly concerned with evaluative and descriptive<br />
research (76%) with validation coming from field studies (27%), laboratory experiments (16%)<br />
and case studies (13%). Discussion on the IT evaluation measure has been, as DeLone and<br />
McLean (1992) put it “the quest for the dependent variable” and it spanned the same period as<br />
the discussion on the IT productivity debate. The results of this discussion are three main<br />
models presented below and their numerous variants.<br />
3.3.1 Technology acceptance model<br />
TAM is one of the models most often used to explain the willingness of potential users to actually<br />
use an information system (Davis 1989). In this, as well as in other IS models technology<br />
is viewed as hardware, software, services and their combinations.<br />
This model was extensively tested empirically (see, Legris et al. 2003 for a review) and several<br />
extensions have been proposed (e.g. Szajna 1996; Al‐Khaldi 1999). According to the basic<br />
model, the actual use of a system is determined by the behavioural intention to use a system.<br />
This intention depends on the attitude towards the system, which is described by two subjective<br />
factors: the perceived usefulness and the perceived ease of use of the system. The model<br />
and the research instruments constructed within its framework have been fo<strong>und</strong> useful in<br />
studying potential and actual information system adoption and use in organizations (e.g. Szajna<br />
1996; Agarwal et al. 1997; Jackson et al. 1997).<br />
More recently Vankatesh et al. (2000) proposed TAM2, which adds social norms to TAM. The<br />
extended model had been tested in field studies and the authors fo<strong>und</strong> that social norms and<br />
perceptions significantly influence user acceptance of technology. It should be noted, however,<br />
according to meta‐analyses and field studies, most versions explain about 40% of the<br />
system’s use (Agarwal et al. 1997; Venkatesh et al. 2000; Legris et al. 2003)<br />
3.3.2 Task‐technology fit model<br />
Technology may be accepted and used, and yet fail to bring forth expected changes. Goodhue<br />
and Thompson (1995) proposed the technology‐to‐performance model and tested its part<br />
known as task‐technology fit (ttf) model. The dependent variable is the individual performance<br />
and the purpose of the model is to measure the impact of the technology on such.<br />
The model considers tasks, which are activities requiring application of knowledge; they involve<br />
synthesis, assessment, problem solving and decision making. It asserts that individual<br />
performance depends on the fit between the task a user needs to <strong>und</strong>ertake and the technology<br />
used for this task (Goodhue 1995).<br />
The fit between task and technology is “the matching of the functional capability of available<br />
information technology with the activity demands of the task at hand” (Dishaw and Strong<br />
1996). The better the fit is the higher user performance and, thus value of the technology. The<br />
model also includes the moderating role of user’s characteristics on the user assessment of the<br />
task‐technology fit.<br />
The TTF model, like many other IS research models, considers technology as a complete computer<br />
system (hardware, software and data) as well as organizational IS policies and services<br />
(Goodhue and Thompson 1995, p. 216). While the individual characteristics of the user (e.g.,<br />
255
13<br />
education, training and experience) are distinguished, the technology is black‐boxed and no<br />
specific features, tools and mechanisms are included. Recent modifications include the “exploration<br />
of the black box” (Gebauer and Ginsburg 2006) in terms of system characteristics<br />
rather than its structure. System characteristics include data quality, ability to retrieve and<br />
consolidate required data and reliability. They are important but their assessment provides<br />
little insight regarding system adaptation and modification.<br />
The model shares its subjective orientation with TAM and other models; it models user attitudes<br />
and behaviours toward the system (Dishaw and Strong 1998). The measurement of fit,<br />
utilization and performance are based on the user beliefs rather than any objectively measurable<br />
variables pertaining to cognitive effort, costs, time or productivity.<br />
3.3.3 DeLone‐McLean model of information system success<br />
The TAM, TTF and other early models could have been used to study almost any technology.<br />
Although developed within the IS field, they do not consider the specific characteristics of<br />
software and make no distinction between software, hardware and services of the IT departments.<br />
The end user computing satisfaction (EUCS) model (Doll and Torkzadeh 1988) and the<br />
information systems success (ISS) model (DeLone and McLean 1992; DeLone and McLean<br />
2003) explicitly consider the attributes of information and system which produces information.<br />
The purpose of EUCS is to measure satisfaction of IS users through the measurement of five<br />
variables describing information content, accuracy and format, and the system ease of use and<br />
timelines in producing data. The model has been widely used and considered as one that is<br />
principally connected to the IS field with the premise that this field differs from social and<br />
cognitive psychology (Doll and Torkzadeh 1991; Pikkarainen et al. 2006).<br />
The ISS model was designed to explain the key factors that are accountable for the success of<br />
information system projects. The key feature of the model is the separation of the quality of<br />
information provided by the system from the operational measures of quality of a system. The<br />
former includes such measures as information accuracy, precision, timeliness, and relevance,<br />
while the latter refers to such indicators as reliability, response time, and ease of use.<br />
The model aims at predicting the individual and organizational impact of the system based on<br />
the attitudes formed by its users. The key factor conveying the attitude is user satisfaction.<br />
According to the model user satisfaction is influenced by the two essential quality‐related factors<br />
of the system. Satisfaction has a significant impact on the actual system use, and the latter<br />
reciprocally affects the level of user satisfaction. Both of these constructs (use and satisfaction)<br />
affect the construct of individual impact, which in turns influences the organizational<br />
impact.<br />
3.4 Lessons learned from information systems research<br />
The major focus of IS research has been on the attitudes and behaviours induced by the use of<br />
instantiated systems in organizations. This perspective brings in the psychological, organizational<br />
and social factors into the study of the impacts of information systems. The variables<br />
that describe subjective perceptions and aim at the facilitation of adoption of information sys‐<br />
256
14<br />
tems and their impact on the individuals and organizations have been identified. These findings<br />
help explain how people perceive systems and which variables help successful adoption<br />
of systems by the users.<br />
Behavioural models discussed in the preceding section and their various refinements are<br />
largely concerned with technology use, adoption, efficacy and satisfaction of its users. They<br />
are less concerned with the issues pertaining to the identification of the concrete IS artefacts,<br />
features and contributions to the achieved results. They are also less concerned with the identification<br />
of types of users, decision problems and organizations that are particularly<br />
(un)suitable for a given technological solution.<br />
In this respect, it is worthwhile mentioning recent important concerns raised by the prominent<br />
researchers in the field. The first one is concerned with the diversity of themes in IS research<br />
and venturing by the researchers into the areas remotely related to the information<br />
systems (Benbasat and Weber 1996). These tendencies, according to the authors are <strong>und</strong>ermining<br />
the very identity of the IS field (Benbasat and Zmud 2003). The authors stress that the<br />
research should focus on the phenomena intimately linked with the core subject of the field,<br />
which is information systems. On the other hand, another important and possibly related issue<br />
is that of relevance vs. rigor. It has been stressed that IS researchers tend to overemphasize<br />
the rigor in conducting research, while largely sacrificing the relevance of the studies<br />
(Benbasat and Zmud 1999). In our view, the incorporation of system features in theoretical<br />
models could help tackle both of these issues. Firstly, the IS artefact would become an organic<br />
part of these models, thus battling the “identity crisis”. Secondly, this would enable illuminating<br />
better design decisions during system development, as the theoretical findings will provide<br />
direction and support as to which salient features and functionalities to incorporate for a<br />
given class of contexts.<br />
To construct prescriptive rather than descriptive models of socio‐technical systems there are<br />
several issues which need to be considered, which are focussed specifically on the ICT technologies<br />
and their configurations.<br />
1. Presumably, technology is the focus when one tries to identify its usefulness, ease of<br />
use and its other user‐oriented attributes. However, in many IS models technology is<br />
not specified or even broadly defined. Models such as TAM and TTF can be applied to,<br />
for example, computing devices as well as to transportation apparatus. There is nothing<br />
that allows for the identification of the special nature of information systems viz.<br />
some mechanical systems.<br />
2. IS researchers have focused on instantiated systems in the past and paid little attention<br />
to abstract features and models (e.g. mechanisms). This effectively restricted the<br />
possibilities of incorporating the characteristics of systems in theoretical models.<br />
While the IS success model does include the informational and system qualities, it<br />
does not provide sufficient insight into the drivers of IS quality, e.g. whether it derives<br />
from the model embedded in the system, or the way the model has been implemented<br />
in the system. It could be beneficial to separate the concrete features of implemented<br />
systems (e.g. user interface) from more abstract characteristics of the <strong>und</strong>erlying<br />
mechanisms.<br />
257
15<br />
3. IS literature tends to <strong>und</strong>erestimate the value of the objective measures (e.g. outcomes)<br />
associated with the system use, paying more attention to the subjective factors.<br />
While the subjective categories (beliefs, attitudes, etc.) are <strong>und</strong>oubtedly important,<br />
we believe that they are intimately linked to the objective variables, especially<br />
when studying new economic institutions, like e‐markets.<br />
4. Mechanism and system design<br />
E‐markets are economic entities as well as information systems. These markets rely (in their<br />
deep structure) on some family of abstractly defined economic mechanisms. They are also information<br />
systems composed of software and hardware, which expose their functionalities<br />
through specific features of user interface. Thus, the theories that would usefully allow researchers<br />
to assess, compare, and prescribe e‐market systems must include both economic<br />
and IS perspectives. One important question arising in connection with the possibility of such<br />
integration is how the issue of mechanism/system design is viewed in economics and IS.<br />
4.1 Basic concepts and theory building<br />
Market design in economics traditionally uses axiomatic models to obtain propositions and<br />
theorems about the impact exchanges have on resource allocation. Unlike other areas of economics,<br />
market design has breed out a well‐accepted methodology for designing and analyzing<br />
auctions. In recent times, laboratory experiments and numerical simulations have been<br />
added to complement the study of auctions (Roth 2002).<br />
A framework for economic market design research has been proposed by Smith (1982; 2003),<br />
who sought to reconcile theoretic research with laboratory experiments. The framework is so<br />
general that it applies to any microeconomic system. In essence, Smith suggests that the<br />
analysis of alternative exchange mechanisms always share a common view on the structure of<br />
the microeconomic system (Hurwicz 1973; Reiter 1977). The framework sketches an economic<br />
system comprising economic environment, individual preferences, behaviour, mechanism,<br />
outcomes and performance (see Figure 4).<br />
258
16<br />
Economic environment;<br />
context<br />
Individuals: preferences,<br />
costs, resources, knowledge<br />
Individual’s choice<br />
behavior<br />
<strong>Institut</strong>ion: market<br />
mechanism, rules of<br />
contracts, fullfilment<br />
Outcomes:<br />
allocations and prices<br />
Mechanism<br />
performance<br />
Figure 1: Microeconomic system framework (adapted from Smith 2003)<br />
The economic environment describes all factors (e.g., legal regulations, types of products and<br />
services, customs and laws), which influence demand and supply in which economic agents<br />
(individuals) operate. These factors are out of direct control of the mechanism designer. Deviating<br />
from Smith’s depiction, we distinguish two parts of the environment, the context and<br />
the individual characteristics.<br />
The context characterizes the situation, in which the resources are being allocated. For example,<br />
the context may describe a procurement setting with multiple sellers competing against<br />
each other to serve one buyer. Economic theory attempts to eliminate context by abstracting<br />
the resource allocation as much as possible.<br />
The individual (economic agent) is described by all characteristics relevant to the decision<br />
making, that is, his or her concrete choices. The key characteristics are preferences, risk attitude,<br />
costs, resources (endowment) and knowledge.<br />
The individuals’ are affected by the context, in which they operate. Their actual preference<br />
structure and other innate characteristics determine their individual choice behaviour which<br />
impacts, in turn, the functioning of the exchange mechanism.<br />
The market mechanism is the set of rules and formulae which specify the way offers (bids,<br />
proposal) can be formulated and how they are translated into outcomes (i.e., allocation and<br />
prices). In other words, it is fed inputs (offers) and yields as its output an allocation of resources<br />
among the participating users. The mechanism can be seen as a dialogue between its<br />
users leading to an allocation of resources.<br />
Individual choice behaviour does not depend solely on the individual and the environment<br />
she or he operates in. The behaviour is a middle layer between the motivations of individual<br />
buyers and sellers embedded in their local environment and the feasible actions confined by<br />
the market mechanism and the resulting outcomes. The participating users need to formulate<br />
their needs in terms, which are acceptable by the mechanism. Therefore, market institutions<br />
restrict the users’ response behaviour, without uniquely prescribing it (Hurwicz 1973).<br />
259
17<br />
The outcomes of the use of the mechanism by individuals are the allocation of the resources<br />
among the individuals and the corresponding prices which they pay. Finally, performance is<br />
determined by the outcomes, the environment and the mechanism users. The performance is<br />
of the mechanism rather than its users; it is defined by the comparison of the outcomes<br />
achieved when two or more mechanisms are employed.<br />
Given the framework (Figure 4), the mechanism design problem is the identification of a set of<br />
rules for which that the equilibrium outcome satisfies the desired performance expressed by<br />
an evaluation function. The evaluation function values the outcome attained by the mechanism<br />
in a specific environment. This formulation is rather general, as it regards not only the<br />
preferences of the individual, but also all other possible arguments of the environment.<br />
Mechanism design seeks to describe mechanisms, which maximize the evaluation function<br />
subject to the following three types of constraints:<br />
1. Incentive compatibility constraints require that the participants truthfully report their<br />
information about their local environment. In such a case, the outcome is the same as<br />
if a benevolent arbitrator would have chosen the outcome on the basis of the full information<br />
about the environment.<br />
2. Computational compatibility refers to the complexity of the outcome function. Outcome<br />
functions can be very demanding concerning computational tractability<br />
(Rothkopf et al. 1998) and these constraints assure the feasibility of the outcome function<br />
achievement.<br />
3. Participation constraints require that the participants voluntarily take part in the<br />
mechanism’s use. The individuals participate if the benefit they draw out of participation<br />
is higher than participation in an alternative mechanism (Ledyard 1993).<br />
In laboratory experiments, both the environment and the institution are controlled by the experimenter.<br />
Achieving control over institution is in principle straightforward, as the experimenter<br />
not only explains the institutional rules but also enforces them. Achieving control<br />
over the environment is less easy, as the humans who participate in the experiment have idiosyncratic<br />
characteristics that are unknown to the experimenter (and can hardly be observed).<br />
If the experimenter wants to examine theories that rely on certain assumptions on the characteristics<br />
of the individuals, the experimenter has to align the unknown characteristics of the<br />
individual with the assumed behaviour. This is particularly challenging as these two can f<strong>und</strong>amentally<br />
differ from each other.<br />
Experimental economics usually employs induced values. By coupling behaviour with a rewards<br />
scheme the experimenter attempts to induce the values upon the individual, overruling<br />
the real characteristics. Since it is very difficult to control the context, experimenters frequently<br />
try to abstract from the context as much as possible testing theory, whereas the exploration<br />
of new mechanisms <strong>und</strong>er real world is often neglected.<br />
4.2 Design research in IS<br />
The problem of systems analysis and design has long been one of the core subjects of study in<br />
the area of information systems. Considerable attempts have been made in the past in devel‐<br />
260
18<br />
oping methods, techniques, and tools for guiding the development of concrete systems. However,<br />
the study of the IS design is a relatively recent and gaining popularity and consideration<br />
of an important part of IS research (Gregg et al. 2001; Markus et al. 2002; Dufner 2003; Hevner<br />
et al. 2004).<br />
Nunamaker et al. (1991) argued that system development is an important research methodology<br />
to study information systems. They encouraged researchers to develop prototypes as<br />
means of better <strong>und</strong>erstanding the research domain, thus emphasizing the instrumental role<br />
of artifacts (i.e., the means) in IS research. Burstein and Gregor (1999) proposed and defended<br />
the view of system development as a type of action research. They stressed that it can play an<br />
important role in theory building, experimentation, and observation. While these views put<br />
forward a model for IS research whereby IS artifact design is the means of conducting research,<br />
a stronger perspective treats the artifact itself as the end.<br />
Herbert Simon’s seminal book “The Sciences of the Artificial” had introduced the issue of design<br />
research in the computing disciplines (1996). One of the most significant insights by<br />
Simon was the separation of the outer environment from the inner environment of an artifact.<br />
The outer environment could be regarded as the requirements imposed on the artifacts’ function.<br />
The inner environment is the internal organization of the artifact. Simon regarded design<br />
as the interface between the outer and inner environments of an artifact. March and<br />
Smith have shown that the design and natural science approaches in IS research should be<br />
complementary, whereby the design phase consists of building and evaluating IS artifacts, and<br />
the “natural science” phase with theorizing abut artifacts and justifying the theories (March<br />
and Smith 1995). According to them there are four types of products of design research, including:<br />
constructs (language), models, methods, and implementations (this included both<br />
prototypes as well as working systems).<br />
Walls et al. (1992) proposed the notion of a “design theory” for information systems, in which<br />
they envisaged the use of “kernel theories” in the design of a class of artifacts. According to<br />
them design theories should consist of type of user requirements (“meta‐requirements”); type<br />
of system solution or class of artifacts (“meta‐design”); and the type of methodology used to<br />
develop such artifacts. Thus, in this work they had explicitly emphasized the shift of focus<br />
from instantiated systems to the classes of artifacts. They had used an example of “vigilant executive<br />
information systems” to present an example of a design theory. More recently, Markus<br />
et al. (2002) employed this concept of a design theory to devise the characteristics of a broad<br />
class of systems that the authors called “systems that support emergent knowledge processes”.<br />
In a recent publication Hevner et al. (2004) stressed the necessity of emphasizing rigor in<br />
conducting design research and advocated a number of guidelines for design researchers.<br />
In summary, the most important contributions of the design research to the study of information<br />
systems include:<br />
1. Approaching the design of IS artifacts as a legitimate and rigorous research initiative;<br />
and<br />
2. Recognition that the IS artifact in the context of design research refers to classes of systems,<br />
rather than specific instantiations.<br />
Together these principles help bridge the gap between the IS community on one hand and<br />
261
19<br />
those interested in the design of markets and economic exchange mechanisms on the other.<br />
4.3 Convergence of IS and economics designs<br />
Notable attempts have been made in the recent past by the researchers in both fields to entertain<br />
the possibilities of adopting alternative perspectives to studying the core subjects of their<br />
respective disciplines. While information systems are engineered artifacts, engineering approaches<br />
seem to have wider applicability in somewhat less expected areas. Roth (2002) have<br />
advocated an engineering view of economics and pointed at the emerging discipline of design<br />
economics: “the part of economics intended to further the design and maintenance of markets<br />
and other economic institutions” that would rely on the use of computational models and experimental<br />
approaches. Subrahmanian and Talukdar (2004) have discussed how market design<br />
can be formulated as an engineering problem.<br />
Researchers in IS have started integrating economic perspective in their assessment of the utility<br />
and impact of systems, in particular those that support e‐commerce applications (Zwass<br />
1996; Bhargava and S<strong>und</strong>aresan 2004; Zhu 2004). The recent discussion on the nature of the<br />
core of the discipline highlights important alternative models for conceptualizing information<br />
systems. Alter (2003) proposes and defends what he calls the IT‐reliant work system view,<br />
where the emphasis is placed on the business and organizational processes as supported by<br />
the information systems, rather than focusing on purely technological aspects. El‐Sawy (2003)<br />
advocates the adoption of the “fusion” perspective, according to which the IS artifacts have<br />
merged with the respective processes and entities in business environment to an extent that it<br />
is no longer possible to separate them from each other.<br />
As it has been stressed earlier, design research aims at producing generic system solutions to<br />
practical problems, while preserving the level of rigor characteristic of other, more traditional<br />
modes of research. The type of system solutions proposed by a design researcher is a class of<br />
systems, or meta‐systems (Walls, Widmeyer et al. 1992; Iivari 2003). According to one proposed<br />
representational framework, a researcher’s view of an abstract IS artifact can be organized<br />
according to a number of perspectives (Vahidov 2006).<br />
Following these lines of interdisciplinary study, in the next section we propose a unifying<br />
theoretical model that allows studying the impact of exchange mechanisms together with<br />
other contextual variables on the relevant dependent variables derived from the fields of information<br />
systems and economics.<br />
5. TIMES framework<br />
The advance of electronic commerce and new forms of technology‐enabled exchange models<br />
brings together the fields of IS and economics closer than ever before (Zwass 1996; Bhargava<br />
and S<strong>und</strong>aresan 2004; Zhu 2004). This de‐facto merging of the structures of exchanges among<br />
participating economic entities and the types of evolving system solutions to facilitate such<br />
exchanges necessitates the development of novel research models to study the resulting forms<br />
of amalgamated mechanisms. Such models must necessarily integrate the existing theoretical<br />
approaches from different and complementary research areas; this is the purpose of the TIMES<br />
framework, it provides a common fo<strong>und</strong>ation for models that may be constructed with a nar‐<br />
262
20<br />
row and concrete focus. Examples of such models are given in Section 6.<br />
5.1 Basic concepts and dependencies of the TIMES framework<br />
The developments discussed in Sections 3 and 4 facilitate construction of a research model<br />
that draws on the theoretical constructs derived both from economics as well as IS field. The<br />
relevance of these constructs has been studied in behavioural economics and IS. An integrated<br />
model puts forward relationships between the independent variables, including type of<br />
mechanism and contextual factors on one hand, and the performance‐related criteria on the<br />
other. The factors that have their origin in economics include mechanism, environment, and<br />
individual characteristics which drive the choice behaviour of the market participants ultimately<br />
resulting in certain outcomes.<br />
The IS literature has been studying, among other phenomena, the effects of the system, individual,<br />
and task characteristics on the performance and on the subjective (perceived) assessments<br />
of the systems. In the integrated model proposed here the characteristics of: task, individual,<br />
mechanism, environment, and system—jointly called TIMES—are included as independent<br />
constructs. Below we will introduce basic concepts and dependencies first by pointing out<br />
the differences with the microeconomic system framework and with the IS research models.<br />
5.1.1 From the economic environment towards task, individuals and IS environment<br />
Not only economic theories but also laboratory experiments too often define task complexity<br />
away (e.g. in the context of market design the determination of the bidding strategy) by assuming<br />
that people can determine an optimal strategy no matter how complex the task might<br />
be. However, humans tend to adopt simplifying strategies if the task is complex. They also<br />
may ignore complexity. For market design, this implies that the designed exchange mechanisms<br />
may be too complex for humans to derive the optimal strategies.<br />
Complex negotiation tasks that require substantial cognitive efforts tend to lead to suboptimal<br />
solutions (Hyder et al. 2000). People’s cognitive limitations, their lack of interest in engaging<br />
in highly complex transactions, and their involvement with many competitive activities often<br />
lead to their selection of a quick and simple tool which does the work even if the results are<br />
not optimal. Some may know that the use of a simple tool allows them doing more elsewhere;<br />
others may be unable of learning the tool’s intricacies. This latter issue has been studied<br />
within the IS domain.<br />
Having in mind the importance of the task, it appears reasonable to separate the task from the<br />
environment, where the task construct refers to the properties of the goals that need to be<br />
accomplished through the use of an exchange mechanism. Rangaswamy and Starke (2000)<br />
provide the characteristics of bargaining orientation, degree of conflict, time pressure, and<br />
complexity for describing tasks.<br />
The environment construct captures the environmental factors that may have impact on the<br />
negotiation outcomes, including type of market, type of product, level of competitiveness<br />
among buyers and sellers, and other important contextual considerations.<br />
5.1.2 From institutions to mechanisms and systems<br />
263
21<br />
One lesson learned by the economics is that the institution matters. We use the term mechanism<br />
here to reflect the economic notion of institutions. The mechanism reflects the convergence<br />
point of institution construct from the economics perspective and meta‐system from<br />
the IS perspective. Accordingly, the mechanism is an abstract artefact describing the protocol<br />
and mode of exchange regardless of its implementation. The description of the mechanism<br />
can be based upon the Montreal Taxonomy, which provides a comprehensive schema for classification<br />
of such mechanisms (Ströbel and Weinhardt 2003). The characteristics include, but<br />
are not limited to: flexibility, ro<strong>und</strong>s, concurrency, number and nature of attributes of offers,<br />
offer matching, offer evaluation, and others. Additionally, the availability of analytical support<br />
(decision support capabilities) and mediation can be added for a useful description<br />
(Rangaswamy and Starke 2000). Taking only the mechanism into consideration is, however,<br />
not sufficient, as it abstracts away from the implementation and thus limits the consideration<br />
of subjective factors explaining its acceptance.<br />
Different from economics, the TIMES model regards the way and form a mechanism is presented<br />
to its users, its embodiment into a system and the interaction process between the system<br />
and the user to be crucial for studying both systems and their mechanism. We know that<br />
the way problems are presented and the way people are prompted to make choices and solve<br />
problems, affects their behaviours. A mechanism has to be implemented in some medium; it<br />
has to communicate with the user using media. These are design issues and they are no less<br />
important for the mechanism, the outcomes and the performance than the mechanism itself.<br />
Thus, we include the construct system in our model; it reflects those characteristics of the instantiated<br />
systems that are implementation‐specific, including user interface, various features,<br />
and functionalities that process and manage information required by the market participants<br />
(Lumuscio et al. 2003).<br />
5.1.3 From individuals’ choice behaviour to individual<br />
The individual construct refers to those aspects of individuals that tend to be relevant to market<br />
processes and outcomes, including: individual characteristics, number of users, as well as<br />
psychological issues such as attitude, beliefs etc. Here, the purpose is to explain the choices<br />
made by the individuals in light of their characteristics, and their interactions with the task,<br />
system, mechanism, and environment.<br />
5.1.4 From individual to individual, group, role and mechanism<br />
Experimental IS research focuses on the users and their behaviours. The focus of economic<br />
research is on the design of market mechanisms and their functioning. In situations when the<br />
use of a system is required and accepted, an important question is what system to use, by<br />
whom and for what types of tasks. These are the types of questions that both system owners<br />
(firms that provide e‐market services) and users ask. User satisfaction, intention to use a system<br />
and other behavioural user perceptions are important. But no less important are the specific<br />
aspects of the system (including its mechanisms and functions).<br />
The independent constructs of the TIMES model impact, through the user‐system interactions,<br />
the individual behavioural perceptions and outcomes and also the economic outcomes. Both<br />
types of outcomes describe performance of individuals, groups participating in a single transaction<br />
(e.g., dyads in bilateral negotiations), roles (e.g., buyers, sellers and auction owners),<br />
264
22<br />
and the mechanism performance (e.g., English auction and Vickrey auction).<br />
5.1.5 From system performance to perceptions, and behavioural and economic outcomes<br />
The assessment of performance used in the TIMES model is more complicated than the economic<br />
notion of performance. The dependent variables—partly derived from the IS research—<br />
are both objective and subjective or “perceived”. The former type seems to be emphasized in<br />
economics, while the latter has been the focus of extensive behavioural investigations in the<br />
area of information systems. The importance of subjective assessments lies in the fact that the<br />
latter tend to contribute towards the adoption of the systems by individuals. We have incorporated<br />
economic performance as an objective dependent construct in our model and two<br />
types of behavioural variables: perceptions and behavioural outcomes. Perceptions describe<br />
user assessments and beliefs of the system and its features and objects, including usefulness,<br />
ease of use for such tasks and aspects as communication and learning about the counterpart<br />
priorities. Behavioural outcomes pertain to the subjective assessments of the final results, e.g.,<br />
satisfaction with the agreement, satisfaction with the process and relationship established<br />
with other participants.<br />
5.2 Dependent constructs of TIMES<br />
The overall impact of TIMES variables on important dependent constructs is shown in Figure 2.<br />
As we have already introduced the TIMES (independent) variables, in the section we will elaborate<br />
on the choice of the dependent ones, as well as the impacts on them.<br />
Figure 5. TIMES framework<br />
5.2.1 Process<br />
The transaction process is clearly identified in the microeconomic framework (Section 4.1);<br />
the choices made by the individuals, and the information provided by the mechanism and<br />
other market participants is the interaction process in which e‐market system plays a role. IS<br />
models, with the exception of the DeLone‐McLean model (Section 3.3.3), tend not to model<br />
265
23<br />
the interaction process. Most models are variance models and according to Seddon et al.<br />
(1999), “variance‐ and process‐model diagrams represent quite different concepts and cannot<br />
be combined meaningfully in one model”. Variance models explain variation in a dependent<br />
variable by studying its association with one or more independent variables while process<br />
models explain an outcome by identifying the sequence of events or states that precede it.<br />
E‐market systems make interactions between market participants possible; these interactions<br />
have social aspects and the process perspective allows studying the different activities <strong>und</strong>ertaken<br />
by the participants. These activities lead to the behavioral and market outcomes, and<br />
thus the introduction of process constructs that allow for the actual use of the system’s services<br />
and their impact on the outcomes. The TIMES framework looks at the efficacy of mechanisms<br />
and services, the development of prescriptions for advocating exchange mechanisms<br />
and service b<strong>und</strong>les, and the need for process consideration is similar as in the IS development<br />
(Robey and Newman 1996; Kim and Pan 2006).<br />
The interaction process includes formulating and submitting offers (possibly accompanied<br />
with and/or embedded in messages or contract forms) and receipt and analysis of offers made<br />
by other participants. There may also be other activities that involve user‐system interactions,<br />
such as, agenda setting, preference elicitation and value (utility) function construction, and<br />
the specification of reservation levels and the best alternative to the negotiated agreement.<br />
The process that involves these activities may lead to the users’ reassessments and change of<br />
their strategies and tactics.<br />
The consideration of the process categories allows including non‐monetary costs of the transaction<br />
and study the relationship between mechanisms and systems and these costs. For example,<br />
in one of the preliminary studies of trade negotiations conducted via an e‐negotiation<br />
system ENS) it was fo<strong>und</strong> that a service that provides graphical representation of the negotiation<br />
dynamics does not affect behavioural and market outcomes but it leads to much shorter<br />
(334 words on average) messages exchanged (Weber et al. 2006). This indicates that graphical<br />
service reduces users’ efforts; we would not be able to observe this relationship without explicit<br />
consideration of process.<br />
5.2.2 Perceptions: ease of use, usefulness and other assessments<br />
The technology acceptance model (TAM) postulates perceived usefulness and ease of use being<br />
important factors that help predicting actual usage of the system in question. Because of system<br />
adaptability and adjustment potential, we are less concerned with the actual usage than<br />
with the users’ perceptions of the services and mechanisms ease of use.<br />
The effect of ease of use on perceived usefulness had been postulated by TAM (Davis 1989) and<br />
widely studied since the early 1990s. The TAM model had been recently criticized for ignoring<br />
important variables related to system features (Wixom and Todd 2005). An alternative stream<br />
of research had been focusing on the way system features influence users’ beliefs and attitudes,<br />
including, most notably user satisfaction (Baroudi and Orlikowski 1988; Doll et al.<br />
1998). We propose to extend the studies to include mechanisms embedded in systems. We<br />
should mention here that the main difference between the usability studies and the services’<br />
ease of use and usefulness assessment is in the detailed, during‐the‐process measurement in<br />
usability engineering (Nielsen 1994) and post‐transaction measurement in the TIMES frame‐<br />
266
24<br />
work.<br />
E‐market systems may be used to engage in purely economic transactions; they may also be<br />
used for socio‐economic transactions which involve such issues as trust, relationships and affect<br />
(Bakos 1998). The system contribution to the users’ perceptions of effective communication,<br />
gaining <strong>und</strong>erstanding of the counterparts and competitors, and trust and relationship<br />
building may be valuable and no less important than its ease of use and usefulness. While usefulness<br />
is considered in a narrow, transaction‐oriented sense, occasionally the social aspects<br />
which arise during the transaction process may be considered more important than the purely<br />
economic ones.<br />
5.2.3 Behavioural outcomes: satisfaction, relationship and other outcomes<br />
A model integrating important service‐based and behaviour‐based beliefs had brought together<br />
the important theoretical constructs to provide a more comprehensive picture of system<br />
adoption (Wixom and Todd 2005). In this model satisfaction with the way the system<br />
provided informational support was fo<strong>und</strong> positively related to the perceived usefulness of the<br />
system.<br />
In the TIMES framework we are interested in, on one hand, the ease of use and usefulness of<br />
services and the satisfaction with different facets of the overall process (experience). Therefore,<br />
the construct satisfaction is a behavioural outcome associated with the achievement of<br />
both goals established prior the transaction process and the factors that the users did not consider<br />
beforehand. Satisfaction is an integrative evaluation of the person’s attitudes as they<br />
form during a process (Yi 1990); it is a high level assessment that integrates other cognitive<br />
components. Thus, we have incorporated satisfaction as an important factor of the perceived<br />
usefulness of the system. Since we are interested in both important objective (economics) as<br />
well as subjective (behavioural research in IS) factors our framework postulates that the satisfaction<br />
and other behavioural outcomes may be influenced by the objective performance of<br />
individuals.<br />
Based on a review of IS literature and research instruments Yu (2007) identifies eight categories<br />
of satisfaction: 1) satisfaction with the achieved outcome, 2) satisfaction with own performance,<br />
3) satisfaction with the relationship with other participants, 4) satisfaction with information<br />
obtained from and via the system, 5) satisfaction with the communication media,<br />
6) satisfaction with transaction process, 7) satisfaction with the system, and 8) overall satisfaction.<br />
We propose considering these categories because they allow for the assessment of different<br />
dimensions of the system, its services and the process.<br />
5.2.4 Market outcomes<br />
Objective (market) outcomes are separated from subjective evaluations because the former<br />
describe results achieved by the individuals and their different groupings, such as, all individuals<br />
who participate in the same process (e.g., bidding), all those who play the same role<br />
(e.g., buyers, sellers), and all those who use the same mechanism or system. Market outcomes<br />
include utility values, prices, distance to efficient frontier, distance to axiomatic solutions, and<br />
social welfare distribution. Performance measures also include agreement rate, balance of the<br />
deal, early withdrawal rate and, in addition to utility or profit, they may also reflect equity.<br />
267
25<br />
Also, the recent study suggests that economic outcomes do not necessarily behave identically<br />
as behavioural ones (Galinsky et al. 2002). For example, when high targets are stated, negotiators<br />
tend to be dissatisfied with the objectively superior outcomes. We also note that the satisfaction<br />
is individual while the market outcomes not necessarily so (some depend on more<br />
than one person).<br />
6. Experimental assessments<br />
The verification of the TIMES framework and its construct requires several experiments and<br />
field studies because the framework includes behavioural, economic and technical perspectives,<br />
all of which allows for the construction of different models. By providing a common<br />
fo<strong>und</strong>ation for comparative behavioural studies of information systems the framework allows<br />
studying various systems and their components. A thorough and exhaustive TIMES assessment<br />
is beyond the scope of this work; we hope that the two studies reported in this section will be<br />
followed by a larger number of experimental and field studies.<br />
6.1 The NorA project<br />
For the purpose of comparison of e‐market systems, we need a detailed specification of the<br />
model constructs and the formulation of the associated research instruments. The comparison<br />
has been <strong>und</strong>ertaken as part of the project called “Negotiations or auctions” (NorA) (InterNeg<br />
2006; Yu 2007). The project focus is the comparison of two market mechanism and two e‐<br />
market systems. The overview of the project and its relationship to the TIMES framework is<br />
illustrated in Figure 6.<br />
Figure 6. Research model for the NorA project<br />
In the first study the same software platform was used with a very similar interface but with<br />
different market mechanisms, i.e., auction and negotiation (Yu 2007). The second study compared<br />
different system interfaces and different internal implementations of the same mechanism<br />
(English auction) in (Kolitz and Neumann 2007). In our attempt towards model building,<br />
we investigate the effects of mechanism and system based on general hypotheses of process<br />
and outcome variables, rather than postulating formal hypotheses.<br />
6.1.1 Two mechanisms<br />
268
26<br />
While widespread popularity of on‐line auctions is well known, however—as we mentioned in<br />
Section 2.1—their claimed superiority is not obvious. Therefore, we would like to attempt delineating<br />
the types of individuals and transactions (tasks) for which auctions are preferable<br />
and those, where negotiations are more suitable. This is one purpose of the NorA project for<br />
which the TIMES framework was used to construct models. Another purpose of the project is<br />
to study the ease of use and usefulness of specific features and tools of the systems, and test<br />
the relationships between the perceived ease of use and usefulness and the individual and task<br />
characteristics.<br />
For the purpose of illustration of the framework application we consider the following two<br />
exchange mechanisms:<br />
1. Multi‐attribute English auction with a buyer being the auction owner; and<br />
2. Multi‐bilateral purchasing negotiation involving one buyer and several suppliers.<br />
The auction bidders can submit multiple bids during the process. Between the bids they can<br />
have an information feedback and they have the chance to adjust their bids. This characteristic<br />
resembles negotiation to some extent, and makes the comparison easier.<br />
The two mechanisms can be further characterised by several parameters, each taking one or<br />
more values. The concrete mechanism implementation requires that the transaction rules and<br />
all their parameters are specified. The main parameters describing the two mechanisms and<br />
the values selected for illustrative purposes (and used in NorA experiments) are listed in Table<br />
1.<br />
Table 1. Two exchange mechanisms and their selected parameters<br />
Parameter Auction Negotiation<br />
Issues (auction attributes) Four Four<br />
Issue values Given by the principal Given by the principal<br />
Knowledge of owner’s utility Partial Partial<br />
Offers Complete Complete or incomplete<br />
Offer commitment Binding Binding<br />
Messages No Yes with owner<br />
Termination Time and reservation level Time and bilateral agreement<br />
Transparency Full, best offer None, final offer<br />
The difference between auction and negotiation has been the subject of numerous studies.<br />
Thomas and Wilson (2002, 2004) fo<strong>und</strong> that negotiations and auctions were similar in economic<br />
efficiency for a market of four bidders or greater competing on a single attribute. But,<br />
when the number of bidders was reduced, negotiation produced higher revenues for the bidders<br />
(i.e., less efficient). Therefore, we hypothesize that:<br />
H1a. Negotiation will generate higher revenue for the bidders.<br />
The behavior outcomes consist of satisfaction with outcome, which is the difference between<br />
the results of the exchange in comparison to the expectations that have anchored the process,<br />
and satisfaction with process points to the emotions expressed and the degree of conflict felt<br />
by the participants (Suh 1999). Ivanova‐Stenzel et al. (2005) fo<strong>und</strong> that participants were more<br />
269
27<br />
satisfied with the results derived from an auction because the process is more transparent. We<br />
expect that:<br />
H1b(i) and H1b(ii). Auction will generate greater satisfaction with outcome and satisfaction<br />
with process for the bidders.<br />
Even though the mechanism is embedded into the e‐market system, the difference in the<br />
market institution would not interfere with the user’s perceptions of the system. We posit<br />
that:<br />
H1c. Auction will have no differential effect on IS perceptions (i.e., usefulness, ease of<br />
use, information quality, system reliability) relative to negotiation.<br />
As auctions require little to no interaction among market participants and the owner, we hypothesis<br />
that:<br />
H1d. Auction will reduce cognitive effort for the bidders.<br />
6.1.2 Two systems<br />
The two mechanisms are embedded in systems generated by the Invite software platform<br />
(Strecker et al. 2006). Invite software platform allows setting up systems supporting different<br />
auction and negotiation processes and varying the range and type features and tools available<br />
to the system users. One of important capabilities of the platform is the ability to easily modify<br />
the interface of the generated systems. Figure 7 shows two screen shots, the left one comes<br />
from the auction system and the right‐hand side comes from the negotiation system. In order<br />
to reduce the potential impact of the system interface “look and feel” on process and its results<br />
the differences between these two systems are minimal (the colouring is different only<br />
for illustrative purposes). However, both systems have complete functionalities and provide<br />
support for offer (bid) analysis, selection and comparison.<br />
270
28<br />
Figure 7. Screen shots (view history) of two Invite systems: English auction (InAuction)<br />
and multi‐bilateral negotiation (Imbins)<br />
meet2trade is a generic, flexible trading platform facilitating the creation and automation of<br />
auction‐based markets. The platform is flexible enough to host markets from a large variety of<br />
domains and to support various market mechanisms. It can be configured by a Market Modeling<br />
Language (MML). This language was developed to describe electronic market parameters<br />
and to enhance the development and creation of electronic auctions (Mäkiö and Weber 2004).<br />
The experimental system MES allows us to conduct economic experiments on electronic markets<br />
with rigorous control on the procedures (Kolitz and Weinhardt 2006). meet2trade follows<br />
a client‐server architecture with a central server. The server provides the running platform<br />
for all available markets as well as the hosting of all data (e.g. user data, account data,<br />
product information, protocol data) and the data preparation. The clients connected to this<br />
central server display this data and provide an interface for submission of bids and for displaying<br />
relevant information. Table 2 provides a comparison of characteristic fo<strong>und</strong> in Invite and<br />
meet2trade, and Figure 8 shows a multi‐attribute, English, auction on meet2trade.<br />
Table 2. Two e‐market system platforms and their characteristics<br />
Characteristics Invite meet2trade<br />
User interface Two screens One screen<br />
Functionality Web browser Java applet<br />
Features Offer rating, activity table, history graph Offer rating, activity table<br />
Figure 8. Screen shots of meet2trade (multi‐attribute English auction)<br />
Given these different instantiation of Invite and meet2trade as independent variables, the following<br />
general propositions are made taking into account the past research in e‐market and<br />
271
29<br />
related exchange systems described in the previous sections.<br />
From the economic perspective, market outcome is the result of the mechanism that governs<br />
the transaction process and not due to the system that mediates the exchange. We hypothesize<br />
that:<br />
H2a. InAuction will have no differential effect on revenue relative to meet2trade.<br />
While the systems have different user interface, they process bidding information in the same<br />
manner, which reduces variation on the impact of sociological aspects on outcome (Delaney<br />
et al. 1997). We postulate that:<br />
H2b. InAuction will have no differential effect on behavioral outcomes (i.e., satisfaction<br />
with outcome and satisfaction with process) relative to meet2trade.<br />
The characteristics of each system play an important on its perception according to cognitive<br />
theory. Given that Invite provides a history graph of the user’s performance during the auction,<br />
we posit that:<br />
H2c(i) and H2(ii). InAuction will be perceived as more useful and as providing higher information<br />
quality.<br />
On the other hand, meet2trade functions on one screen such that users readily see all bidding<br />
information. In addition, meet2trade uses Java applet technology, which does not require refresh<br />
time that a browser would need. Thus, we expect that:<br />
H2c(iii) and H2(iv). meet2trade will be perceived as easier to use and as being more reliable.<br />
6.1.3 The experimental setup<br />
Both studies employed a contract case involving: (1) an agent representing an artist, and (2)<br />
representatives of three entertainment firms interested in signing up the artist. The case entails<br />
four discrete issues: (1) the number of song the artist must introduce each year, (2) the<br />
royalties given to the artist for each CD sold, (3) a contract signing bonus, and (4) the number<br />
of promotional concerts that the artist must perform each year, all of which are comprised of<br />
various options. The importance for each of these issues was explicitly defined for the subjects<br />
in the negotiation.<br />
Participants consist of university students from a major Canadian city. They play the role of<br />
agents who represent one side of the case, and they are randomly assigned to different treatments<br />
depending on the study. The experiments were divided into three phases: preexchange,<br />
whereby participants answer questions about the case, their reservation values and<br />
expectation, as well as their conflict approach; exchange, whereby subjects interact with the<br />
system in hopes of winning a successful contact that is favorable for the principal; and postexchange,<br />
whereby participants answer questions on their experience related with the research<br />
variables.<br />
6.1.4 The measures<br />
272
30<br />
The variables are categorized as: (1) independent variables, (2) subjective evaluations of IS<br />
perceptions and behavioral outcomes, and (3) objective measure of market outcome.<br />
The independent variables were manipulated according to the e‐market system (Imbins, In‐<br />
Auction and meet2trade) following their difference in mechanism and system.<br />
The subjective dependent variables were measured using slightly altered items from previous<br />
research to meet the needs of this project (satisfaction with outcome and process were<br />
adapted from (Suh 1999); cognitive effort was developed based on (Lilien et al. 2004); usefulness<br />
and ease of use were modified from (Davis, Bagozzi et al. 1989); and information quality<br />
and system reliability were adapted from (Wixom and Todd 2005)). Factor analysis served to<br />
assess the quality of the seven constructs (information quality, system reliability, usefulness,<br />
ease of use, satisfaction with outcome and satisfaction with process). In Table A of the appendix<br />
show the factor rotation whereby convergent and discriminant validity are supported. It<br />
also relates the reliability of the items assessed by Cronbach alphas between 0.708 and 0.896).<br />
The objective market variable is the revenue obtained by participants after the exchange.<br />
Since only one bidder can win the auction or negotiation in each market, this variable is determined<br />
solely by the winners.<br />
6.2 Results<br />
In experiment 312 students participated; most were aged between 20 and 30 years old; and the<br />
male‐female proportion was the same for both mechanisms. There were 20 multi‐bilateral negotiations<br />
(three parties negotiated with a single counterpart) and 27 auctions with three bidders.<br />
The effective samples for this study (i.e. those instances with complete questionnaires)<br />
enabled us to conduct analysis on both objective and cognitive measures. The data analysis of<br />
subjective measures is only for the buyer side (i.e. the music companies) because there was no<br />
auctioneer in auction sessions, such that the sample consist of only 272 records.<br />
6.2.1 Impact of mechanism and system<br />
The responses collected from the participants were tested using a series of analysis of variance<br />
(ANOVA). The results are reported in Table4 with significant effects highlighted.<br />
The market outcome was expected to be affected by the mechanism (H1a) and not the system<br />
(H2a). However the results show the opposite, meaning that negotiation is no better than auction,<br />
but InAuction generates higher revenues than meet2trade.<br />
Users related greater satisfaction with outcome and process for auction compared to negotiation<br />
as predicted by H1b (i) and H1b (ii). There was no difference between the two auction systems,<br />
which supports H2b (i) and H2b (ii).<br />
H1c predicted that difference in mechanism would not result in difference in IS perceptions,<br />
and it is supported. In terms of IS perceptions when different systems are used, meet2trade<br />
was fo<strong>und</strong> to be easier to use and more reliable, supporting H2c (iii) and H2c(iv). InAuction<br />
273
31<br />
did not result in greater usefulness or information quality as expected in H2c (i) and H2c(ii).<br />
The cognitive effort required in the exchange process follows H1d and H2d, which means that<br />
auction requires less effort than negotiation and systems have no effect.<br />
Market outcome<br />
Table 4. Summary of experimental results of mechanism and system comparisons<br />
Imbins InAuction meet2trade<br />
Negotiation Auction Auction<br />
Invite Invite meet2trade<br />
Mechanism<br />
ANOVA<br />
System<br />
ANOVA<br />
mean (SD) mean (SD) mean (SD) F (p‐value) F (p‐value)<br />
• revenue 4.95 (16.48) 7.27 (11.04) 0.57 (8.51) 0.44 (0.51) 6.46 (0.013)<br />
Behavioral outcomes<br />
• satisfaction<br />
‐0.32 (1.86) 0.46 (1.60) 0.28 (1.33) 7.44 (0.007) 0.66 (0.42)<br />
with outcome<br />
• satisfaction<br />
1.35 (1.33) 1.72 (0.84) 1.56 (0.84) 4.30(0.04) 1.48 (0.23)<br />
with process<br />
IS perceptions<br />
• usefulness 1.21 (1.34) 1.31 (1.06) 1.45 (1.11) 0.57 (0.45) 0.77 (0.38)<br />
• information<br />
1.50 (1.16) 1.58 (0.91) 1.57 (0.99) 0.27(0.61) 0.013 (0.91)<br />
quality<br />
• ease of use 1.93 (1.14) 1.81 (0.98) 2.14 (0.72) 0.37 (0.54) 7.08 (0.008)<br />
• system reliability<br />
0.93 (1.40) 1.07 (1.35) 1.82 (0.91) 0.49 (0.48) 19.11 (0.00)<br />
Process<br />
• cognitive effort 0.42 (1.48) 0.92 (1.24) 0.61 (1.20) 4.84 (0.029) 2.62 (0.11)<br />
6.2.2 Impact of process<br />
Further analysis was performed to determine the influence of the process, in view of only cognitive<br />
effort, on the outcome variables. We ran exploratory regression analyses with cognitive<br />
effort as the antecedent on market and behavioral outcome, as well as IS perceptions. The results<br />
are summarized in Table 5.<br />
In general, the cognitive effort reported does not affect the participant’s revenue, satisfaction<br />
with outcome or system reliability. But all e‐market systems led users to express that cognitive<br />
effort influences their perception of information quality and reliability of the system. When<br />
using a negotiation mechanism, users describe that less effort influences greater satisfaction<br />
with process. In instances of Invite, less effort allows for higher perception of usefulness.<br />
Table 5. Regression of process (cognitive effort) on outcome variables<br />
Imbins InAuction meet2trade<br />
System Invite Invite meet2trade<br />
Mechanism Negotiation Auction Auction<br />
βˆ<br />
F<br />
(p‐value)<br />
2<br />
R<br />
βˆ<br />
F<br />
(p‐value)<br />
2<br />
R<br />
βˆ<br />
F<br />
(p‐value)<br />
2<br />
R<br />
274
32<br />
Market outcome<br />
• revenue ns ns ns<br />
Behavioral outcomes<br />
• satisfaction<br />
with outcome<br />
• satisfaction<br />
0.53 22.91<br />
(0.000)<br />
with process<br />
IS perceptions<br />
• usefulness 0.50 19.31<br />
(0.000)<br />
ns ns ns<br />
0.28 ns ns<br />
0.25 0.31 8.71 0.10 ns<br />
(0.004)<br />
• information<br />
quality<br />
0.45 14.71<br />
(0.000)<br />
0.20 0.31 8.63<br />
(0.004)<br />
0.10 0.33 9.78<br />
(0.002)<br />
• ease of use 0.41 12.00 0.17 0.43 18.30 0.18 0.43 17.97<br />
(0.001)<br />
(0.000)<br />
(0.000)<br />
• system reliability<br />
ns ns ns<br />
0.33<br />
0.19<br />
6.3 Summary<br />
The mechanism and system comparisons revealed that economic revenue acquired by bidders<br />
is affected by the difference in system rather than mechanism. However users report contrary<br />
results, in that they are more satisfied with the outcome and process of an auction versus a<br />
negotiation. IS perception is rightfully unaffected by the mechanism, but a faster system with<br />
one screen (meet2trade) impacts perceived ease of use and system reliability. Adding a history<br />
graph (InAuction) has not enabled greater perception of usefulness or information quality.<br />
The process, in terms of reported cognitive effort, influences perceived information quality<br />
and ease of use for all e‐market systems, because these two construct describe what is need on<br />
how the participants interact with the system.<br />
7. Discussion and future work<br />
Electronic commerce has led to the emergence of the new forms of electronic exchange<br />
mechanisms driven by the convergence of economic and technological factors. The complexities<br />
of these formations preclude one‐sided attempts to meaningfully study them from either<br />
side. Instead, a comprehensive approach to investigate the components and workings and<br />
guide the design of electronic markets demand an adoption of the integrated technoeconomical<br />
frameworks of reference. The purpose of this paper has been to propose a model<br />
for studying the impacts of electronic exchange mechanisms on key variables of interests,<br />
both objective, as well as subjective ones. To this end we had reviewed and incorporated relevant<br />
concepts from both fields of Economics and Information Systems.<br />
The proposed TIMES model provides a framework that allows studying of types of exchange<br />
mechanisms in their various implementations within different task, environment, and individual<br />
contexts. These mechanisms could range from the simplest catalogue‐based models to<br />
advanced auction and negotiation schemas. Thus, the model can accommodate continuity in<br />
the key design principles of the mechanisms, as opposed to considering them as distinct<br />
275
33<br />
classes. Therefore, one of the key contributions of the model is that it enables the comparison<br />
of various exchange structures in terms of the same set of key dependent factors.<br />
One research project that will utilize TIMES to compare particular forms of auctions vs. negotiations<br />
has been briefly discussed earlier (NorA). This project will provide empirical data for<br />
testing the theoretical model, and provide key insights into the implications of design of market<br />
mechanisms for different classes of tasks. In particular, the key objective process and performance<br />
characteristics, as well as subjective perceptions and evaluations induced by adopting<br />
auctions vs. negotiations for specific tasks will be revealed. Comparisons of other relevant<br />
types of mechanisms within the TIMES framework will become the core part of future empirical<br />
efforts.<br />
While the study of electronic exchange mechanisms has been the primary motivation for developing<br />
the TIMES model, we believe it is not limited to studying information systems for<br />
conducting market transactions only. It can be used to study also other information systems<br />
for which the issues of their ease of use, performance and usefulness are of interest. In this<br />
respect, the inclusion of the abstract representation of the <strong>und</strong>erlying “mechanism” in addition<br />
to the concrete implementation‐specific features would enable studying broad classes of<br />
systems. The proposed model thus could be potentially extended to become a powerful tool<br />
for the design research community, as the latter is focusing on developing innovative classes<br />
of systems. This extension of the TIMES model would however require to expand the notion of<br />
the mechanism towards the task. The emphasis of the mechanism (and algorithms) embedded<br />
in the system f<strong>und</strong>amentally affect the perceived usefulness of the system.<br />
References<br />
Al-Khaldi, M. A. (1999). "The Influence of Attitudes on Personal Computer Utilization among<br />
Knowledge Workers, The Case of Saudi Arabia." Information and Management 36(4): 185-204.<br />
Alter, S. (2003). "18 Reasons Why IT-Reliant Work Systems Should Replace “The IT Artifact” as the<br />
Core Subject Matter of the IS Field." Communications of the Association for Information Systems<br />
12(23): 366-395.<br />
Bajari, P., R. McMillan, et al. (2002). "Auctions versus Negotiations in Procurement: An empirical<br />
analysis." Working Paper.<br />
Baroudi, J. and W. Orlikowski (1988). "A Short-form Measure of User Information Satisfaction: A<br />
Psychometric Evaluation and Notes on Use." Journal of Management Information Systems 4(4): 44–<br />
59.<br />
Beckert, J. (2003). "Economic Sociology and Embeddedness: How shall we conceptualize Economic<br />
Action?" Journal of Economic Issues 27(3): 769-787.<br />
Beil, D. R. and L. Wein (2003). "An inverse-optimization-based auction mechanisms to support a<br />
multiattribute RFQ process." Management Science 49(11): 1529-1545.<br />
Benbasat, I. and R. Weber (1996). "Research commentary: Rethinking "diversity" in information systems<br />
research." Information Systems Research 7(4): 389-399.<br />
276
34<br />
Benbasat, I. and R. W. Zmud (1999). "Empirical Research in Information Systems: The Practice of<br />
Relevance." MIS Quarterly 23(1): 3-16.<br />
Benbasat, I. and R. W. Zmud (2003). "The Identity Crisis within the IS Discipline: Defining and<br />
Communicating the Discipline's Core Properties." MIS Quarterly 27(2): 183-194.<br />
Bhargava, H. K. and S. S<strong>und</strong>aresan (2004). "Computing as Utility: Managing Availability, Commitment,<br />
and Pricing through Contingent Bid Auctions." Journal of Management Information Systems<br />
21(2): 201-227.<br />
Bichler, M. (2000). "An Experimental Analysis of Multi-attribute Auctions." Decision Support Systems<br />
29(3): 249-268.<br />
Bichler, M., G. E. Kersten, et al. (2003). "Towards a Structured Design of Electronic Negotiations."<br />
Group Decision and Negotiation 12(4): 311-335.<br />
Branco, F. (1997). "The Design of Multidimensional Auctions." Rand Journal of Economics 28(1):<br />
63-81.<br />
Brynjolfsson, E. and S. Yang (1996). "Information Technology and Productivity: A Review of the<br />
Literature." Advances in Computers 43(1): 179-214.<br />
Bulow, J. and P. Klemperer (1996). "Auctions versus Negotiations." American Economic Review<br />
86(1): 80-194.<br />
Che, Y.-K. (1993). "Design Competition through multidimensional auctions." RAND Journal of Economics<br />
24(4): 668-680.<br />
Chen-Ritzo, C.-H., T. P. Harrison, et al. (2005). "Better, Faster, Cheaper: An Experimental Analysis<br />
of a Multi-attribute Reverse Auction Mechanism with Restricted Information Feedback." Management<br />
Science 51(12): 1753-1762.<br />
Davis, F. D. (1989). "Perceived Usefulness, Perceived Ease of Use and User Acceptance of Information<br />
Technology." MIS Quarterly 13(3): 319-340.<br />
Davis, F. D., R. P. Bagozzi, et al. (1989). "User Acceptance of Information Technology: A Comparison<br />
of Two Theoretical Models." Management Science 35(8): 982-1003.<br />
De Fraja, G. and J. Sakovics (2001). "Walras Retrouve: Decentralized Trading Mechanisms and the<br />
Competitive Price." Journal of Political Economy 109(4): 842-863.<br />
Delaney, M. M., A. Foroughi, et al. (1997). "An Empirical Study of the Efficacy of a Computerized<br />
Negotiaiton Support System." Decision Support Systems 20: 185-197.<br />
DeLone, W. and E. McLean (1992). "Information Systems Success: The Quest for the Dependent<br />
Variable." Information Systems Research 3(1): 60-95.<br />
DeLone, W. H. and E. R. McLean (2003). "The DeLone and McLean Model of Information Systems<br />
Success: A Ten Years Update." Journal of Management Information Systems 19(4): 9-30.<br />
Dishaw, M. T. and D. M. Strong (1996). Explaining Information Technology Utilization with the<br />
Task-technology Fit Conctructs. First INFORMS Conference on Information Systems and Technol-<br />
277
35<br />
ogy, Providence, RI, INFORMS.<br />
Dishaw, M. T. and D. M. Strong (1998). "Assessing Software Maintenance Tool Utilization using<br />
Task–technology Fit and Fitness-for-use Models." Journal of Software Maintenance: Research and<br />
Practice 10(3): 151-179.<br />
Doll, W. J., A. Hendrickson, et al. (1998). "Using Davis's Perceived Usefulness and Ease-of-use Instruments<br />
for Decision Making: A Confirmatory and Multigroup Invariance Analysis." Decision Sciences<br />
29(4): 839-869.<br />
Doll, W. J. and G. Torkzadeh (1988). "The Measurement of End User Computing Satisfaction Issues."<br />
MIS Quarterly 12: 259-274.<br />
Doll, W. J. and G. Torkzadeh (1991). "Issues and Options. The Measurement of End User Computing<br />
Satisfaction: Theoretical and Methodological Issues." MIS Quarterly 15: 5-10.<br />
Friedman, D. and S. S<strong>und</strong>er (1994). Experimental Methods: A Primer for Economists. Cambridge,<br />
Cambridge University Press.<br />
Galinsky, A. D., T. Mussweiler, et al. (2002). "Disconnecting Outcomes and Evaluations: The Role of<br />
Negotiator Focus." Journal of Personality and Social Psychology 83(5): 1131–1140.<br />
Garrity, E. J., B. Glassberg, et al. (2005). "An Experimental Investigation of Web-based Information<br />
Systems Success in the Context of Electronic Commerce." Decision Support Systems 39(3): 485-503.<br />
Gebauer, J. and M. Ginsburg (2006). Exploring the Black Box of Task-Technology Fit: The Case of<br />
Mobile Information Systems, University of Illinois at Urbana-Champaign.<br />
Glass, R. L., V. Ramesh, et al. (2004). "An Analysis of Research in Computing Disciplines." Communications<br />
of the ACM 47(6): 89-94.<br />
Goodhue, D. L. (1995). "Understanding User Evaluation of Information Systems." Management Science<br />
41: 1827-1844.<br />
Goodhue, D. L. and R. L. Thompson (1995). "Task-Technology Fit and Individual Performance."<br />
MIS Quarterly 19(2): 213-236.<br />
Hayek, F. (1945). "The Use of Knowledge in Society." American Economic Review 35(4): 519-530.<br />
Hevner, A. R., S. T. March, et al. (2004). "Design Science in Information Systems Research." MIS<br />
Quarterly 28(1): 75-105.<br />
Hurwicz, L. (1973). "The Design of Mechanisms for Resource Allocation." American Economic Review<br />
63(2): 1-30.<br />
Hyder, E. B., M. J. Prietula, et al. (2000). "Getting to Best: Efficiency vs. Optimality in Negotiation."<br />
Cognitive Science 24(2): 169-204.<br />
Iivari, J. (2003). "The IS Core - VII: Towards Information Systems as a Science of Meta-Artifacts."<br />
Communications of the Association for Information Systems 12(37): 568-581.<br />
Iivari, J. (2005). "An Empirical Test of the DeLone-McLean Model of Information System Success."<br />
278
36<br />
The DATA BASE for Advances in Information Systems 36(2): 8-27.<br />
InterNeg (2006). Negotiations or Auctions. Montreal, InterNeg Research Centre. 2006.<br />
ISR (2002). "Editorial Statement and Policy." Information Systems Research 13(4).<br />
Kim, H. W. and S. L. Pan (2006). "Towards a Process Model of Information Systems Implementation:<br />
The Case of Customer Relationship Management (CRM)." ACM SIGMIS Database 37(1): 59-<br />
76.<br />
Kirkegaard, R. (2004). Auctions versus Negotiations Revisited. Working Paper. Aarhus, Department<br />
of Economics, University of Aarhus.<br />
Kolitz, K. and D. Neumann (2007). The Impact of Information System Design on E-marketplaces. An<br />
Experimental Analysis. Group Decision and Negotiation 2007, Mt. Tremblant, InterNeg.<br />
Koppius, O. and E. van Heck (2002). Information Architecture and Electronic Market Performance in<br />
Multidimensional Auctions. Erasmus Research <strong>Institut</strong>e of Management. Rotterdam, Erasmus University,<br />
Rotterdam: 38.<br />
Kumar, M. and S.I.F. (1999). Internet Auctions. IBM Research Division, T.J. Watson Research Center.<br />
Ledyard, J. (1993). "The Design of Coordination Mechanisms and Organizational Computing." Journal<br />
of Organizational Computing 3(1): 121-134.<br />
Legris, P., J. Ingham, et al. (2003). "Why Do People Use Information Technology? A Critical Review<br />
of the Technology Acceptance Model." Information and Management 40: 191-204.<br />
Leskela, R. L., J. Teich, et al. (2006). Decision Support for Multi-Unit Combinatorial B<strong>und</strong>le Auctions.<br />
Helsinki, Helsinki University of Technology.<br />
Lilien, G. L., A. Rangaswamy, et al. (2004). "DSS effectiveness in marketing resource allocation decisions:<br />
reality versus perception." Information Systems Research 15(3): 216-235.<br />
Lucking-Reiley, D. (2000). "Auctions on the Internet: What's Being Auctioned, and How?" Journal of<br />
Industrial Economics 48(3): 227-252.<br />
March, S. T. and G. F. Smith (1995). "Design and Natural Science Research on Information Technology."<br />
Decision Support Systems 15: 251-266.<br />
Markus, M. L., A. Majchrzak, et al. (2002). "A Design Theory for Systems that Support Emergent<br />
Knowledge Processes." MIS Quarterly 26(3): 179-212.<br />
Maskin, E. and T. Sjöström (2002). Implementation Theory. Handbook of Social Choice and Welfare.<br />
K. J. Arrow, A. Sen and K. Suzumura. Amsterdam, NL, Elsevier Science B.V. 1: 237-288.<br />
McAfee, R. P. and J. McMillan (1987). "Auctions and Bidding." Journal of Economic Literature<br />
25(2): 699-738.<br />
Mount, K. and S. Reiter (1974). "The Informational Size of Message Spaces." Journal of Economic<br />
Theory 8(1): 161-192.<br />
279
37<br />
Nash, J. F. (1954). "The Bargaining Problem." Econometrica 18: 155-162.<br />
Neumann, D., M. Benyoucef, et al. (2003). "Applying the Montreal Taxonomy to State of the Art E-<br />
negotiation Systems." Group Decision and Negotiation 12(4): 287-310.<br />
Nielsen, J. (1994). Usability Engineering, Morgan Kaufmann.<br />
Nunamaker, J. F. J., M. Chen, et al. (1991). "Systems Development in Information Systems Research."<br />
Journal of Management Information Systems 7(3): 89 - 106.<br />
Orlikowski, W. J. and C. S. Lacono (2001). "Research Commentary: Desperately Seeking the 'IT' in<br />
IT Research - A Call to Theorizing the IT Artifact." Information Systems Research 12(2): 121-134.<br />
Palfrey, T. (2001). Implementation Theory. Handbook of Game Theory. R. J. Aumann and S. Hart.<br />
Amsterdam, North-Holland. 3.<br />
Parkes, D. C. (2001). Iterative Combinatorial Auctions:<br />
Achieving Economic and Computational Efficiency. Department of Computer and Information Science,<br />
University of Pennsylvania.<br />
Pikkarainen, K., T. Pikkarainen, et al. (2006). "The Measuremnet of End-user Computing Satisfaction<br />
of online Banking Services: Empirical Evidence from Finland." International Journal of Banki Marketing<br />
24(3): 158-172.<br />
Rangaswamy, A. and K. Starke (2000). Computer-Mediated Neotiations: Review and Research opportunities.<br />
Encyclopedia of Microcomputers. A. Kent and J. G. Williams. New York, NY, Marcel<br />
Dekker. 25: 47-72.<br />
Reiter, S. (1977). "Information and Performance in the (New)² Welfare Economics." American Economic<br />
Review 67(1): 226-234.<br />
Robey, D. and M. Newman (1996). "Sequential Patterns in Information Systems Development: An<br />
Application of a Social Process Model." ACM Transactions on Information Systems 14(1): 30-63.<br />
Roth, A. (2002). "The Economist as Engineer: Game Theory, Experimentation, and Computation as<br />
Tools for Design Economics." Econometrica 70(4): 1341-1378.<br />
Roth, A. E. (1995). Introduction to Experimental Economics. The Handbook of Experimental Economics.<br />
J. H. Kagel and A. E. Roth. Princeton, NJ, Princeton University Press: 3-110.<br />
Rothkopf, M. H., A. Pekec, et al. (1998). "Computationally Manageable Combinatorial Auctions."<br />
Management Science 44(8): 1131-1147.<br />
Satterthwaite, M. and A. Shneyerov (2003). "Convergence of a Dynamic Matching and Bargaining<br />
Market with Two-sided Incomplete Information to Perfect Competition." Working Paper.<br />
Satterthwaite, M. and A. Shneyerov (2006). "Dynamic Matching, Two-sided Incomplete Information,<br />
and Participation Costs: Existence and Convergence to Perfect Competition." Econometrica: in print.<br />
Seddon, P. B., D. S. Staples, et al. (1999). "Dimensions of IS Success." Communications of the AIS<br />
2(20): 1-61.<br />
280
38<br />
Segev, A. and C. Beam (1999). A New Market-based Negotiation Paradigm.<br />
Simon, H. (1996). The Sciences of the Artificial. Cambridge, Massachusetts, The MIT Press.<br />
Simon, H. A. (1986). "Rationality in Psychology and Economics." The Journal of Business 59(4):<br />
209-224.<br />
Smith, V. (1982). "Microeconomic Systems as an Experimental Science." American Economic Review<br />
72(5): 923-955.<br />
Smith, V. (1994). "Economics in the Laboratory." Journal of Economic Perspectives 8(1): 113-131.<br />
Smith, V. (2003). Markets, <strong>Institut</strong>ions and Experiments. Encyclopedia of Cognitive Science. L.<br />
Nadel. London, Nature Publishing Group. 2: 991-998.<br />
Strecker, S. (2004). Multiattribute Auctions in Electronic Procurement - Theory and Experiment -.<br />
Economics and Business Engineering. Karlsruhe, Germany, University of Karlsruhe (TH).<br />
Strecker, S., G. E. Kersten, et al. (2006). Electronic Negotiation Systems: The Invite Prototype. Collaborative<br />
Business MKWI’06, Passau, Germany.<br />
Ströbel, M. and C. Weinhardt (2003). "The Montreal Taxonomy for Electronic Negotiations." Group<br />
Decision and Negotiation 12(2): 143-164.<br />
Subrahmanian, E. and S. N. Talukdar (2004). "Engineering of Markets and Artifacts." Electronic<br />
Commerce: Research and Applications 3(4): 369-380.<br />
Suh, K. S. (1999). "Impact of Communication Medium on Task Performance and Satisfaction: An<br />
Examination of Media-Richness Theory." Information and Management 35: 295-312.<br />
Szajna, B. (1996). "Empirical Evaluation of the Revised Technology Acceptance Model." Management<br />
Science 42(1): 85-92.<br />
Teich, J., H. Wallenius, et al. (2001). "Designing Electronic Auctions: An Internet-Based Hybrid Procedure<br />
Combining Aspects of Negotiations and Auctions." Journal of Electronic Commerce Research<br />
1: 301-314.<br />
Thomas, C. J. and B. J. Wilson (2002). "A Comparison of Auctions and Multilateral Negotiations."<br />
RAND Journal of Economics 33(1): 140-155.<br />
Thomas, C. J. and B. J. Wilson (2005). "Verifiable Offers and the Relationship Between Auctions and<br />
Multilateral Negotiations." The Economic Journal 115 (506): 1016–1031.<br />
Torkzadeh, G. and W. J. Doll (1999). "The Development of a Tool for Measuring the Perceived Impact<br />
of Information Technology on Work." Omega 27(3): 327-339.<br />
Turner, M., D. Budgen, et al. (2003). "Turning Software into a Service." Computer 36(10): 38-44.<br />
Vahidov, R. (2006). Design Researcher's IS Artifact: A Representational Framework. First International<br />
Conference on Design Science Research in Information Systems and Technology (CD-ROM<br />
Proceedings), Claremont, CA.<br />
281
39<br />
van de Ven, A. H. (1992). "Suggestions for Studying Strategy Process: A Research Note." Strategic<br />
Management Journal 13: 169-191.<br />
Venkatesh, V. and F. D. Davis (2000). "A Theoretical Extension of the Technology Acceptance<br />
model: Four Longitudinal Field Studies." Management Science 46(2): 186-204.<br />
Venkatesh, V., M. G. Morris, et al. (2003). "User Acceptance of Information Technology: Toward A<br />
Unified View." MIS Quarterly 27(3): 425-478.<br />
Walls, J. G., G. R. Widmeyer, et al. (1992). "Building an Information System Design Theory for<br />
Vigilant EIS." Information Systems Research 3(1): 36-59.<br />
Weber, M., G. E. Kersten, et al. (2006). "An Inspire ENS Graph is Worth 334 Words, on Average."<br />
Electronic Markets 16(3): 186-200.<br />
Wilson, D. D. (1995). "IT Investment and Its Productivity Effects: An Organizational Sociologist's<br />
Perspective on Directions for Future Research." Economics of Innovation and New Technology 3(3-<br />
4): 235-251.<br />
Wixom, B. H. and P. A. Todd (2005). "A Theoretical Integration of User Satisfaction and Technology<br />
Acceptance." Information Systems Research 16(1): 85–102.<br />
Wolfstetter, E. (1995). "Auctions: An Introduction." Journal of Economic Surveys 10(4): 367-420.<br />
Wolfstetter, E. (2000). Topics In Microeconomics. Industrial Organization, Auctions, and Incentives.<br />
Cambridge, Cambridge University Press.<br />
Yi, Y. (1990). A Critical Review of Consumer Satisfaction. Review of Marketing. V. A. Zeithaml.<br />
Chicago, American Marketing Association: 68-129.<br />
Yu, B. (2007). Negotiations or Auctions: Experimental Comparison of Two E-market Mechanisms. J.<br />
Molson School of Business. Montreal, ConcordiaUniversity: 138.<br />
Zhu, K. (2004). "The Complementarity of Information Technology Infrastructure and E-Commerce<br />
Capability: A Resource-Based Assessment of Their Business Value." Journal of Management Information<br />
Systems 21(1): 167 - 202.<br />
Zwass, V. (1996). "Electronic Commerce: Structures and Issues." International Journal of Electronic<br />
Commerce 1(2): 3 - 23.<br />
282
40<br />
Appendix<br />
Information<br />
quality<br />
System reliability<br />
Usefulness<br />
Ease of use<br />
Satisfaction<br />
with outcome<br />
Cognitive effort<br />
Satisfaction<br />
with process<br />
Table A. Factor analysis of subjective variables 1<br />
SR U IQ SO EU CE SP<br />
IQ1 .162 .337 .791 .134 .009 .128 .057<br />
IQ2 .204 .263 .836 .135 .079 .100 .136<br />
IQ3 .267 .039 .649 .029 .377 .104 .175<br />
IQ4 .230 .194 .762 .113 .275 .113 .117<br />
SR1 .742 .134 .208 .094 .267 ‐.006 .186<br />
SR2 .794 .187 .207 .027 .211 ‐.032 .128<br />
SR3 .801 .176 .161 .109 .059 .104 .036<br />
SR4 .812 .220 .143 .081 .084 .097 ‐.023<br />
U1 .236 .764 .252 .101 .124 .178 .185<br />
U2 .178 .834 .210 .203 .156 .125 .109<br />
U3 .302 .750 .137 .176 .233 .114 .098<br />
U4 .172 .733 .269 .114 .189 .083 .238<br />
EU1 .225 .270 .157 .067 .776 .163 .150<br />
EU2 .321 .276 .088 .046 .722 .135 .159<br />
EU3 .084 .086 .215 .071 .839 .131 .049<br />
SO1 .123 .102 .088 .889 .043 .111 .136<br />
SO2 .066 .176 .113 .859 .050 .122 .120<br />
SO3 .064 .123 .092 .872 .067 .072 .074<br />
CE1 ‐.056 .100 .095 .137 .150 .855 .058<br />
CE2 .066 .084 .241 .163 .015 .797 .165<br />
CE3 .192 .217 ‐.007 .019 .266 .729 .225<br />
SP1 .013 .109 .106 ‐.028 .310 .078 .698<br />
SP2 .111 .237 .147 .321 ‐.057 .165 .710<br />
SP3 .142 .149 .118 .175 .076 .197 .753<br />
Reliability<br />
(Cronbach<br />
alpha)<br />
0.879<br />
0.874<br />
0.895<br />
0.858<br />
0.896<br />
0.708<br />
0.809<br />
1 Factor analysis was rotated by Varimax method with Kaiser normalization.<br />
All seven factors have eigenvalues above 1.<br />
283
Comparing Ingress and Egress Detection to Secure<br />
Inter-domain Routing: An Experimental Analysis<br />
Christoph Goebel<br />
<strong>Institut</strong>e of Information Systems<br />
Humboldt Universitaet zu Berlin<br />
10178 Berlin<br />
Germany<br />
Dirk Neumann<br />
<strong>Institut</strong>e of Information Systems and Management<br />
University of Karlsruhe<br />
76131 Karlsruhe<br />
Germany<br />
Ramayya Krishnan<br />
H. John Heinz III School of Public Policy and Management<br />
Carnegie Mellon University<br />
Pittsburgh, PA 15213-3890<br />
USA<br />
July 18, 2007<br />
Abstract<br />
The global economy and society is increasingly dependent on computer networks linked<br />
together by the Internet. The importance of networks reaches far beyond the telecommunications<br />
sector since they have become a critical factor for many other crucial infrastructures<br />
and markets. With threats mounting and security incidents becoming more frequent, concerns<br />
about network security grow.<br />
It is an acknowledged fact that some of the most f<strong>und</strong>amental network protocols that make<br />
the Internet work are exposed to serious threats. One of them is the Border Gateway Protocol<br />
(BGP) which determines how Internet traffic is routed through the topology of administratively<br />
independent networks the Internet is comprised of. Despite the existence of a steadily<br />
growing number of BGP security proposals, to date neither of them is even close to being<br />
adopted.<br />
The purpose of this work is to contemplate BGP security from a theoretical point of view<br />
in order to take a first step toward <strong>und</strong>erstanding the factors that complicate secure BGP<br />
adoption. Using a definition of BGP robustness we experimentally show that the degree of<br />
robustness is distributed unequally across the administrative domains of the Internet, the<br />
so-called Autonomous Systems (ASs). The experiments confirm the intuition that the contribution<br />
ASs are able to make towards securing the correct working of the inter-domain<br />
routing infrastructure by deploying countermeasures against routing attacks differ depending<br />
on their position in the AS topology. We also show that the degree of this asymmetry<br />
can be controlled by the choice of the security strategy. We compare the strengths and<br />
weaknesses of two f<strong>und</strong>amentally different approaches in increasing the BGP’s robustness<br />
which we termed ingress and egress detection of false route advertisements and indicate<br />
their economic implications.<br />
1 284
1 Introduction<br />
The global society and economy is increasingly dependent on the effective working of communication<br />
networks. These networks have created connectivity linking together millions of users<br />
including private users, businesses and public institutions. They also have become a critical<br />
factor for infrastructures (e.g. water and electricity supply) and markets other than the telecommunications<br />
market (e.g. the worldwide financial market) [2, 23]. Consequently, concerns about<br />
the security of these global networks and information systems have been growing along with the<br />
rapid increase in the number of users and the importance of the information transmitted.<br />
Although targeted attacks on the confidentiality of information assets and the availability of particular<br />
services have been at the center of public attention so far, the Internet infrastructure as<br />
a whole suffers from more f<strong>und</strong>amental vulnerabilities due to its design. Security considerations<br />
did not play a major role when the f<strong>und</strong>amental protocols were devised, which still orchestrate<br />
global data flows in today’s Internet.<br />
Ironically, the same mechanisms that have contributed substantially to the success of many<br />
Internet protocols have severe drawbacks when it comes to securing them against malicious<br />
activities. In the absence of a central authority, the different components of the Internet infrastructure<br />
rely on distributed control protocols. This cooperative control effort can only lead<br />
to the desired outcome if every control unit behaves in the way it is supposed to, i.e. behave<br />
according to the protocol it implements. Consequently, the misbehavior of one control unit can<br />
have adverse effects on the way the infrastructure works as a whole [16].<br />
One of the key Internet protocols suffering from a f<strong>und</strong>amental lack of security is the Border<br />
Gateway Protocol (BGP). The BGP is the core routing protocol of the Internet. It enables the<br />
exchange of reachability information based on the Internet Protocol (IP) across administratively<br />
independent networks, the so-called Autonomous Systems (ASs).<br />
In this work we address the vulnerability of the BGP to routing attacks. This is a particularly<br />
virulent type of attack targeted at the Internet infrastructure as a whole. The contributions of<br />
this paper are threefold. Firstly, a new countermeasure approach, egress detection, is motivated.<br />
Secondly, a simulation design is proposed that has the potential to gauge the effectiveness of<br />
alternative countermeasures. Thirdly, a rigorous analysis showing the properties of egress detection<br />
and comparing it with state-of-the-art ingress detection approaches. This research is<br />
important, as it supports the design of a viable strategy to secure the global routing architecture.<br />
This work is original as we adopt an abstract approach that allows us to compare the efficiency<br />
of different strategies on a general level. Our approach simplifies the problem of determining the<br />
degree of routing security by not making too detailed assumptions. Closely related earlier work<br />
by Chan et. al [4] on the topic attempts to compare details in the protocol, which is hampered<br />
by the fact that the introduction of a large number of additional assumptions increases the<br />
complexity of the model and therefore both validation and interpretation of simulation results.<br />
This work is conceived to be a primer for better <strong>und</strong>erstanding the dominant utility effects<br />
<strong>und</strong>erlying the adoption process, which will fuel analyses on the protocol level. Section 2 gives<br />
an overview of related work. In section 3 we provide an overview of inter-domain routing, routing<br />
attacks and countermeasures. In section 4 we present a basic approach to measure routing<br />
security based on previous work by Chan et al. [4]. This measurement methodology is applied<br />
to artificial Internet topologies in section 5 in order to conduct an extensive sensitivity analysis<br />
both revealing f<strong>und</strong>amental coherences and comparing the effects of two different approaches<br />
to secure inter-domain routing (ingress and egress detection) on two different security metrics<br />
(one based on robustness and the other one based on semi-robustness). In sections 6 and 7 we<br />
summarize our results, conclude and provide an outlook for future research.<br />
2 285
2 Related Work<br />
An impressive amount of work has been devoted to the specification of different BGP security<br />
protocols recently. S-BGP, the first and most comprehensive proposal was made by Kent et<br />
al. [13]. It uses several Public Key Infrastructures (PKIs) to enable BGP routers to authenticate<br />
outgoing route announcements and validate incoming ones. Inter-domain hops are signed<br />
recursively as paths are constructed in the forwarding process described in section 3. Since<br />
S-BGP implies a significant protocol overhead, other authors tried to develop less expensive<br />
solutions. For instance, White et al. [24] propose a centralized heuristic solution to path validation<br />
that has the potential to reduce complexity and overhead significantly but also leads to less<br />
accurate detection of falsified routes. Unlike S-BGP, which uses a push-approach to distribute<br />
authentication information, IRV allows for authentication on-demand using a dedicated server<br />
infrastructure (pull-approach) [8]. Hu et al. [10] have proposed SPV which reduces security overhead<br />
by substituting expensive but concise public key cryptography by less expensive but more<br />
complicated procedures based on symmetrical primitives and one-time signatures. Additional<br />
work on inter-domain routing security includes research on ways how to make a security protocol<br />
proposed earlier more efficient [17, 12], the specification of less comprehensive approaches<br />
to BGP security [21, 26], and the advancement of causal research and attack detection methods<br />
[27, 14].<br />
Chan at al. [4] have raised the issue of adoptability of secure BGP protocols. They provide a<br />
taxonomy of properties in order to classify the numerous secure BGP schemes that have been<br />
proposed so far. Based on these properties they model the adoption process and compare the<br />
adoption thresholds of the different protocols <strong>und</strong>er numerous assumptions, e.g. regarding attacker<br />
capability and traffic load. Although our approach to measure the degree of security of<br />
AS paths follows the lines laid out by Chan et al., the level of analysis differs. While Chan et al.<br />
dive into the analysis of different protocol classes based on detailed protocol specifications, we<br />
follow a more general way exploring the pros and cons of two different security strategies. The<br />
reduction of protocol details on the one hand and the abstraction from the security protocol<br />
level to the security strategy level on the other hand allow us to make more general statements.<br />
3 Backgro<strong>und</strong><br />
3.1 Inter-domain Routing and the BGP<br />
The Internet is a network of networks. It is currently comprised of more than 20,000 interconnected<br />
computer networks, the so-called Autonomous Systems (ASs). We will refer to this network<br />
of ASs as the AS topology. The physical coverage of an AS can vary substantially: Large<br />
backbone providers like AT&T own physical networks that have Points of Presence (PoPs) 1<br />
aro<strong>und</strong> the globe, whereas a university network is usually confined to some campus area.<br />
The AS topology is a scale-free topology, i.e. a relatively small subset of the ASs, the transit<br />
ASs, forward the traffic originated elsewhere, while most ASs, the stub ASs, serve as mere traffic<br />
end-points. Physically, the stub ASs depend on the transit ASs to carry their traffic over long<br />
distances.<br />
Inter-domain routing refers to the path selection process on the AS level. The outcome of this<br />
process is a complete prescription of how IP traffic is forwarded through the AS topology. The<br />
information required to compute this global routing table is communicated by dedicated routers<br />
implementing an inter-domain routing protocol. The de-facto standard protocol used for interdomain<br />
routing is the Border Gateway Protocol (BGP) [20]. The routers which implement the<br />
1 A Point of Presence (PoP) is a place where an AS provides the required physical infrastructure to establish<br />
connections with other ASs.<br />
3 286
1.1.1.0/24<br />
r51<br />
AS5<br />
1.1.1.0/24 r51 AS5<br />
AS1<br />
r11<br />
1.1.1.0/24 r41 AS4_AS5<br />
AS4<br />
r41<br />
r42<br />
1.1.1.0/24 r21 AS2_AS4_AS5<br />
r21<br />
r22<br />
1.1.1.0/24 r42 AS4_AS5<br />
AS2<br />
r23<br />
1.1.1.0/24 r31 AS3_AS4_AS5<br />
r31<br />
AS3<br />
Figure 1: Propagation of route advertisements according to the BGP.<br />
BGP are oftentimes denoted as BGP speakers.<br />
The BGP belongs to the family of path vector protocols, i.e. the basic information exchanged<br />
by the BGP speakers is comprised of (destination,path)-tuples. 2 The destinations are ranges<br />
of IP addresses (oftentimes denoted as prefixes) and the paths are concatenations of AS identifiers.<br />
IP ranges and AS numbers are allocated by the Internet Corporation for Assigned Names<br />
and Numbers (ICANN) or the allocation task is delegated to local Internet registries. If an AS<br />
originates a certain prefix, this information is passed on to the BGP speakers in adjacent ASs<br />
and so forth. Consequently, each BGP speaker eventually receives a number of routes leading to<br />
the same address space. BGP speakers performs a standardized path selection process in order<br />
to determine which of these paths will be added to their local routing table. When the BGP<br />
has reached a steady state, each router knows for any IP address which adjacent router an IP<br />
packet needs to be forwarded to reach its destination. It is important to note that currently<br />
only selected paths are forwarded to other BGP speakers, not all of them.<br />
The BGP path selection process takes a large number of criteria into consideration. A comprehensive<br />
description of the path selection process can be fo<strong>und</strong> in [20]. One of the most important<br />
criteria is path length (in terms of AS hops). The shorter the path, the higher the probability<br />
that it is chosen by a router.<br />
Example 1: Propagation of Route Advertisements<br />
If nothing but the shortest path criterion were used in the path selection process,<br />
the prefix 1.1.1.0/24 in figure 1 would be propagated in the following way:<br />
The prefix 1.1.1.0/24 is owned and originated by AS5. Accordingly, router r51 announces<br />
this information to router r42 in the adjacent AS4. AS4 is connected to<br />
AS2 and AS3 and forwards the advertisement over the BGP sessions between the<br />
routers r42 and r31 and the routers r41 and r22. This process continues until all<br />
2 The current specification of the BGP encompasses many more attributes which we cannot describe here due<br />
to space constraints.<br />
4 287
outers are aware of the fact that AS5 originates prefix 1.1.1.0/24 and a corresponding<br />
inter-domain path. Each time the route advertisement crosses an AS border,<br />
an AS identifier is appended to the path. Router r21 receives two different paths<br />
leading up to the same prefix. Considering the shortest path criterion, it will prefer<br />
the path AS4 AS5 over the path AS3 AS4 AS5. Only the announcement containing<br />
the shorter path is then forwarded to router r11 in AS1.<br />
After the routing tables maintained by the routers have reached a converged state,<br />
all traffic from AS1 to AS5 will travel on the path leading through AS2 and AS4,<br />
which is emphasized in figure 1.<br />
3.2 BGP Routing Attacks<br />
It is widely known that the BGP is vulnerable to a whole range of possible attacks [16]. When<br />
considering attacks on a complex system like the inter-domain routing infrastructure, it is advisable<br />
to put oneself into the position of an attacker and explore possible ways how to achieve<br />
certain attack goals. Using the scope of an attack’s impact as a means of differentiation, we<br />
distinguish two general types of attacks on the BGP: Attacks on single BGP sessions (small scale<br />
attacks) and attacks intended to affect the global routing infrastructure as a whole (large scale<br />
attacks) [3]. Particular sessions may be attacked in order to terminate them or to eavesdrop on<br />
the control messages exchanged. Carrying out large scale attacks on the routing infrastructure<br />
may, for instance, be conducted with the intention to alter the global routing table, i.e. in order<br />
to change the paths inter-domain traffic travels on. We denote this kind of attack a routing<br />
attack. While the ultimate goal of routing attacks can be manifold, their consequences mostly<br />
include disruption of reachability, so that effective traffic delivery is prevented [3].<br />
Both small and large scale attacks can only be carried out if the attacker finds a way to gain control<br />
over BGP speakers or to at least tamper with BGP control messages on their way from one<br />
BGP speaker to another. Thus, BGP security is inherently linked to classical security problems<br />
concerning host and network security. If we assume that attackers are always in the position to<br />
control one or several BGP speakers, providing BGP security appears to be a question of achieving<br />
stronger protocol robustness. Generally, a distributed protocol is robust, if the misbehavior<br />
of one or several of the entities do not have any negative impact on the outcome.<br />
Taking into account the distributed nature of the Internet infrastructure, router and session<br />
security can only be considered as a first line of defense. This is because as long as humans<br />
are involved, computer and network security can never be fully guaranteed, no matter how sophisticated<br />
the security technologies and policies are. Security technology should at least aim<br />
at putting the right ’speed bumps’ into place, such that the velocity and impact of electronic<br />
attacks are decreased to a level where other protection mechanisms can operate [18]. 3<br />
In terms of security, inter-domain routing exhibits weakest link properties: To date, if an attacker<br />
succeeds in compromising the BGP speakers of one AS, many other ASs may be hurt by<br />
a routing attack. Therefore, even the deployment of host and network security on the Internet<br />
backbone can only reduce the probability of a threat occurance, it cannot effectively delimit<br />
the extent of the damage caused in case the threat is realized. As a consequence, securing the<br />
inter-domain routing infrastructure requires on the one hand, enhancing the security of BGP<br />
sessions and routers and on the other hand making the protocol itself more robust.<br />
In this paper, we concentrate on two important types of routing attacks, namely prefix hijacking<br />
and path spoofing. Prefix hijacking takes advantage of the fact that current BGP implementations<br />
do not prevent a compromised or miss-configured router from illegitimately announcing<br />
prefixes it does not originate. The router in adjacent ASs are not able to distinguish the real<br />
3 Intentional misbehavior (i.e. attacks) is only one part of the BGP security problem. In fact, the incidents that<br />
have been recorded so far were due to unintentional misconfiguration of BGP speakers [15]. The configuration of<br />
BGP speakers is error-prone, since it is still done manually to a significant extent. Unintentional misconfiguration<br />
cannot be prevented by more rigid access controls or the protection of BGP sessions.<br />
5 288
1.1.1.0/24<br />
r51<br />
AS5<br />
1.1.1.0/24 r51 AS5<br />
AS1<br />
r11<br />
1.1.1.0/24 r41 AS4_AS5<br />
AS4<br />
r41<br />
r42<br />
legitimate advertisement:<br />
1.1.1.0/24 r21 AS2_AS4_AS5<br />
illegitimate advertisement:<br />
r21<br />
r22<br />
1.1.1.0/24 r42 AS4_AS5<br />
1.1.1.0/24 r21 AS2_AS3<br />
AS2<br />
r23<br />
hijacked prefix<br />
legitimate advertisement:<br />
1.1.1.0/24 r31 AS3_AS4_AS5<br />
illegitimate advertisement:<br />
1.1.1.0/24 r31 AS3<br />
r31<br />
1.1.1.0/24<br />
AS3<br />
Figure 2: Prefix Hijacking<br />
from the fake owner, leaving a significant probability that the fake advertisement is selected and<br />
subsequently forwarded. Since AS path length is one of the most important criteria in the best<br />
path selection algorithm, the fake announcements are more likely to be adopted by routers in<br />
ASs which are further away from the legitimate originator of the prefix. 4 Let us illustrate the<br />
working of prefix hijacking.<br />
Example 2: Prefix Hijacking<br />
Using the same topology as in example 1, suppose that router r31 in AS3 has been<br />
compromised and is used for a prefix hijacking attack (figure 2). It injects a route<br />
announcement claiming that AS3 originates the prefix 1.1.1.0/24. All routers located<br />
in ASs other than AS5 and AS3 not know which AS legitimately originates<br />
the prefix. Router r21 cannot validate these route advertisements and will therefore<br />
be likely to adopt the advertisement containing the shorter path. Consequently, the<br />
traffic that previously used to travel on the inter-domain path leading over AS2 and<br />
AS4 will now be forwarded to AS3. Hence the traffic will never be delivered to the<br />
intended destination.<br />
Path spoofing works in a similar way as prefix hijacking but with the slight difference that not<br />
the originating AS but the remaining ASs present on the AS path are faked. A router can<br />
change route announcements received from other routers by inserting or deleting ASs from the<br />
path attribute. Malicious ASs can take advantage of this possibility forcing traffic to make a<br />
detour.<br />
4 If an AS illegitimately claims to be the origin of some IP address space and the prefix it announces is more<br />
specific than the one announced by the legitimate AS, prefix hijacking is particularly effective. Since BGP performs<br />
longest prefix matching, prefixes that are more de-aggregated, i.e. longer, are preferred. Prefix de-aggregation<br />
forces routers to adopt the fake announcement even if they have received the legitimate announcement on a shorter<br />
path before. We do not consider this particular type of prefix hijacking in this paper.<br />
6 289
Example 3: Path Spoofing<br />
Considering the scenario depicted in figure 2, router r31 could apply path spoofing<br />
by faking a direct connection to AS5. This results in forwarding the advertisement<br />
1.1.1.0/24 r31 AS3 AS5 to router r23. Since r21 would then receive two paths with<br />
equal length leading to the same origin, there is a good chance that this path spoofing<br />
attack has the same effect as the prefix hijacking attack described above.<br />
3.3 BGP Security Countermeasures: Ingress and Egress Detection<br />
As described above, the protection of BGP routers and sessions is an important first step toward<br />
securing the inter-domain routing infrastructure. AS operators have begun to deploy security<br />
mechanisms in their networks, e.g. the BGP TTL Security Hack [7] and the MD5 signature option<br />
[9]. However, the f<strong>und</strong>amental lack of robustness inherent to the BGP cannot be removed<br />
by these individual security efforts. Effective protection against large scale attacks requires a<br />
concerted security effort by many AS operators [3].<br />
We describe two different approaches to prevent illegitimate route announcements from adversely<br />
affecting inter-domain routing. The first enables BGP speakers to validate the legitimacy of received<br />
route announcements. The second prevents BGP speakers from announcing illegitimate<br />
routes. In the following we will call these two approaches ingress and egress detection, respectively.<br />
3.3.1 Ingress Detection<br />
The need for a comprehensive BGP security architecture has lead to an ab<strong>und</strong>ance of proposals<br />
[3]. Almost all of them follow the ingress detection approach, as they enable routers implementing<br />
the same ingress detection protocol to exchange authoritative routing information. This<br />
information can be used by routers to validate incoming route advertisements, e.g. to check if<br />
the origin of an announced route is correct. Comparing two of the most discussed comprehensive<br />
BGP security proposals, S-BGP [13] and IRV [8], we find that both of them basically call for<br />
an additional communication channel between BGP speakers. Although different techniques<br />
are applied to facilitate the transfer of authentication data, the degree of security achieved is<br />
approximately the same [3].<br />
3.3.2 Egress Detection<br />
Hitherto, egress detection has not received much attention as a viable solution to the routing<br />
security problem so far - albeit the general idea ’taking care of one’s own front garden’ is<br />
certainly not new. An implementation of egress detection includes some security mechanism<br />
that prevents or at least complicates the announcement of false routes. For instance, the routes<br />
being announced by an AS could be double-checked by a trusted and secure third party within<br />
the AS before they are released. The trusted instances could in turn be bo<strong>und</strong> to validate all<br />
route announcements with certificates distributed through a PKI as proposed by Aiello et. al [1].<br />
Egress detection has the advantage that no additional communication channels between routers<br />
located in different ASs would be needed. Therefore, issues such as communication overhead<br />
and Internet-wide protocol standardization can be avoided.<br />
An obvious disadvantage of egress detection is that it cannot be used by a subset of ASs to<br />
achieve guaranteed routing security of the connecting inter-domain paths. With ingress detection<br />
deployed in all ASs on a path, none of the legitimate traffic flowing on that path can be diverted<br />
by prefix hijacking or path spoofing. However, this does not imply that egress detection performs<br />
worse with respect to the number of paths that can be secured given a set of adopters. The<br />
protection effort simply cannot be targeted onto particular routes the way it could be if ingress<br />
detection was used as a countermeasure.<br />
7 290
The egress detection approach appears to be particularly attractive regarding the fact that many<br />
false announcements are actually caused by the unintended misconfiguration of routers and not<br />
by malicious behavior [15, 14].<br />
4 Experimental Setup<br />
This section introduces a basic quantitative methodology to measure the security level interdomain<br />
routing. Using this measurement method, we can experimentally show important features<br />
of inter-domain routing security depending on the employed countermeasure strategy,<br />
either ingress or egress detection. We focus on topological characteristics and the dynamics of<br />
selected security metrics depending on the number of secure BGP adopters.<br />
4.1 Preliminaries<br />
Emphasizing the interdependencies of topological features, router configurations and policies,<br />
the whole path selection process needs to be taken into account. As earlier research has shown,<br />
models of inter-domain routing which equate routing and administrative domains do not yield<br />
accurate results. BGP speakers would have to be modeled independently from each other in<br />
order to reconstruct or accurately predict inter-domain paths [5]. Furthermore, the impact of<br />
intra-domain routing on inter-domain paths needs to be explicitly taken into account [19]. 5<br />
Policy routing is another major factor determining how traffic is routed through the AS topology<br />
[22]. The business relationships between AS operators find their expression in the way traffic<br />
engineering is done: The standardized path selection process running on every BGP router is<br />
influenced by ingress and egress filters configured in a way that business objectives are achieved.<br />
One typical routing policy implemented by transit AS operators consists of announcing only<br />
customer routes to peers minimizing their own transit fees [11].<br />
Clearly, the emphasis of this work is not on the prediction of realistic inter-domain paths. As<br />
this would require the inclusion of router-level topologies, intra-domain routing mechanisms and<br />
most importantly as many protocol details as possible into the model. The purpose of our<br />
work is to quantify the effects illegitimate route announcements have on the robustness of interdomain<br />
routes. This study is designed to reveal how effective ingress and egress detection are<br />
in preventing the most straight-forward routing attacks depending on how many ASs support<br />
ingress and egress detection, on their position in the network and on the order of deployment.<br />
With respect to topology detail the model is restricted to the AS level, ignoring the effects<br />
of individual router behavior and intra-domain routing. For simplicity, we make the following<br />
assumptions on the simulation set-up. The path selection process is reduced to a single but<br />
crucial criterion, namely the AS path length.<br />
The routers in each AS are aware of one AS path leading from their AS to all other ASs in the<br />
Internet, i.e. n − 1 paths in total where n is the number of ASs in the Internet. There exist<br />
n(n − 1) AS paths at a time, which form the global routing table.<br />
4.2 Threat Model<br />
Operationalizing our questions, we will set up a basic threat model. In our model attackers<br />
are assumed to have two basic techniques at their disposal, prefix hijacking and path spoofing.<br />
In the absence of any countermeasures deployed in the ASs on the path, the success of either<br />
attack depends on the shape of the topology in the region aro<strong>und</strong> that path. The attack can be<br />
5 Intra-domain routing refers to the way IP traffic is routed through the network topology of an AS. Intradomain<br />
routing also has an important influence on the way inter-domain paths are selected because it determines<br />
at which border gateway router an IP packet leaves the AS. The chosen egress point can in turn have a significant<br />
impact on the inter-domain path a packet travels on in the following.<br />
8 291
thwarted if certain ASs on the path implement ingress detection or if egress detection is implemented<br />
by the infiltrated AS. Let us explain these relationships using the generic configuration<br />
depicted in figures 3 and 4.<br />
m<br />
m<br />
. . .<br />
. . .<br />
j ... p k<br />
. . .<br />
. . .<br />
j p‘ k<br />
p ...<br />
. . .<br />
alternative path<br />
. . .<br />
alternative path<br />
(a)<br />
(b)<br />
Figure 3: Prefix Hijacking<br />
Node m represents the compromised AS. Consider a legitimate path from AS j to AS k emphasized<br />
by a bold line, where AS k legitimately advertises a prefix. If no countermeasures<br />
have been implemented, the nodes in the network cannot distinguish the legitimate originator<br />
k from the illegitimate originator m and will thus accept the announcement from the AS which<br />
is closer in terms of AS hops. We assume that if m announces a path which is as long as the<br />
legitimate path, the nodes will pick the illegitimate path. 6 An attacker controlling AS m can<br />
thus change the paths if all ASs on the bold path except k because at least AS p will accept m’s<br />
announcements and forward all traffic intended for k to m instead. If AS m has deployed egress<br />
filtering, the path is totally secure from attacks using m’s routers: All route advertisements are<br />
’double-checked’ using authoritative information kept by AS m before they leave it and false<br />
ones are prevented from passing the egress filter. Now consider the case when both, j and k<br />
use ingress detection. This boils down to j and k being able to communicate authentication<br />
information. Therefore AS j knows who is the legitimate originator of the prefix and will therefore<br />
choose a path other than the bold path leading to AS k given it is aware of such a path.<br />
This result can be generalized: Falsified AS paths connecting two endpoints will never carry the<br />
traffic between those endpoints if both implement ingress detection. This is only a first step<br />
towards routing security, however. Note that the attacker has at least succeeded in preventing<br />
the traffic from flowing on the original path. 7 This is why we define this security property of a<br />
legitimate inter-domain path semi-robustness.<br />
Definition : Semi-Robustness of Inter-domain Paths<br />
A legitimate inter-domain path is semi-robust if it cannot be changed in a way that its traffic<br />
passes through or terminates in the AS that launched the routing attack. Either there exists an<br />
alternative path that can be used instead, or the corresponding traffic does not flow at all.<br />
Complete path robustness can only be assured if all ASs on the bold path that have deployed<br />
ingress filtering and at the same time no AS is connected to m via a shorter AS path than with<br />
k. We define inter-domain path robustness as follows:<br />
Definition : Robustness of Inter-domain Paths<br />
6 Note that this can be considered a worst case assumption. Since we only consider the relative results of our<br />
measurements, this decision is not very important with respect to our conclusions.<br />
7 Either the router at the two ends of the AS path know an alternative legitimate path through the AS topology<br />
or the connection is disrupted.<br />
9 292
A legitimate inter-domain path is robust if it is entirely resistant to the invalid behavior of<br />
routers located in ASs that are not positioned along the path.<br />
Inter-domain path robustness trivially implies inter-domain path semi-robustness: If an AS path<br />
is robust, the traffic flowing on that path cannot be diverted such that it is impossible not to<br />
reach the destination. The reverse is, however, not true.<br />
m<br />
m<br />
. . .<br />
. . .<br />
spoofed link<br />
. . .<br />
. . .<br />
spoofed link<br />
j ... p k<br />
j<br />
...<br />
p<br />
q<br />
k<br />
. . .<br />
alternative path<br />
. . .<br />
alternative path<br />
(a)<br />
(b)<br />
Figure 4: Path Spoofing<br />
Figure 4 depicts the generic situation in which an attacker uses path spoofing to divert traffic<br />
from its legitimate path. AS m announces routes indicating that it is directly connected to k<br />
while in reality it is not. Figure 4a) illustrates that path spoofing results in almost the same<br />
threat for the legitimate path except that the distance between mislead AS j and m has to be at<br />
least one AS hop smaller than the distance between AS j and k. Consequently, the situations in<br />
which path spoofing is successful are less numerous than the situations in which prefix hijacking<br />
is successful. If ingress detection was deployed both in j and k, k could tell j that it is not<br />
directly connected to m and j would therefore not accept route announcements claiming that<br />
this is the case. AS j would then forward traffic destined to k via another path. In fact, the<br />
attacker would have to spoof a direct connection to the nearest AS on the legitimate path that<br />
has not deployed ingress detection to maximize the chances of a successful attack. The legitimate<br />
path can only be completely secured against path spoofing attacks if all ASs on the legitimate<br />
route, except the two ASs that are one and two hops away from k, implement ingress detection.<br />
4.3 Security Metrics<br />
Based on the threat model we can now formally define the security metrics we use to determine<br />
the security level of each AS in the topology.<br />
Consider the generic configurations depicted in figures 3 and 4; false route announcements from<br />
AS m can have adverse effects on the AS path connecting AS j and k. Let the AS topology<br />
by denoted by G(V, E) where V stands for the set of ASs and E for the set of connections in<br />
the topology. Based on this topology the global routing table is constructed. We represent this<br />
construction process as a function denoted by Ψ. Consequently, Ψ maps the topology G(V, E)<br />
on a global routing table satisfying the shortest path criterion. Let ψ denote a possible global<br />
routing table, i.e. the relation Ψ(G(V, E)) ↦→ ψ holds. Any intermediate AS p on a legitimate<br />
path from AS i to AS j therefore has the property p ∈ R ij where R ij is given by ψ.<br />
We define a binary function describing the security level of that path, which takes k, j, m,<br />
G(V, E) and ψ as arguments.<br />
Definition : Security Indicator Function<br />
The function sec(j, k, m, G(V, E), ψ) is equal to 1 if false route announcements injected by BGP<br />
speakers in m will have no influence at all on the legitimate path from j to k, and equal to 0<br />
10 293
otherwise.<br />
We now define the probability that a legitimate path from j to k is vulnerable to an attack from<br />
m. It is implicitly assumed that all ASs are equally likely to be used for routing attacks. The<br />
probability is referred to as the route security metric in the following way.<br />
Definition : Route Security Metric<br />
s route (j, k, G(V, E), ψ) = 1 ∑<br />
n m∈V,m/∈R jk<br />
sec(j, k, m, G(V, E), ψ)<br />
Using the route security metric we can measure the security of all n(n − 1) routes provided<br />
the network topology G(V, E) and an instance of the global routing table ψ determined by the<br />
function Ψ.<br />
We compute the security metric describing the security level of AS i by averaging the route<br />
security metrics of the (n − 1) routes leading form i to all other ASs j ∈ V and back.<br />
Definition : AS Security Metric<br />
s node (i, G(V, E), ψ) = 1<br />
n−1<br />
∑<br />
j∈V,j≠i sroute (i, j, G(V, E), ψ)<br />
Regarding these definitions the security measurement model can be represented as a function S<br />
mapping the (n × 1)-vector of binary security protocol adoption decisions to another (n × 1)-<br />
vector of node security metrics s node .<br />
Definition : Security Measurement Function<br />
S(e, G(V, E), ψ) = s node where e ∈ {0, 1} n and s node ∈ [0, 1] n<br />
The route security metric is apparently a worst case metric, as it only assesses a route as secure<br />
if it cannot be influenced by any means. In contrast to that the AS security metric is not a worst<br />
case metric, as it rather represents the expected value of the event that the communication of a<br />
particular AS is not affected by false routes announcements.<br />
Example 4: Calculation of the Security Metrics<br />
We demonstrate the calculation of the security measurement function for prefix hijacking<br />
<strong>und</strong>er the requirement of full path robustness using the topology introduced<br />
in the earlier examples (n = 5). Assume that AS1 and AS5 have deployed ingress<br />
detection. In order to calculate the AS security metric for AS1 we need to compute<br />
the route security metric of all four legitimate routes which connect AS1 to the four<br />
other ASs. The calculation of the route security metric of the legitimate path from<br />
AS1 to AS2 implies evaluation of the security indicator function three times using<br />
m = {AS3, AS4, AS5}. It does not have to be evaluated for m = AS1, AS2 since<br />
these ASs lie on the legitimate path. It turns out that AS1’s prefixes can be hijacked<br />
using AS3 and AS4, but not using AS5.<br />
Using the formula for the route security metric we get<br />
s route (AS1, AS2, G(V, E), ψ) = 1 5<br />
= 0.6<br />
∑<br />
m∈{AS3,AS3,AS5}<br />
sec(AS1, AS2, m, G(V, E), ψ)<br />
Applying the same reasoning to the remaining three legitimate AS paths we obtain<br />
s route (AS1, AS3, G(V, E), ψ) = 0.4<br />
s route (AS1, AS4, G(V, E), ψ) = 0.6<br />
s route (AS1, AS5, G(V, E), ψ) = 0.8<br />
11 294
Therefore, the AS security metric of AS1 evaluates to<br />
s node (AS1, G(V, E), ψ) = 1<br />
n − 1<br />
∑<br />
j∈{AS2,AS3,AS4,AS5}<br />
= 1 (0.6 + 0.4 + 0.6 + 0.8)<br />
4<br />
= 0.6<br />
s route (AS1, j, G(V, E), ψ)<br />
Note that for the full evaluation of the security measurement function this procedure<br />
would have to be repeated for the remaining four ASs. For the other ASs we get<br />
s node (AS2, G(V, E), ψ) = 0.55<br />
s node (AS3, G(V, E), ψ) = 0.6<br />
s node (AS4, G(V, E), ψ) = 0.55<br />
s node (AS5, G(V, E), ψ) = 0.65<br />
Thus the security measurement function evaluates to<br />
S((1, 0, 0, 0, 1) T , G(V, E), ψ) = (0.6, 0.55, 0.6, 0.55, 0.65) T<br />
The average AS security metric - which we will use as a proxy for the network’s<br />
routing security - is the average of all AS security metrics, i.e. 0.59.<br />
4.4 Simulation Procedure<br />
The initial step of the simulation is the generation of a global routing table with n(n − 1) entries<br />
for a given network topology. 8 As aforementioned, the sole criterion for path selection is path<br />
length. In case there exist several paths with minimal length, one path is chosen with equal<br />
probability. Thus, the function Ψ is inherently indeterministic. Throughout our experiments,<br />
we used only one out of many possible global routing configurations. This simplification was<br />
necessary due to computational constraints. We used artificial AS topologies of different shapes<br />
and sizes for the experiments. In terms of the shape of topologies an important criterion of<br />
differentiation is the distribution of node degrees. We used scale-free topologies as well as<br />
topologies where the distribution of the node degree follows a normal distribution with little<br />
variance, i.e. where all nodes had approximately the same node degree. In order to determine<br />
the effect of topology size on our results, we compared the results from measurements on 20-<br />
node topologies and 4000-node topologies. As the differences are almost negligible, we report<br />
the results of measurements using a 4000-node Internet-like, i.e. scale free topology generated<br />
by Inet-3.0 [25].<br />
After the global routing table ψ has been constructed, the simulation applies the measurement<br />
function S(e, G(V, E), ψ), which maps a vector of adoption decisions onto a vector of AS security<br />
levels given the AS topology defined by G(V, E). In addition to the adoption decisions e, the<br />
topology G(V, E) and the global routing table ψ, we considered three more independent variables<br />
in our model not included in the formal definitions for notational convenience:<br />
• The security scheme implemented by the adopters, i.e. ingress or egress detection<br />
• The capability of the attackers, i.e. prefix hijacking, path spoofing or both attacks 9<br />
• The security goal <strong>und</strong>erlying the security metric, i.e. either semi-robustness or (complete)<br />
robustness<br />
8 Thus, exactly one path connecting each AS to all other ASs is stored although there may be many more.<br />
9 If the attacker is able to use both kinds of attacks we assume that she first tries to use prefix hijacking and<br />
if this should not be successful uses path spoofing.<br />
12 295
Independent Variable<br />
configuration of adoption decisions<br />
AS topology<br />
global routing table<br />
security scheme<br />
attacker capability<br />
security goal<br />
Values<br />
ascending and descending deployment a<br />
scale-free and random, 20 and 4000 ASs<br />
shortest path routing b<br />
ingress and egress detection<br />
prefix hijacking, path spoofing and both attacks<br />
semi-robustness and (complete) robustness<br />
a Ascending deployment refers to a sequential deployment process starting with the AS that has the least interdomain<br />
paths leading through it until full adoption is reached. Descending deployment proceeds in the opposite<br />
way.<br />
b For each topology we based the simulation on one random instance of the global routing table only. We do<br />
not expect the indeterministic nature of the routing table construction to have a significant impact on our results.<br />
Table 1: Summary of independent variables<br />
Altogether 32 simulation treatments were run with different parameter configurations. Table 1<br />
gives an overview of the independent variables and corresponding values we considered in our<br />
simulation study.<br />
5 Experimental Results<br />
5.1 F<strong>und</strong>amental Vulnerability<br />
The benchmark model assumes that no countermeasures have been implemented. For this<br />
benchmark, we measure the f<strong>und</strong>amental vulnerability of ASs. In terms of our notation, this<br />
boils down to evaluating the security measurement function S with the parameter e = (0, ..., 0).<br />
Shortest path selection in a scale free topology naturally results in few ASs with high node degree<br />
being responsible for carrying the traffic of a much larger portion of ASs with low outdegree.<br />
Subsequently, we counted the number of (legitimate) inter-domain paths leading through each<br />
AS and formed groups of ASs with the same number of paths leading through them.<br />
mean<br />
0.9 1<br />
0.8<br />
0.7<br />
0.6<br />
0.5<br />
0.4<br />
0.3<br />
0.2<br />
0.0001 0.001 0.01 0.1 1<br />
mean<br />
0.9 1<br />
0.8<br />
0.7<br />
0.6<br />
0.5<br />
0.4<br />
0.3<br />
0.2<br />
0.0001 0.001 0.01 0.1 1<br />
path fraction<br />
path fraction<br />
(a)<br />
(b)<br />
standard deviation<br />
0.14<br />
0.12<br />
0.1<br />
0.08<br />
0.06<br />
0.04<br />
0.02<br />
0<br />
0.0001 0.001 0.01 0.1 1<br />
path fraction<br />
standard deviation<br />
0.14<br />
0.12<br />
0.1<br />
0.08<br />
0.06<br />
0.04<br />
0.02<br />
0<br />
0.0001 0.001 0.01 0.1 1<br />
path fraction<br />
(c)<br />
(d)<br />
Figure 5: Mean and standard deviation of AS security metrics without countermeasures<br />
Figure 5 shows plots of the mean (a) and standard deviation (c) of the AS robustness metric for<br />
13 296
each of these AS groups if attackers are only capable of prefix hijacking. Plots (b) and (d) of<br />
figure 5 show the same measurements applied to path spoofing. We restricted the presentation<br />
of results to the robustness metric. The conclusions drawn from the depicted results are the<br />
same for the semi-robustness metric. The measurement results for the third attack mode (prefix<br />
hijacking combined with path spoofing) coincides with the results for prefix hijacking since prefix<br />
hijacking is always more effective than path spoofing in the absence of countermeasures. Each<br />
impulse in the plots represents one group of ASs with an equal number of inter-domain paths<br />
leading through them. Accordingly, all stub ASs are allocated to the same group and this group<br />
is represented by the first impulse from the left. 10 We normalized the horizontal axis to the unit<br />
interval and applied a log scaling in order to improve visibility.<br />
The plots indicate that there is a significant positive correlation between the number of interdomain<br />
paths leading through ASs and their vulnerability to routing attacks. We obtained a<br />
correlation coefficient of 0.1763 for prefix hijacking and 0.1205 for path spoofing. Thus, the<br />
security level of an AS with more inter-domain paths going through it tends to be lower for<br />
both kinds of attacks and this relationship is more significant for prefix hijacking. The plots<br />
also show that vulnerability to prefix hijacking attacks is generally much higher than to path<br />
spoofing. The observed correlation would be much higher if the security metrics of ASs with a<br />
relatively small number of inter-domain paths leading through them were less scattered. The<br />
plots of the standard deviations across the different groups show that the security metrics are<br />
less scattered in groups of ASs that have more transit paths. The standard deviation of the stub<br />
ASes’ security metric is particularly high, i.e. there are significant differences among their levels<br />
of vulnerability.<br />
In the Internet-like topology the ASs are more specialized on either serving as transit ASs or<br />
mere traffic end-points (aka stubs). Given that paths are constructed solely based on the shortest<br />
path criterion, this does not come as a surprise. If flat topologies are used instead of scale-free<br />
ones, the number of inter-domain paths leading through the ASs is distributed more evenly<br />
because there are fewer network hubs accommodating shortest paths. In our experiments using<br />
this kind of topology we observed the same correlation between the number of paths leading<br />
through an AS and its security metrics.<br />
5.2 The Effects of Ingress Detection<br />
To measure the effect of the deployment of ingress detection in the AS topology, we aggregated<br />
the AS security metrics by computing the average over all ASs (including the non-adopters).<br />
The aggregate metric therefore serves as a metric for the resilience against routing attacks of<br />
the whole network. Figure 6 shows how the mean AS security metrics based on path robustness<br />
evolve for all three attack modes if ingress detection is deployed in an increasing number of ASs.<br />
While figure 6a) shows the measurement results obtained when deployment was performed in<br />
ascending order of the number of inter-domain paths leading through the ASs, figure 6b) shows<br />
the results obtained for the opposite deployment order, i.e. descending deployment.<br />
Figure 6a) indicates that the mean AS security metrics for path robustness significantly increase<br />
only if more than 90% of the AS population deploy ingress detection. Furthermore, the metrics<br />
hardly move until about half of the ASs take part in the effort. Remember that this approximately<br />
accounts for the number of stub ASs in the sample. Therefore we can conclude that<br />
stub ASs contribute almost nothing to the overall robustness of the network if they are alone to<br />
deploy ingress detection.<br />
From the comparison of figures 6a) and 6b) one can observe that if the security protocol is<br />
deployed in ascending order, the marginal mean AS security metric increases at a lower rate in<br />
10 For the 4000 node topology we used in our experiment, the algorithm responsible for constructing the global<br />
routing table terminated with more than half of the ASs being stubs.<br />
14 297
mean AS security metric (robustness)<br />
1<br />
0.9<br />
0.8<br />
0.7<br />
0.6<br />
0.5<br />
only prefix hijacking<br />
only path spoofing<br />
both attacks<br />
0.4<br />
0 0.2 0.4 0.6 0.8 1<br />
adopting fraction<br />
mean AS security metric (robustness)<br />
1<br />
0.9<br />
0.8<br />
0.7<br />
0.6<br />
0.5<br />
only prefix hijacking<br />
only path spoofing<br />
both attacks<br />
0.4<br />
0 0.2 0.4 0.6 0.8 1<br />
adopting fraction<br />
(a)<br />
(b)<br />
Figure 6: Mean robustness for ascending (a) and descending (b) deployment of ingress detection<br />
mean AS security metric (semi-robustness)<br />
1<br />
0.9<br />
0.8<br />
0.7<br />
0.6<br />
0.5<br />
only prefix hijacking<br />
only path spoofing<br />
both attacks<br />
0.4<br />
0 0.2 0.4 0.6 0.8 1<br />
adopting fraction<br />
mean AS security metric (semi-robustness)<br />
1<br />
0.9<br />
0.8<br />
0.7<br />
0.6<br />
0.5<br />
only prefix hijacking<br />
only path spoofing<br />
both attacks<br />
0.4<br />
0 0.2 0.4 0.6 0.8 1<br />
adopting fraction<br />
(a)<br />
(b)<br />
Figure 7: Mean semi-robustness for ascending (a) and descending (b) deployment of ingress<br />
detection<br />
the beginning and at a higher rate toward the end compared to the case where deployment takes<br />
place in the opposite order (descending deployment). The major transit ASs therefore have a<br />
crucial influence on the overall routing security in the AS topology. Without their participation<br />
hardly any additional inter-domain paths can be made robust against routing attacks. If they<br />
deploy ingress detection right at the beginning, they pave the way for a faster increase of the<br />
mean path robustness metric as the ASs with fewer or no transit paths leading through them<br />
join.<br />
If semi-robustness is measured instead of complete robustness, the mean security metrics evolve<br />
differently. Figure 7 depicts the development of the mean security metric based on semirobustness.<br />
Comparing 6b) and 7b) we see that if the ASs deploy ingress detection in descending<br />
order, the mean security metrics for semi-robustness and complete robustness almost coincide. 11<br />
Therefore the number of paths which are semi-robust but not robust at the same time is very<br />
small during the whole deployment process. If the deployment order is reversed, however, this<br />
number becomes much greater. The explanation for this observation can be fo<strong>und</strong> in the fact<br />
that semi-robustness is a weaker security requirement. More importantly, it shows that if the<br />
11 In fact, the former is always slightly greater than the latter.<br />
15 298
aim is to only enhance the overall semi-robustness of the network, the order of deployment plays<br />
a much less significant role. If complete robustness is required, descending deployment is clearly<br />
superior to ascending deployment in terms of overall resilience against routing attacks.<br />
Let us now have a closer look at some of the more subtle differences between the curves in<br />
figure 6a) and 7a). We have already noted that the importance of the deployment order is less<br />
significant if we consider semi-robustness instead of complete robustness. This can be attributed<br />
to the fact that a path can be made semi-robust against prefix hijacking if ingress detection is<br />
deployed at both ends of the path whereas complete robustness in the worst case requires deployment<br />
of security filters along the whole inter-domain path. However, this does not explain<br />
why the mean semi-robustness metric in figure 7a) tends to be higher than the one in figure<br />
7b). Apparently, the ASs with fewer legitimate paths leading through them contribute more to<br />
the increase of the mean AS security than the major transit ASs do. If one considers the initial<br />
distribution of security levels described in section 5.1, the reason for this effect can be explained<br />
by the relatively low baseline vulnerability of transit ASs. Since many of their inter-domain<br />
paths are robust (or at least semi-robust) even in case no countermeasures are deployed in the<br />
network, there is little additional security to be gained among them. Those ASs accomodating<br />
fewer or no transit paths gain more on average if more ASs deploy ingress detection. Since<br />
the f<strong>und</strong>amental distribution of vulnerability has the same properties both for semi-robustness<br />
as well as complete robustness, this explanation given above applies even more generally. The<br />
reason that the effect cannot be observed in figure 6a) is that transit ASs play such an important<br />
role in providing (complete) robustness of their inter-domain paths. Thus they prevent stub ASs<br />
from realizing security gains as long as they have not deployed ingress detection.<br />
Another interesting observation from the comparison of the plots is that the mean semi-robustness<br />
against prefix hijacking increases particularly quickly if deployment takes place in increasing order.<br />
This can be explained by the fact that assuring semi-robustness against prefix hijacking<br />
only requires the deployment of ingress detection at both ends of the legitimate path. However,<br />
if the attacker is able to path spoof, this might not be enough. In this case the structural<br />
position of the path within the topology remains an important factor in determining the degree<br />
of semi-robustness. For example, if the infiltrated AS (1) spoofs a link to one of the neighbors<br />
of the destination AS which has not deployed ingress detection and (2) the spoofed path is not<br />
longer than the legitimate one, the path is not considered semi-robust according to our simulation.<br />
Thus, resilience against path spoofing in many cases requires the deployment of ingress<br />
detection in intermediate ASs. 12<br />
Our next step was to divide the set of ASs into stub and transit ASs and compute the mean AS<br />
security metric of both groups separately. We did so in order to find out more about the mutual<br />
dependencies of the two groups.<br />
Figure 8 depicts the mean security metrics of stub and transit ASs based on complete robustness<br />
(figure 8) and on semi-robustness (figure 9) given the respective group is the first deploying<br />
ingress detection. The gap between the last measurement point in these diagrams and complete<br />
security (i.e. a mean AS security metric of 1) represents the security increase of the observed<br />
group that would be achieved if the other group also adopted ingress detection. In economic<br />
terms, the effect of the action of one entity on the utility of another entity which has no influence<br />
on the effect itself is called externality. Borrowing from the economics literature we therefore<br />
denote this effect security externality. The comparison of figures 8a) and b) reveals that stub<br />
ASs can achieve almost no increase of their mean security metric without the adoption of transit<br />
ASs whereas the transit ASs can achieve a significant increase without the adoption of the<br />
stub ASs. Accordingly, stub ASs in terms of complete robustness depend more strongly on the<br />
adoption of the transit ASs than vice versa.<br />
Figure 9 shows the development of the mean security metric of the two groups if the weaker<br />
security requirement semi-robustness is the objective. 9a) and b) indicate that the asymmetry<br />
12 You may want to refer to the description of the simulation model to clarify this point.<br />
16 299
mean AS security metric (robustness)<br />
1<br />
0.9<br />
0.8<br />
0.7<br />
0.6<br />
0.5<br />
0.4<br />
0.3<br />
only prefix hijacking<br />
only path spoofing<br />
both attacks<br />
0.2<br />
0 0.2 0.4 0.6 0.8 1<br />
adopting fraction<br />
mean AS security metric (robustness)<br />
1<br />
0.95<br />
0.9<br />
0.85<br />
0.8<br />
0.75<br />
0.7<br />
0.65<br />
0.6<br />
0.55<br />
only prefix hijacking<br />
only path spoofing<br />
both attacks<br />
0.5<br />
0 0.2 0.4 0.6 0.8 1<br />
adopting fraction<br />
(a)<br />
(b)<br />
Figure 8: Comparison of the robustness of stub (a) and transit ASs (b)<br />
mean AS security metric (semi-robustness)<br />
1<br />
0.9<br />
0.8<br />
0.7<br />
0.6<br />
0.5<br />
0.4<br />
only prefix hijacking<br />
only path spoofing<br />
both attacks<br />
0.3<br />
0 0.2 0.4 0.6 0.8 1<br />
adopting fraction<br />
mean AS security metric (semi-robustness)<br />
1<br />
0.95<br />
0.9<br />
0.85<br />
0.8<br />
0.75<br />
0.7<br />
0.65<br />
0.6<br />
0.55<br />
only prefix hijacking<br />
only path spoofing<br />
both attacks<br />
0.5<br />
0 0.2 0.4 0.6 0.8 1<br />
adopting fraction<br />
(a)<br />
(b)<br />
Figure 9: Comparison of the semi-robustness of stub (a) and transit ASs (b)<br />
of dependencies is less significant compared to the case where complete robustness of the paths<br />
is required. We have noted earlier that the deployment of ingress detection in intermediate ASs<br />
is oftentimes not required in order to make a path semi-robust. Hence, the stub ASs can secure<br />
a significant fraction of the paths connecting them to each other without the participation of<br />
transit ASs in this case.<br />
All mean security metrics depicted in figures 8 and 9 are higher for the group of transit ASs.<br />
This is because the more inter-domain paths lead through an AS, the more secure it usually<br />
is. Since stub ASs are present on the minimum number of paths, namely just the n − 1 paths<br />
connecting each of them with all other ASs in the Internet, their average security metric is lower<br />
than the one of the remaining ASs on average.<br />
5.2.1 The Effects of Egress Detection<br />
The development of the mean AS security metric changes substantially if egress instead of ingress<br />
detection is deployed in the topology.<br />
Figure 10 shows how the this metric evolves as egress detection is deployed sequentially in ascending<br />
and descending order. In both deployment modes the mean robustness increases with<br />
each adopting AS. According to our model, deploying egress detection in an AS is equivalent<br />
17 300
mean AS security metric (robustness)<br />
1<br />
0.9<br />
0.8<br />
0.7<br />
0.6<br />
0.5<br />
only prefix hijacking<br />
only path spoofing<br />
both attacks<br />
0.4<br />
0 0.2 0.4 0.6 0.8 1<br />
adopting fraction<br />
mean AS security metric (robustness)<br />
1<br />
0.9<br />
0.8<br />
0.7<br />
0.6<br />
0.5<br />
only prefix hijacking<br />
only path spoofing<br />
both attacks<br />
0.4<br />
0 0.2 0.4 0.6 0.8 1<br />
adopting fraction<br />
(a)<br />
(b)<br />
Figure 10: Mean robustness for ascending (a) and descending (b) deployment of egress detection<br />
mean AS security metric (robustness)<br />
1<br />
0.9<br />
0.8<br />
0.7<br />
0.6<br />
0.5<br />
0.4<br />
only prefix hijacking<br />
0.3<br />
only path spoofing<br />
both attacks<br />
0.2<br />
0 0.2 0.4 0.6 0.8 1<br />
adopting fraction<br />
mean AS security metric (robustness)<br />
1<br />
0.95<br />
0.9<br />
0.85<br />
0.8<br />
0.75<br />
0.7<br />
0.65<br />
0.6<br />
only prefix hijacking<br />
0.55<br />
only path spoofing<br />
both attacks<br />
0.5<br />
0 0.2 0.4 0.6 0.8 1<br />
adopting fraction<br />
(a)<br />
(b)<br />
Figure 11: Comparison of the mean robustness of stub (a) and transit ASs (b)<br />
to eliminating this AS as a potential threat for the inter-domain paths linking together other<br />
ASs. Therefore the linear shape of the curves depicted in figure 10 indicates that each AS can<br />
be used to conduct successful attacks on at least some portion of the paths. Comparing figures<br />
10a) and 10b) more closely, we see that - although the metrics increase in an almost linear way<br />
in both deployment modes - securing the transit ASs first causes the metric to rise more quickly<br />
in the beginning. This shows that attacks launched from core ASs can be expected to have more<br />
severe consequences in terms of the number of inter-domain paths affected.<br />
Comparing the plots of figure 10 with the ones obtained for ingress detection (figure 6), we<br />
observe that switching the deployment mode has much more severe consequences for the development<br />
of the mean security metrics in the case of ingress detection. Furthermore we find<br />
that irrespective of the deployment order, the egress detection approach dominates the ingress<br />
detection approach: For all analyzed configurations of the input vector e (except zero and full<br />
adoption) it is greater.<br />
Since the results for semi-robustness do not differ from the mean security metric evolves, we<br />
omit them. The explanation for this similarity is straightforward given the premises of our simulation:<br />
Deploying egress detection in an AS causes the potential adverse influence of the AS on<br />
legitimate paths in the topology to vanish completely. Thus, the difference between robustness<br />
18 301
and semi-robustness does not have any influence on the way the mean security metric evolves. 13<br />
Figure 11 shows the development of the mean security metric based on complete robustness of<br />
the groups of stub and transit ASs separately. The plots reveal that the security externalities<br />
we have already observed in the ingress detection case also exist in the egress detection case.<br />
However, the dependency of the stub on the transit ASs as compared to ingress detection is far<br />
less significant. The stub ASs can significantly improve their own security by adopting egress<br />
detection while they can achieve almost no additional security if they deploy ingress detection.<br />
6 Summary and Discussion<br />
This paper provides several contributions to the discussion of how tomorrow’s Internet infrastructure<br />
can be protected against routing attacks, which opens up new avenues for protocol<br />
research. The contributions can be summarized as follows:<br />
First and most importantly, taking a step back by questioning the approach that takes ingress<br />
detection for granted has brought us to the definition and motivation of an alternative approach<br />
which we call egress detection. Egress detection requires less communication overhead and fewer<br />
standardization efforts compared to ingress detection and therefore represents a much less complex<br />
and less expensive solution. On the other hand, egress detection fails in terms of securing<br />
specific paths. In order to quantify the effects ingress and egress detection have we adapt a<br />
simulation model that was introduced by Chan et al. [4].<br />
While the security measurement methodology used in our paper is based on the same principle,<br />
the model we use is less detailed with respect to attacker characteristics 14 and makes less stringent<br />
assumptions 15 . We have noted earlier that the model we use does not address many of the<br />
complexities of the BGP. In order to compare the efficiency of ingress and egress detection on<br />
a strategic level our simulation can yield important insights. The implementation of our model<br />
is capable of showing the effects of prefix hijacking and path spoofing attacks separately. This<br />
way we have shown that the inter-domain routing infrastructure is about twice as vulnerable to<br />
prefix hijacking than to path spoofing attacks and that if attackers have both types of attack at<br />
their disposal, path spoofing does not significantly increase the impact of attacks.<br />
Furthermore, we have defined two different security metrics based on different robustness requirements<br />
(robustness and semi-robustness of inter-domain paths).<br />
Our measurement study has revealed that the vulnerability of the different ASs is correlated<br />
with the number of inter-domain paths leading through them and therefore depends on the<br />
roles the ASs play in provisioning connectivity. The results suggest that stub ASs are more<br />
vulnerable to routing attacks than transit ASs. Furthermore, we have shown experimentally<br />
how ingress and egress detection differ with respect to how efficiently they enhance the routing<br />
security of the whole network. We have discovered that deploying ingress and egress detection<br />
in increasing and decreasing order of the number of paths that lead through each AS is an<br />
effective way to reveal existing dependencies between ASs. By separating the set of ASs in the<br />
sample topology into stub and transit ASs, we have directly shown the existence of what we call<br />
security externalities, i.e. that the action of deploying either countermeasure in one AS has a<br />
positive effect on the security metric of other ASs. If ingress detection is used, security benefits<br />
13 The only difference between the robustness and semi-robustness plots is that the semi-robustness curve possesses<br />
a higher intercept. This can again be explained by the fact that semi-robustness is the weaker requirement.<br />
We have seen earlier that there are a few more paths that are semi-robust than there are robust paths given no<br />
countermeasures are deployed.<br />
14 Chan et al. use two different attacker models based on different degrees of knowledge that the attacker has<br />
about the topology. We adopted what Chan et al. denote as the strong attacker model, i.e. attackers are expected<br />
to know how the global routing table looks like before the attack.<br />
15 Chan et al. use an AS security metric based on the assumption that all inter-domain paths leading through<br />
an AS are relevant for that AS, whereas we only consider the n-1 paths that connect that AS with all other ASs<br />
in the topology. They also introduce a weighing factor differentiating the importance of different paths based on<br />
some estimate of for traffic load while we treat all paths as equal.<br />
19 302
Property Ingress Detection Egress Detection<br />
type of externality positive network positive pure<br />
mean (semi-)robustness metric convex linear<br />
asymmetric dependency stub ASs depend on transit ASs a none<br />
subnet (semi-)robustness practical impractical<br />
rank of dominance b 2 1<br />
a Only applies if complete robustness is required.<br />
b The term dominance refers to the number of paths that are (semi-)robust given a fraction of adopters. A<br />
lower number translates to a higher rank, i.e. egress detection dominates ingress detection with respect to this<br />
definition.<br />
Table 2: Comparison of ingress and egress detection<br />
are restricted to the set of adopters while they spill over from adopters to non-adopters in case<br />
egress detection is used. Therefore we are dealing with two different kinds of externality, namely<br />
network externalities in the case of ingress detection and pure externalities in the case of egress<br />
detection.<br />
The results of our experimental analysis suggest that security externalities are more or less<br />
asymmetric depending on the countermeasure used (ingress or egress detection), how security<br />
is measured (requiring semi-robustness or complete robustness) and the kind of ASs that the<br />
countermeasure is deployed in. If ingress detection is used, stub ASs can achieve no significant<br />
increase of their path robustness levels without the help of transit ASs whereas transit ASs are<br />
much less dependent on the stub ASs. However, this asymmetry does not exist if semi-robustness<br />
instead of complete robustness is being measured. Therefore we argue that a more explicit definition<br />
of the security goal is needed as the objective regarding robustness requirements may<br />
influence adoption incentives. Do we only want to make sure that traffic cannot be hijacked by<br />
the attacker (then semi-robustness would be a sufficient security requirement), or do we want<br />
to maintain the legitimate paths in any event (then complete robustness would be required)?<br />
Another important result of the experiments is that using egress detection as a countermeasure<br />
more overall security can be achieved by fewer adopters, i.e. that the egress detection approach<br />
dominates the ingress detection approach with respect to efficiency. However, egress detection<br />
has the downside that neither robustness nor semi-robustness of particular paths can be guaranteed<br />
if it is only deployed in the ASs on these paths. 16 Thus, complete robustness of a subset<br />
of the inter-domain paths can only be effectively achieved by the use of ingress detection. We<br />
call this property subnet (semi-)robustness.<br />
We have summarized our findings concerning the comparison of ingress and egress detection in<br />
table 2.<br />
7 Conclusions and Future Work<br />
Inter-domain routing security has been on the agenda of the networking community for quite<br />
some time now. Although an impressive number of proposals for security patches lie on the table,<br />
there has still not been any substantial progress with respect their adoption. All comprehensive<br />
security solutions put forward to make the Internet’s infrastructure more robust to routing<br />
attacks suffer from high adoption barriers due to technical issues (protocol standardization and<br />
router overhead) which are in turn amplified by network effects. We conclude that these ingress<br />
detection schemes would have to be deployed by a significant number of AS operators in order<br />
to reliably prevent serious outages in the case of routing attacks. Furthermore, our analysis has<br />
16 This statement has to be seen within the bo<strong>und</strong>s of our model, of course. In reality inter-domain routing can<br />
be a highly dynamic process and therefore the guaranteed robustness of a particular end-to-end connection may<br />
require adoption of ingress detection by a different set of ASs in the course of time.<br />
20 303
evealed strong asymmetries in the dependency relationship of the ASs at the top of the Internet<br />
hierarchy and the ones at the bottom: Stub ASs for instance are more dependent on transit ASs<br />
to deploy ingress detection than vice versa. This dependency is very relevant since the major<br />
transit ASs have little incentive to take part in the security effort given the fact that their paths<br />
are pretty safe from routing attacks anyway. Following the ingress detection approach may thus<br />
lead to a situation where adoption is prevented due to strategic considerations on both sides.<br />
Egress detection could represent an alternative security approach. It is less demanding in terms<br />
of overhead and standardization efforts. According to our results, deployment of egress detection<br />
in a given set of ASs provides more routing security on average than the deployment of<br />
ingress detection. Furthermore, the asymmetries in the dependency structure do not come into<br />
existence if the egress detection approach is being followed. Also, since many routing attacks<br />
occur rather by chance than by intuition, egress detection is an inexpensive alternative to ingress<br />
detection when it comes to preventing unintended misconfiguration. However, egress detection<br />
cannot be effectively used to make a particular set of inter-domain paths more robust, which<br />
might be regarded as crucial by AS operators eager to protect the most important connections<br />
first. Furthermore, egress detection suffers from pure externalities and thus adoption is not<br />
individually incentive-compatible.<br />
More efforts should be devoted to the specification of more realistic simulation models. For<br />
instance, the behavior of the BGP could be modeled more realistically (e.g. using C-BGP [19]).<br />
Furthermore, the effects of policy routing on path selection have been ignored throughout this<br />
work and may also be a worthwhile topic of future research with respect to the security backgro<strong>und</strong><br />
[22].<br />
Although we have outlined the general functionality of egress detection, a technical analysis<br />
would have to follow in order to determine whether corresponding mechanisms are feasible at all<br />
and if so how they could be implemented. One could also imagine to mix the two approaches, i.e.<br />
to deploy ingress and egress detection in parallel, in order to further optimize robustness for a<br />
given set of adopters. From a technical viewpoint it would be interesting to see what additional<br />
infrastructure is needed to implement egress detection on top of ingress detection.<br />
Routing attacks certainly are no purely BGP-specific threat. They should be included into security<br />
considerations regarding all networked systems. Recent research in related areas like sensor<br />
networks show interesting parallels [6].<br />
What makes this work particularly interesting is the inherent connection of network security and<br />
economics. Some of the results we obtained can actually be used as a fo<strong>und</strong>ation of economic<br />
argumentation and provide decision support for policy makers. We have seen that, depending on<br />
the technological approach of increasing the robustness of inter-domain routing, different types<br />
of externalities come into existence. This in turn leads to different economic incentives and corresponding<br />
instruments to overcome the economic adoption barriers. A promising research avenue<br />
would therefore be to further investigate the interdependency between technological features of<br />
BGP security schemes, economic incentives and appropriate policies.<br />
References<br />
[1] W. Aiello, J. Ioannidis, and P. McDaniel. Origin Authentication in Interdomain Routing.<br />
In 10th ACM Conference on Computer and Communications Security, 2003.<br />
[2] The President’s Critical Infrastructure Protection Board. The National Strategy to Secure<br />
Cyberspace, 2003.<br />
[3] K. Butler, T. Farley, P. Mcdaniel, and J. Rexford. A Survey of BGP Security Issues and<br />
Solutions. Technical report, AT&T Labs - Research, Florham Park, NJ, February 2004.<br />
21 304
[4] H. Chan, D. Dash, A. Perrig, and H. Zhang. Modeling Adoptability of Secure BGP Protocols.<br />
In Raymond A. Marie, Peter B. Key, and Evgenia Smirni, editors, SIGCOMM, pages<br />
389–390. ACM, 2006.<br />
[5] H. Chang, R. Govindan, S. Jamin, S. J. Shenker, and W. Willinger. Towards Capturing<br />
Representative AS-Level Internet Topologies. Computer Networks, 44(6):737–755, 2004.<br />
[6] M. Feldman, J. Chuang, I. Stoica, and S. Shenker. Hidden-Action in Multi-Hop Routing.<br />
In CECOMM: ACM Conference on Electronic Commerce, 2005.<br />
[7] V. Gill, J. Heasley, and D. Meyer. The Generalized TTL Security Mechanism (GTSM).<br />
RFC 3682, February 2004.<br />
[8] G. Goodell, W. Aiello, T. Griffin, J. Ioannidis, P. Mcdaniel, and A. Rubin. Working Aro<strong>und</strong><br />
BGP: An Incremental Approach to Improving Security and Accuracy of Interdomain Routing.<br />
In NDSS, April 2003.<br />
[9] A. Heffernan. Protection of BGP Sessions via the TCP MD5 Signature Option. RFC 2385,<br />
March 2002.<br />
[10] Y. Hu, A. Perrig, and M. A. Sirbu. SPV: Secure Path Vector Routing for Securing BGP.<br />
In SIGCOMM, pages 179–192, 2004.<br />
[11] G. Huston. Interconnection, Peering, and Settlements. http://www.potaroo.net, 2003.<br />
[12] S. Kent, C. Lynn, J. Mikkelson, and K. Seo. Secure Border Gateway Protocol (S-BGP) -<br />
Real World Performance and Deployment Issues. February 2000.<br />
[13] S. Kent, C. Lynn, and K. Seo. Secure Border Gateway Protocol (S-BGP). IEEE Journal<br />
on Selected Areas in Communications, 18, 2000.<br />
[14] Ratul Mahajan, David Wetherall, and Tom Anderson. Understanding BGP Misconfiguration.<br />
In SIGCOMM, pages 3–16. ACM, 2002.<br />
[15] S. Misel. Wow, AS7007! Merit NANOG Archive, 1997.<br />
[16] S. Murphy. BGP Security Vulnerabilities Analysis. RFC 4272, January 2006.<br />
[17] D. M. Nicol, S. W. Smith, and M. Zhao. Efficient Security for BGP Route Announcements.<br />
Technical Report TR2003-440, Dartmouth College, Computer Science, Hanover, NH, May<br />
2003.<br />
[18] A. Odlyzko. Economics, Psychology, and Sociology of Security, 2003.<br />
[19] B. Quoitin and S. Uhlig. Modeling the Routing of an Autonomous System with C-BGP.<br />
IEEE Network, 2005.<br />
[20] Y. Rekhter and T. Li. A Border Gateway Protocol 4 (BGP 4). RFC 4271, January 2006.<br />
[21] L. Subramanian, V. Roth, I. Stoica, S. Shenker, and R. H. Katz. Listen and Whisper:<br />
Security Mechanisms for BGP. In NSDI, pages 127–140. USENIX, 2004.<br />
[22] H. Tangmunarunkit, R. Govindan, S. Shenker, and D. Estrin. The Impact of Routing Policy<br />
on Internet Paths. In IEEE INFOCOM, December 2001.<br />
[23] European Union. Network and Information Security: Proposal for a European Policy Approach.<br />
COM(2001)298 final, June 2001.<br />
22 305
[24] R. White. Deployment Considerations for secure origin BGP (soBGP). Internet Draft,<br />
2002.<br />
[25] J. Winick and S. Jamin. Inet-3.0: Internet Topology Generator, July 11 2002.<br />
[26] H. Yu, J. Rexford, and E. W. Felten. A Distributed Reputation Approach Cooperative<br />
Internet Routing Protection. In Workshop on Secure Network Protocols, 2005.<br />
[27] X. Zhao, D. Pei, L. Wang, D. Massey, A. Mankin, S. F. Wu, and L. Zhang. Detection of<br />
Invalid Routing Announcement in the Internet. In International Conference on Dependable<br />
Systems and Networks (DSN), 2002.<br />
23 306
A Market Mechanism for Energy Allocation in Micro-CHP Grids<br />
Carsten Block, Dirk Neumann, Christof Weinhardt<br />
Universität Karlsruhe (TH), Germany<br />
{block, neumann, weinhardt}@iism.uni-karlsruhe.de<br />
Abstract<br />
Achieving a sustainable level of energy production and<br />
consumption is one of the major challenges in our<br />
society. This paper contributes to the objective of<br />
increasing energy efficiency by introducing a market<br />
mechanism that facilitates the efficient matching of<br />
energy (i.e. electricity and heat) demand and supply<br />
in Micro Energy Grids. More precisely we propose<br />
a combinatorial double auction mechanism for the<br />
allocation and pricing of energy resources that especially<br />
takes the specific requirements of energy producers and<br />
consumers into account. We describe the potential role<br />
of decentralized micro energy grids and their coupling<br />
to the large scale power grid. Furthermore we introduce<br />
an emergency fail over procedure that keeps the micro<br />
energy grid stable even in cases where the auction mechanism<br />
fails. As the <strong>und</strong>erlying energy allocation problem<br />
itself is NP-hard, we derive a fast heuristic for finding<br />
efficient supply and demand allocations. In addition we<br />
show the applicability of this approach through numerica.<br />
Keywords: Combined Heat and Power; Market<br />
mechanism; Double Auction; Combinatorial allocation<br />
1 Introduction<br />
Over the last years fossil energy prices were rising remarkably.<br />
E.g. the price for Crude oil increased from approximately<br />
20 USD in 1995 to more than than 65 USD<br />
as of today [25] while the prices for other fossil resources<br />
evolved similarly [13]. On the one hand rising energy<br />
prices indicate the increasing scarcity of these resources<br />
and thus implicate a need for change in our energy consumption<br />
habits, on the other hand high prices foster the<br />
development of alternative technologies for energy generation.<br />
A study from Emerging Energy Research [16]<br />
states that today wind power technology already allows<br />
electricity generation at cost below those of conventional<br />
coal fired power plants if carbon penalties of more than<br />
30 EUR per tonne are applied. 1 Similarly Krewitt and<br />
Schlomann [17] state that energy produced from renewable<br />
resources is already competitive if external cost are<br />
included.<br />
Taking this ongoing change process as given one<br />
needs to determine the challenges that come along with<br />
this development. In particular the effects on the future<br />
energy generation and distribution infrastructures need<br />
to be addressed [6]. Traditionally energy generation is<br />
a centralized business, mainly because the required (fossil)<br />
resources were (and still are) available “on the spot”<br />
and central generation allows to exploit some economies<br />
of scale. Throughout the last years the development<br />
of new energy generation technologies fosters the utilization<br />
of renewable energy resources. But compared<br />
to large scale power plants, energy generation through<br />
photovoltaic systems, biogas plants or even windparks is<br />
(i) a much more decentralized and (ii) a much less predictable<br />
business. Thus this development puts a lot of<br />
pressure on the current energy distribution infrastructure.<br />
Our proposition to address this issue is to develop<br />
semi-autonomous micro energy grids that bind the<br />
distributed energy generation facilities to the local demands.<br />
The local netting of energy demand and supply<br />
inside a microgrid could already reduce parts of the randomness<br />
and thus should enable the microgrid to appear<br />
as a rather predictable “citizen” in the large scale grid<br />
where it buys peak demands from the outside outside<br />
world and offers free generation capacities as balancing<br />
power to others. Furthermore a combined usage of heat<br />
1 In May 2006 carbon prices in the European Cap-and-Trade<br />
System already reached 30 EUR per tonne of CO 2<br />
1<br />
307
and power inside the microgrid will increase the overall<br />
energy efficiency.<br />
Besides difficulties in predicting and coordinating<br />
distributed energy supply, energy consumption forecasting<br />
– especially for private households – is problematic as<br />
well. This is mainly due to two reasons: On the one hand<br />
old metering technology limits readouts in practice to annual<br />
cycles, on the other hand flat fee tariffs (e.g. 18 Ct<br />
/ kWh of electricity) do not set any economic incentives<br />
for consumers to shift energy consumption from peak<br />
hours to periods of low consumption. The result is that<br />
consumers use energy more or less “at random” while energy<br />
providers are more or less blind on the energy flows<br />
inside their low voltage distribution networks. At the<br />
same time it is known from field studies that about 50%<br />
of the private households’ electricity consumption is dedicated<br />
to the operation of refrigerators, freezers, (water)<br />
heaters, washing mashines or dryers [24] and thus could<br />
follow – at least within certain bo<strong>und</strong>aries – a planned<br />
schedule.<br />
Our solution for addressing the aformentioned challenges<br />
is to develop combined heat and power micro<br />
energy grids (CHPMEG) that are “loosely” connected<br />
to the conventional large-scale power grid minimizing<br />
energy transmission losses while maximizing generation<br />
efficiency ratios. Within a CHPMEG energy producers<br />
and energy consumers trade their energy (i.e. heat<br />
and electricity) supplies and demands on local marketplaces.<br />
Arbitrage agents ensure that the resulting energy<br />
prices within the microgrid are levelled to those on<br />
external marketplaces such as the European Energy Exchange<br />
(EEX). As markets provide an efficient matching<br />
of demand and supply on average, a technical balancing<br />
mechanism (spinning reserve) complements the market<br />
mechanisms to ensure stable operation of the local energy<br />
grid, even in those cases where the market-based allocation<br />
might fail without compromising the economic<br />
2 http://www.sesam.uni-karlsruhe.de<br />
tract making on the platform[8]. In this paper we focus<br />
on the hybrid control mechanism for CHPMEG that is<br />
developed as part of this project. The applicability of<br />
incentives set by the markets. The work presented in<br />
this paper is part of the project “Self-organization and<br />
Spontaneity in Liberalized and harmonized Markets” 2<br />
(SESAM). The main focus of the project ist to develop<br />
an electronic energy market platform that also features<br />
mechanisms for secure and non repudiable communication<br />
between the market participants, permits the utilization<br />
of electronic trading agents and provides an electronic<br />
law mediator who supervises the electronic con-<br />
our approach is demonstrated through numerica. The<br />
remainder of this paper is structured as follows. Section<br />
2 describes the typical setup of a CHPMEG and describes<br />
the importance and role of energy markets within<br />
the microgrids. Based on this several requirements for<br />
the operation of CHPMEG are deduced which serve as<br />
benchmarks for the literature review in section 3. In section<br />
4 the formal market model is introduced along with<br />
a fast heuristic for implementation purposes. Section 5<br />
concludes with an application case study and an outlook<br />
on future work.<br />
2 Motivational Scenario<br />
In this section we describe an application for a market<br />
based micro CHP grid which typically consists of several<br />
components connected to each other trough heat pipes<br />
and power / communication lines. Usually five key components<br />
can be identified in a micro grid as shown in<br />
figure 1 [1, 18].<br />
1. Consumers are private households and small business<br />
that require electricity and / or heat.<br />
2. Producers are photovoltaic systems, heaters,<br />
micro-turbines, biogas plants, etc. They generate<br />
electricity, heat or both.<br />
3. Spinning Reserves provide ad-hoc energy reserves<br />
that is used to stabilize voltage and frequency of<br />
the power grid. Battery banks or spinning wheels<br />
usually provide facilities for quick electricity storage<br />
and retrieval and thus serve as spinning reserves.<br />
3 Heat-wise grid stability is not too critical,<br />
still therms can take over the role of heat regulators<br />
as they are able to temporarily store certain<br />
amounts of heat.<br />
4. The coupling point is the connection between CH-<br />
PMEG and the large scale grid. At this point in-<br />
3 Balancing power from the large scale grid can complement the<br />
migrogrid’s spinning reserves. Still, distinct spinning reserves are<br />
oftentimes build into the micro grid as well in order to gain a<br />
certain degree of independence from power outages in the large<br />
grid.<br />
308
o<strong>und</strong> and outbo<strong>und</strong> power flows are metered and<br />
regulated.<br />
5. The controller is the core of the CHPMEG. It<br />
is responsible for matching the consumer’s energy<br />
demandy by dispatching appropriate generators for<br />
heat and / or electricity, i.e. it has to determine an<br />
optimal production schedule given the consumption<br />
requests.<br />
Although figure 1 depicts consumers and producers<br />
as distinct entities these roles can switch dynamically.<br />
A household might generate and sell excess electricity<br />
from its own photovoltaic system during daytime while<br />
it might consume power during night time. We require<br />
all entities within a CHPMEG, i.e. producers, consumers<br />
and spinning reserves to be connected to the controller<br />
through power lines, local area network (LAN) and –<br />
where applicable – through heat pipes as well. Furthermore<br />
we assume that heat pipe and power line capacities<br />
are sufficiently dimensioned to meet all transmission demands<br />
that might occur.<br />
As mentioned above, the central component of this<br />
microgrid is the Controller which determines energy production<br />
and consumption of all grid participants and ensures<br />
that (i) the electricity grid is always balanced in<br />
terms of voltage and frequency and (ii) the heat grid<br />
always provides at least as much heat as required, storing<br />
or disposing overproduction in therms. Furthermore<br />
the controller should provide incentives to consumers to<br />
shift their energy consumption from peak hours to time<br />
periods with low energy demands if possible. Likewise<br />
on the producer side the controller needs to determine<br />
an operation schedule that always favors the producer<br />
with the lowest production cost for generating the required<br />
energy. If e.g. a photovoltaic system and a microgas<br />
turbine are ready to produce power for the microgrid,<br />
the photovoltaic system will likely have comparably<br />
lower production cost during sunshine periods and thus<br />
should produce the energy saving the microturbine from<br />
burning gas.<br />
This idea could be extended even further: The controller<br />
needs to make sure that not only the cheapest<br />
producers inside the microgrid generate the required energy<br />
but that the overall cheapest producers will take<br />
over this responsibility, i.e. if the spot market price for<br />
electricity e.g. at the EEX is lower than the marginal<br />
production cost of the most expensive currently running<br />
generator in the microgrid, it should be turned off and<br />
the electricity should be purchased from the EEX instead<br />
(and vice versa). The same procedure should be applied<br />
if – during peak hours – the local generation capacity is<br />
insufficient to meet the local demands. From a technical<br />
perspective is important to note that many generators,<br />
especially combustion engines are not able to operate at<br />
below certain load levels. Thus the CHPMEG controller<br />
hast to take minimum and maximum production constraints<br />
of the respective energy producers into account<br />
as well.<br />
Finally the netting of energy demand and supply<br />
within the CHPMEG should reduce the randomness<br />
in energy production and consumption to the outside<br />
world. At best the CHPMEG is able to announce future<br />
energy shortages (or energy reserves) as early as possible<br />
to the outside world, making its behavior as predictable<br />
as possible.<br />
In this section we collected several different requirements<br />
that CHPMEGs and especially the CHPMEG<br />
controllers should meet. These can be summarized as<br />
follows:<br />
R1<br />
R2<br />
R3<br />
R4<br />
R5<br />
R6<br />
R7<br />
Online Mechanism (instantaneous matching<br />
of demand and supply)<br />
B<strong>und</strong>led allocation of electricity and heat<br />
Price signals to indicate scarcity<br />
Min and max. allocation constraints<br />
Stable operation (balanced demand & supply)<br />
Coupling with large-scale power grid<br />
Forecasts for demand and supply<br />
In the following section we will review related approaches<br />
for realizing this control component before we<br />
introduce our market based controller in section 4.<br />
3 Related Work<br />
The idea to employ market mechanisms for distributed<br />
environments is not new. Since the 60s researchers have<br />
motivated the use of markets as a means to cope with<br />
incentival problems especially for distributed computing.<br />
But it was only recently that the idea of decentralization<br />
spilled over from computer grids to power grids [4, 10].<br />
309
Figure 1: Schematic Layout of a Micro CHP Grid<br />
Also relatively new is the idea to use markets for the<br />
coordination of decentralized resources. Among the first,<br />
Buyya, Wolski et al. [5], and Subramoniam et al. [23]<br />
motivated the use of auctions and negotiations for Computational<br />
Grid. In their first paper, Wolski et al. [27]<br />
suggested the use of traditional auction formats such as<br />
English auctions. Eymann et al. [12] introduce a decentralized<br />
bargaining system for resource allocation. Regev<br />
et al. [21] propose the use of a Vickrey auction for allocation<br />
computational resources in distributed systems.<br />
Although often suggested in literature, the effectiveness<br />
of traditional bargaining and auction systems in<br />
grid environments is conceivably delimited as the trading<br />
objects are traded as unb<strong>und</strong>led standardized commodities.<br />
Consequently, these auction formats fail to express<br />
demand on b<strong>und</strong>les – exposing especially owners of CHP<br />
generators to the risk of selling only e.g. electricity without<br />
selling the co-generated heat.<br />
Reviewing the requirements upon the mechanism,<br />
it becomes obvious that the previous described mechanisms<br />
fail to satisfy these requirements. Most of the<br />
mechanisms proposed in literature do not meet the immediacy<br />
requirement in a way that they are either iterative<br />
in nature or NP hard to compute. Both properties<br />
rule out a use in highly interactive markets for different<br />
reasons. While iterative mechanisms are unsuited due<br />
to the fact that they frequent user feedback is needed,<br />
NP hard mechanisms consume too much time to be of<br />
use in large scale markets. Beside the requirement of immediacy,<br />
the negligence of capacity constraints for bids<br />
diminish the use of the proposed market mechanisms.<br />
To account for predictability of the system time<br />
attributes, Wellman et al. model single-sided auction<br />
protocols for the allocation and scheduling of resources<br />
<strong>und</strong>er consideration of different time constraints . Conen<br />
goes one step further by designing a combinatorial<br />
bidding procedure to improve the resource scheduling offering<br />
different running, starting, and ending times for<br />
different resources. Furthermore, Bapna et al propose<br />
an auction that allows to bid on b<strong>und</strong>les and on time<br />
slots. In addition, the authors also suggest a heuristic<br />
that approximates the outcome of the auction in polynomial<br />
time. However, these approaches are single-sided<br />
and hence do not create competition on both sides. Furthermore<br />
in electricity grids it might not be clear, when<br />
a consumer wants to start consuming energy and how<br />
long his consumption will take. Thus integrating time<br />
schedules into the auction mechanism does not solve the<br />
310
Requirements R1 R2 R3 R4 R5 R6 R7<br />
√<br />
√<br />
Eymann et al. 2003<br />
√<br />
–<br />
√<br />
– – – –<br />
Buyya et al. 2001<br />
√<br />
–<br />
√<br />
– – – –<br />
Wolski et al. 2003<br />
√<br />
–<br />
√<br />
– – – –<br />
Regev et al. 1998<br />
–<br />
√ √<br />
– – – –<br />
Wellman et al. 2001 –<br />
√ √<br />
– – – –<br />
Conen 2002 –<br />
√ √<br />
– – – –<br />
Parkes et al. 2001 –<br />
√ √<br />
– – – –<br />
Biswas et al. 2003 –<br />
– – – –<br />
Bapna et al. 2006 –/ √ √ √ √ √ √<br />
– – – –<br />
√<br />
Schnizler et al. 2006<br />
√<br />
–<br />
√<br />
– –<br />
Entchev 2003<br />
– – –/ √ √<br />
√ √ √<br />
– –<br />
Engler 2005<br />
– –<br />
–<br />
Legend: R1: online mechanism; R2: b<strong>und</strong>le allocation; R3: price signals; R4: capacity<br />
constraints; R5: stable operation; R6: coupling with large grid; R7: forecasts<br />
Table 1: Literature Overview<br />
problem of providing precise estimates on future consumption<br />
and production of the grid.<br />
Demanding competition on both sides suggests the<br />
development of a combinatorial exchange. In literature,<br />
Parkes et al. introduce the first combinatorial exchange<br />
as a single-shot sealed bid auction . As payment scheme,<br />
Vickrey discounts are approximated. Biswas and Narahari<br />
propose an iterative combinatorial exchange based<br />
on a primal/dual programming formulation of the allocation<br />
problem. By doing so, the preference elicitation<br />
problem can be alleviated, as the bidders can restrict<br />
their attention to some preferred b<strong>und</strong>les in contrast<br />
to all 2n1 possible combinations. Obviously, both<br />
approaches do not account for immediacy demands are<br />
thus not directly applicable for the problem at hand.<br />
The MACE mechanism developed by Schnizler et<br />
al. satisfies the b<strong>und</strong>ling requirement and also allows the<br />
modeling of capacity constraints but is NP hard and cannot<br />
account for application scenarios with more than 500<br />
bids and asks within a reasonable time span . Accordingly,<br />
this mechanism can be used for batch applications,<br />
but not for those applications that require immediacy as<br />
is the case in the motivating scenario.<br />
Enchev [11] proposes a different approach in that he<br />
uses a fuzzy logic based optimization algorithm for adjusting<br />
the energy production to a given (and possibly<br />
changing) demand pattern. This approach appears to be<br />
suitable for ensuring stable operation of the micro electricity<br />
grid, but as there are no price signals provided to<br />
indicate the scarcity of resources, consumers do not have<br />
an incentive to optimize there consumption schedules.<br />
Engler et. al. [10] exploit the potentials of an<br />
agent based system for controlling a micro electricity<br />
grid. They show that the system operates stable <strong>und</strong>er<br />
realistic conditions and even recovers automatically from<br />
exogenous shocks (brownouts). Still in this approach<br />
heat is not considered as a resource, furthermore price<br />
signals are missing so that also in this case consumers<br />
do not have an incentive to optimize their consumption.<br />
This paper intends to tailor a mechanism for the<br />
simultaneous allocation of electricity and heat resources<br />
by coupling two separate call markets together [9] so that<br />
it accounts for the aforementioned requirements.<br />
4 Modeling an Auction Based<br />
Energy Allocation Mechanism<br />
The base model<br />
Based on the aforementioned requirements we propose<br />
a double auction mechanism that is capable of coallocating<br />
electricity and heat and additionally takes<br />
minimum and maximum allocation constraints for both<br />
goods into account (if specified by the bidders). For the<br />
description of our model we use the following notation<br />
311
with the superscripts e and h indicating the variable’s<br />
dedication to electricity or heat respectively:<br />
i<br />
j<br />
h i<br />
h j<br />
e i<br />
e j<br />
b h i<br />
a h j<br />
βi<br />
h<br />
βi<br />
h<br />
αj<br />
h<br />
αj<br />
h<br />
x h i<br />
yj<br />
h<br />
Index for buyers i = 1 . . . I<br />
Index for sellers j = 1 . . . J<br />
Quantity of heat requested by buyer i<br />
Quantity of heat offered by seller j<br />
Quantity of electricity requested by buyer i<br />
Quantity of electricity offered by seller j<br />
Maximum price per heat unit<br />
buyer i is bidding<br />
Minimum price per heat unit seller j<br />
is asking for<br />
Market allocation for buyer i‘s bid on heat<br />
Min. heat allocation constraint of buyer i<br />
Market allocation for seller j‘s ask for heat<br />
Minimum allocation constraint of seller j<br />
Binary decision variable for buyer i‘s bid on heat<br />
Binary decision variable for seller j‘s ask for heat<br />
Using this notation we can formulate our market<br />
model as a mixed-integer maximization problem. In particular<br />
the total welfare, i.e. the difference between all<br />
allocated buyers bids b i and all allocated sellers asks<br />
a j is maximized, subject to the resulting solution being<br />
feasible. A solution is feasible only if the consumed<br />
amount of heat is leq the generated amount of heat (1)<br />
and the generated amount of electricity equals the consumed<br />
amount of electricity (2). Furthermore if a buyer’s<br />
bid or a seller’s ask is allocated by the market for either<br />
heat or electricity it also has to be allocated for the other<br />
product, i.e. combinatorial bids on both products cannot<br />
be split by the auctioneer but have to be allocated<br />
simultaneously. Lastly a consumer can set minimum allocation<br />
constraints for heat and / or electricity (5)-(8)<br />
that have to be taken into account as well.<br />
max<br />
s.t.<br />
∑<br />
βi h x h i b h i + βi e x e i b e i − ∑ αj h yj h a h j + αjy e j e a e j<br />
j∈J<br />
i∈I<br />
∑<br />
β h i x h i h i ≥ 0 (1)<br />
αj h yj h h j − ∑<br />
j∈J i∈I<br />
∑<br />
βi e x e i e i = 0 (2)<br />
j∈J<br />
α e jy e j e j − ∑ i∈I<br />
y h j = y e j ∀j ∈ J (3)<br />
x h i = x e i ∀i ∈ I (4)<br />
0 ≤ α e j ≤ α e j ≤ 1 ∀j ∈ J (5)<br />
0 ≤ α h j ≤ α h j ≤ 1 ∀j ∈ J (6)<br />
0 ≤ β e i ≤ β e i ≤ 1 ∀i ∈ I (7)<br />
0 ≤ β h i ≤ β h i ≤ 1 ∀i ∈ I (8)<br />
x h i , x e i ∈ {0, 1} ∀i ∈ I (9)<br />
y h j , y e j ∈ {0, 1} ∀j ∈ J (10)<br />
Assume a seller j who operates a combined heat<br />
and power (CHP) generator. He is able to produce<br />
e j = 10 units of electricity which he wants to sell at<br />
a price of at least a e j = 20 ct/unit. As his generator<br />
always co-generates heat when producing electricity,<br />
j wants to sell the heat (h j = 30 units) simultaneously<br />
at a price of at least a h j 5 ct/unit. Furthermore, as<br />
his generator is not able to operate at a capacity of <<br />
50% of the norm capacity, he wishes to make sure that<br />
– in case his offer is matched – the allocated amount<br />
of electricity is ≥ αj<br />
e = 50% of the originally specified<br />
amount. For the heat on the other hand, seller<br />
j might have a thermae available capable of storing<br />
the produced heat if it cannot be sold, thus αj h = 0.<br />
In summary, the ask of seller j will consist of a tuple<br />
A(e j ; a e j ; αe j ; h j; a h j ; αh j ) = A(10; 20; 50; 30; 5; 0).<br />
Similarly a buyer i might want to buy e i = 10units<br />
of electricity for at most b e i = 25ct/unit but he is<br />
neither interested in heat nor in partial allocations.<br />
Thus buyer i‘s bid will be B(e i ; b e i ; βe i ; h i; a h i ; βh i ) =<br />
A(10; 25; 100; Nil; Nil; Nil).)<br />
In general, combinatorial allocation problems as the<br />
one described above belong to the complexity class of<br />
NP-hard problems but this does not mean that every<br />
possible problem instance will be NP hard. Instead there<br />
usually exist problem instances that can be easily computed<br />
in polynomial time. We will further exploit this<br />
312
particular property for the design of our heuristic in the<br />
extended model described in the following section. For<br />
the example above we can easily solve the allocation<br />
problem by awarding the 10 units of electricity offered<br />
by seller j to consumer i ignoring the heat that j offered<br />
as well (αj h = αj h = 0). Having solved the allocation<br />
problem like this, the pricing for the transaction still<br />
needs to be determined. In general one possible solution<br />
is to employ a Vickrey Clarke Groves (VCG) like<br />
mechanism, which is the only class of mechanisms that<br />
provides allocative efficient, individual rational and incentive<br />
compatible solutions [14]. But as solving VCG<br />
problems is NP-hard and furthermore VCG mechanisms<br />
cannot be budget balanced [19], thus we will use the<br />
maximum execution principle to determine prices. The<br />
idea is described in more detail in the next section.<br />
The extended model<br />
In this section we describe a heuristic that is capable<br />
of efficiently finding near-optimal solutions solving the<br />
aforementioned allocation and pricing problem in polynomial<br />
time by particularly exploiting the fact that in<br />
micro electricity grids, most of the consumers and some<br />
of the producers do not need to allocate electricity and<br />
heat at the same time. In other words, in most of the<br />
cases separate double auctions for heat and electricity<br />
would be sufficient, only e.g. for producers that rely on<br />
CHP generators, submitting b<strong>und</strong>le offers might be advantageous.<br />
Thus our solution is to set up two slightly adapted<br />
open book call markets [9] running parallel and independently<br />
from each other. Table 2 depicts the orderbooks<br />
of an electricity and a heat call market that are meant<br />
to run in parallel. Call market trading basically means<br />
that incoming orders are collected in an open order book<br />
up to a predefined point in time. Then all orders are executed<br />
in a single multilateral trade at a price that best<br />
matches the aggregated asks and bids. On the buy side,<br />
all bids with limit prices greater or equal to the matching<br />
price are executed and vice versa for the sell side.<br />
If e.g. the ask side of the orderbook is longer, meaning<br />
more ask than buy orders exist at the clearing price,<br />
those ask orders are executed at the same fraction (pro<br />
Our<br />
rata) ensuring for each order that a potential minimum<br />
allocation constraint αj e (αh j ) is not violated.<br />
solution for coupling the two markets together<br />
thus allowing combinatorial bids is simple. Besides the<br />
two orderbooks we use an extra table to keep track of the<br />
combinatorial constraints. In our case Table 3 is used to<br />
record these constraints 4 . The first row states that the<br />
order with order id oid e = 1 and the one with order id<br />
oid h = 2 have to be either executed together or not at<br />
all (AND order). Now, when the time for matching the<br />
markets is reached, each of the markets determines its<br />
optimal set of matched orders independently from the<br />
other market. In this case we extended the previous<br />
example. The electricity market is matched as before<br />
but for the heat market we now have an additional ask<br />
(oid 4) and an additional buy order (oid 5). With the<br />
only bidder in this auction offering to buy heat at most<br />
6 ct per unit and two potential sellers offering 10 units<br />
of heat for at least 4 ct/unit and 30 units for at least<br />
5 ct/unit respectively, the price will be set to 5 ct/unit<br />
in order to maximize the exchange volume. As a result,<br />
seller j = 3 sells 10 units of heat, seller j = 1 sells 20<br />
units of heat, and buyer i = 4 buys 30 units of heat in<br />
this auction, each for 5 ct/unit.<br />
As soon as this task is accomplished both markets<br />
indicate the results of the matching in the b<strong>und</strong>ling constraint<br />
table. Orders for which the b<strong>und</strong>ling constraint is<br />
violated are then deleted from both, the order books and<br />
the b<strong>und</strong>ling constraint table before the market matching<br />
is executed again. This procedure is repeated until<br />
no violations occur anymore. In our case, the b<strong>und</strong>ling<br />
constrained is not violated, thus the matching process is<br />
completed.<br />
A controller that implements the mechanism as described<br />
so far would already fulfill requirements R2 (b<strong>und</strong>le<br />
allocation), R3 (price signals), and R4 (capacity constraints)<br />
but, as stated before, this is not sufficient for<br />
operating a micro CHP grid. In particular, this mechanism<br />
does not describe when the allocated resources<br />
need to be available and for how long. We overcome this<br />
limitation by discretizing time into distinct, consecutive<br />
slots of a predefined duration d, e.g. d = 15min. Then<br />
n b<strong>und</strong>le call auctions are set up similar futures markets<br />
at the stock exchanges one for each of the next n time<br />
slots (n being sufficiently large). We adjust the auctions’<br />
execution time intervals intervals t such that t ≤ d. Also<br />
we ensure that a future market is closed well timed be-<br />
4 Non-combinatorial orders go directly into the respective markets<br />
and are thus not recorded in this table.<br />
313
Orderbook Electricity<br />
Orderbook Heat<br />
Ask Bid Ask Bid<br />
oid j e j a e j αj e oid i e i b e i βi e oid j e j a e j αj e oid i e i b e i βi<br />
e<br />
1 1 10 20 5 3 2 10 25 1 4 3 10 4 .2 5 4 30 6 1<br />
2 1 30 5 nil<br />
Table 2: Sample Orderbooks<br />
oid e oid h type match e match h<br />
1 2 AND true true<br />
Table 3: B<strong>und</strong>ling Constraints<br />
fore the <strong>und</strong>erlying time slot starts. Like this all market<br />
participants can submit several orders to the respective<br />
market adjusting their combined electricity and heat demands<br />
and supplies for the next n time slots.<br />
Setting up our mechanism like this has several advantages.<br />
Market prices on the respective markets indicate<br />
the energy scarcity in the respective time slot and<br />
thus help users decide whether to use (or not to use)<br />
energy in certain time slots. In case a user decides to<br />
change his consumption schedule he can then immediately<br />
submit orders to sell energy in case of excess capacities<br />
or to buy energy in case of increased demand. Thus<br />
micro grid particpants can continuously adjust their demand<br />
and supply profiles over time, which in turn meets<br />
our requirement R1 (online mechanism).<br />
The two requirements R6 (coupling with large-scale<br />
power grid) and R7 (forecasts for demand and supply)<br />
are now relatively easy to satisfy by introducing a special<br />
market participant namely, a so-called arbitrage agent.<br />
Located at the coupling point (c.f. Figure 1 (4)) of the<br />
microgrid this agent can buy electricity from the largescale<br />
grid (e.g. at the EEX) and immediately sell it<br />
inside the microgrid markets at a higher price if electricity<br />
is cheaper in the outside world and vice versa. On<br />
average this agent (i) levels market prices in both markets,<br />
(ii) ensures that generators in the microgrid are<br />
only turned on if they can be operated at competitive<br />
costs, and (iii) communicates the future aggregated energy<br />
demand (supply) of the microgrid to the large-scale<br />
grid through placing his orders on the outside market.<br />
With the arbitrage agent in place our mechanism<br />
now fulfills all requirements but one for operating a micro<br />
energy: Stable operation (R5)). We claim that one<br />
average markets are very good of matching demand and<br />
supply. But in particular on average means not always.<br />
We already described the procedure for allocating energy<br />
resources in case of unbalanced orderbooks, which<br />
basically condenses to rationing one side of the market.<br />
In case supply is rationed, nothing much will happen besides<br />
some generators being operated at a reduced workload<br />
or even being turned off. But in case of demand rationing<br />
this means at worst that energy production and<br />
consumption become unbalanced and thus a brownout<br />
or a blackout might occur.<br />
In order to cope with this challenge we need to introduce<br />
another (market) institution. Consider the point<br />
in time where a market will be closed because the <strong>und</strong>erlying<br />
timeslot starts. At this moment energy trading<br />
becomes impossible for the market participants leaving<br />
them with the risk that they can no longer intervent (i.e.<br />
buy or sell) in case predicted and actual demand (supply)<br />
diverge due to unforeseen events (e.g. a defect generator).<br />
In such a situation the whole micro grid, especially<br />
the electricity subgrid, is likely to become unstable and<br />
will eventually collapse.<br />
This is where the spinning reserves (c.f. Figure 1<br />
(3)) take over control as emergency coordinators. Spinning<br />
reserves do not trade their energy capacities on the<br />
above mentioned future markets instead they provide<br />
balancing power on demand throughout the currently<br />
running time slot. In other words, the spinning reserves<br />
are responsible to level out energy (i.e. heat and electricity)<br />
demand and supply on the spot beyond the market’s<br />
capabilities ensuring a stable operation of the micro grid<br />
and thus fulfilling our last requirement R5.<br />
In order to set an incentive for regular consumers<br />
and producers not to rely on the spinning reserves, the<br />
effectively provided balancing power (heat) is charged<br />
314
ex post to those market participants who originated the<br />
market imbalance by deviating from their originally negotiated<br />
energy consumption (production) level. The<br />
spinning reserves are price sufficiently above the market<br />
price in order to provide market participants with a<br />
strong incentive to negotiate exactly the amount of energy<br />
that they are going to consume (produce) during a<br />
certain time slot. In other words, truth revelation becomes<br />
a dominant strategy for the market participants<br />
the shorter the remaining time until the start of a slot<br />
becomes.<br />
5 Conclusion & Outlook<br />
In this paper we describe a market based system for coordinating<br />
combined heat and power micro energy grids.<br />
We use b<strong>und</strong>led open book call markets to let consumers<br />
and producers of the microgrid negotiate the allocation<br />
their future energy (i.e. heat and electricity) demand<br />
and supply. Furthermore we propose the utilization of<br />
an arbitrage agent for connecting the micro (electricity)<br />
grid to the large-scale grid. We also describe a way to<br />
employ spinning reserves for stabilizing the micro grid<br />
on the spot without inferencing the principal of market<br />
based coordination.<br />
Overall the approach for operating micro energy<br />
grids described in this paper is very interesting as it proposes<br />
several new research avenues:<br />
• The efficiency of the mechanism introduced in this<br />
paper needs to be further evaluated, especially the<br />
average gap between optimal allocations and those<br />
allocations generated by the heuristic needs to be<br />
determined in more detail.<br />
• Subsequently, agent based simulations could be used<br />
to further study the system’s dynamics, e.g. its reactivity<br />
to exogenous shocks such as a sudden generator<br />
breakdown [15].<br />
• Electronic agents need to be developed that automate<br />
the process of energy trading. Here a clear<br />
advantage of our model is its simplicity. The fact<br />
that e.g. consumer agents can be implemented for<br />
procuring only heat or electricity reduces complexity<br />
and allows the recource to related literature from<br />
algorithmic trading research.<br />
• Trading strategies for different types of producers<br />
(i.e. wind turbines vs. micro CHP generator) and<br />
consumers need to be developed.<br />
• Innovative preference elicitation methods are required<br />
in order to allow e.g. consumers to only specify<br />
target temperatures for certain times of the day<br />
thus leaving the task of determining the resulting<br />
optimal heat consumption profile to the agents.<br />
• Different energy prices for different timeslots set an<br />
incentive for demand side optimization. In this area,<br />
new models need to be developed that support consumers<br />
in finding optimal or at least more efficient<br />
consumption schedules without reducing their quality<br />
of life.<br />
• The potentials and dynamics of hierarchically organized<br />
energy grids as well as potential yo-yo effects,<br />
which might occur if a large number of microgrids<br />
are coupled together hierarchically, need further investigation.<br />
6 Acknowledgement<br />
This work was supported in part by the German Federal<br />
Ministry of Education and Research (SESAM - Selforganization<br />
and Spontaneity in Liberalized and harmonized<br />
Markets, Grant 01-AK-703).<br />
References<br />
[1] S. Abu-Sharkh, R. J. Arnold, J. Kohler, R. Li, T. Markvart,<br />
J. N. Ross, K. Steemers, P. Wilson, and R. Yao.<br />
Can microgrids make a major contribution to uk energy<br />
supply? Renewable and Sustainable Energy Reviews,<br />
10(2):78–127, April 2006.<br />
[2] R. Bapna. When snipers become predators: Can mechanism<br />
design save online auctions? Communications of<br />
the ACM, 46(12):152–158, 2003.<br />
[3] S. Biswas and Y. Narahari. Iterative dutch combinatorial<br />
auctions. Annals of Mathematics and Artificial<br />
Intelligence, page forthcoming, 2003.<br />
[4] B. Buchholz and U. Schluecking. Energy <strong>management</strong><br />
in distribution grids european cases. Power Engineering<br />
Society General Meeting, 2006. IEEE, pages 2 pp.+,<br />
2006.<br />
315
[5] R. Buyya, H. Stockinger, J. Ghiddy, and D. Abramson.<br />
Economic models for <strong>management</strong> of resources in peerto-peer<br />
and grid computing. In International Conference<br />
on Commercial Applications for High Performance<br />
Computing, Denver, CO, 2001.<br />
[6] D. Coll-Mayor, M. Paget, and E. Lightner. Future intelligent<br />
power grids: Analysis of the vision in the european<br />
union and the united states. Energy Policy,<br />
35(4):2453–2465, April 2007.<br />
[7] W. Conen. Economically coordinated job shop scheduling<br />
and decision point bidding - an example for economic<br />
coordination in manufacturing and logistics. In Workshop<br />
on Planen, Scheduling <strong>und</strong> Konfigurieren, Entwerfen,<br />
Freiburg, 2002.<br />
[8] M. Conrad, J. Dinger, H. Hartenstein, D. Rolli,<br />
M. Schller, and M. Zitterbart. A peer-to-peer framework<br />
for electronic markets. In R. Steinmetz and K. Wehrle,<br />
editors, Peer-to-Peer Systems and Applications, volume<br />
Lecture Notes in Computer Science 3485. Springer,<br />
2005.<br />
[9] N. Economides and R. Schwartz. Electronic call market<br />
trading. Journal of Portfolio Management, 21(3):10–18,<br />
1995.<br />
[10] A. Engler, C. Hardt, J. Jimeno, J. Ruela, and J. Oyarzabal.<br />
Agent based micro grid <strong>management</strong> system.<br />
In International Conference on Future Power Systems,<br />
November 2005.<br />
[11] E. Entchev. Residential fuel cell energy systems performance<br />
optimization using ”soft computing” techniques.<br />
Journal of Power Sources, 118(1-2):212–217, May 2003.<br />
[12] T. Eymann, S. Sackmann, and G. Mueller. Hayeks<br />
katallaxie - ein zukunftsweisendes konzept fuer<br />
die wirtschaftsinformatik. Wirtschaftsinformatik,<br />
45(5):491–496, 2003.<br />
[13] P. Gerling, H. Rempel, U. Schwarz-Schampera, and<br />
T. Thielemann. Reserven, ressourcen <strong>und</strong> verfügbarkeit<br />
von energierohstoffen. Technical report, B<strong>und</strong>esanstalt<br />
für Geowissenschaften <strong>und</strong> Rohstoffe, 2005.<br />
[14] J. Green and J.-J. Laffont. Characterization of satisfactory<br />
mechanisms for the revelation of preferences for<br />
public goods. Econometrica, 45(2):427.438, 1977.<br />
[15] Haque, Nadim, Jennings, Nicholas, Moreau, and Luc.<br />
Scalability and robustness of a network resource allocation<br />
system using market-based agents. Netnomics,<br />
7(2):69–96, August 2005.<br />
[16] A. Klein. Comparative costs of energy in europe 2007.<br />
Clean Power Generation Advisory CPG 725-070110,<br />
Emerging Energy Research, January 2007.<br />
[17] W. Krewitt and B. Schlomann. Externe kosten der<br />
stromerzeugung aus erneuerbaren energien im vergleich<br />
zur stromerzeugung aus fossilen energietraegern. Technical<br />
report, Deutsches Zentrum fuer Luft- <strong>und</strong> Raumfahrt<br />
and Fraunhofer <strong>Institut</strong> fuer System- <strong>und</strong> Innovationsforschung,<br />
2006.<br />
[18] R. Lasseter, A. Akhil, C. Marnay, J. Stevens, J. Dagle,<br />
R. Guttromson, S. A. Meliopoulous, R. Yinger, and<br />
J. Eto. The certs microgrid concept - white paper on<br />
integration of distributed energy resources. Technical<br />
report, U.S. Department of Energy, April 2002.<br />
[19] R. B. Myerson and M. A. Satterthwaite. Efficient mechanisms<br />
for bilateral trading. Journal of Economic Theory,<br />
28:265–281, 1983.<br />
[20] D. C. Parkes, J. Kalagnanam, and M. Eso. Achieving<br />
budget-balance with vickrey-based payment schemes<br />
in exchanges. In Proceedings of the 17th International<br />
Joint Conference on Artificial Intelligence, pages 1161–<br />
1168, 2001.<br />
[21] O. Regev and N. Nisan. The popcorn market - an<br />
online market for computational resources. In First<br />
international conference on Information and computation<br />
economies, Charleston, South Carolina, 1998. ACM<br />
Press.<br />
[22] B. Schnizler, D. Neumann, D. Veit, and C. Weinhardt.<br />
Trading grid services - a multi-attribute combinatorial<br />
approach. European Journal of Operational Research,<br />
In Press, Corrected Proof:forthcoming, 2006.<br />
[23] K. Subramoniam, M. Maheswaran, and M. Toulouse.<br />
Towards a micro-economic model for resource allocation<br />
in grid computing systems. In Subramoniam, editor,<br />
Proceedings of the 2002 IEEE Canadian Conference<br />
on Electrical & Computer Engineering, volume 2, pages<br />
782–785 vol.2, 2002.<br />
[24] V. VDEW. Stromverbrauch der privaten haushalte –<br />
zehn prozent des stroms für kommunikation <strong>und</strong> information.<br />
Online Article, May 2006.<br />
[25] J. A. Villar and F. L. Joutz. The relationship between<br />
crude oil and natural gas prices. Technical report, Energy<br />
Information Administration, 2006.<br />
[26] M. P. Wellman, W. E. Walsh, P. Wurman, and<br />
J. MacKie-Mason. Auction protocols for decentralized<br />
scheduling. Games and Economic Behavior, 35(271-<br />
303):271–303, 2001.<br />
[27] R. Wolski, J. Brevik, J. Plank, and T. Bryan. Grid<br />
resource allocation and control using computational<br />
economies. In Grid Computing - Making The Global<br />
Infrastructure a Reality, page chapter 32. John Wiley &<br />
Sons, 2004.<br />
316
Lebenslauf<br />
Dr. Dirk Neumann, M.A. (Univ. Wisc) Tel. (privat) 0721-6238678<br />
Rintheimer Str. 78 Tel. (univ.) 0721-6088377<br />
76131 Karlsruhe Email: neumann@kit.edu<br />
Geboren am 15. März 1974<br />
Geboren in Frankfurt/Main<br />
Familienstand ledig<br />
1. Ausbildung<br />
15.07.2004 Promotion zum Dr. rer. pol. in Wirtschaftswissenschaften (summa<br />
cum laude)<br />
Universität Karlsruhe (TH) Fakultät für Wirtschaftswissenschaften<br />
Thema der Dissertation „Market Engineering – A Structured<br />
Design Process for Electronic Markets”. Doktorvater: Prof. Dr.<br />
Christof Weinhardt<br />
08.06.2000 Diplom in Ökonomie an der Justus-Liebig Universität Gießen<br />
(Note: 1,3: 1. von 83)<br />
16.05.1999 Master of Arts (M.A.) in Economics an der University of Wisconsin,<br />
Milwaukee (GPA: 3,97/4,00)<br />
18.08.1998 Immatrikulation für den Masterstudiengang Economics<br />
an der University of Wisconsin, Milwaukee<br />
29.07.1996 Vordiplom in Wirtschaftswissenschaften<br />
an der Justus-Liebig Universität, Giessen<br />
03.10.1994 Immatrikulation für den Diplomstudiengang Wirtschaftswissenschaften<br />
an der Justus-Liebig Universität, Giessen<br />
14.06.1993 Abitur an dem katholischen Privatgymnasium St. Lioba<br />
in Bad Nauheim, Leistungskurse Mathematik <strong>und</strong> Physik<br />
August 1984 St. Lioba Gymnasium, Bad Nauheim<br />
August 1980 Frauenwald Gr<strong>und</strong>schule, Bad Nauheim, Nieder-Mörlen<br />
317
2. Berufserfahrung<br />
1.10.2004 – heute Wissenschaftlicher Assistent (C1) am Lehrstuhl für Informationsbetriebswirtschaftslehre<br />
der Universität Karlsruhe (TH)<br />
1.6.2000–30.9.2004 Wissenschaftlicher Mitarbeiter<br />
2001 – 2004 Wissenschaftlicher Mitarbeiter (BAT IIa) am Lehrstuhl<br />
für Informationsbetriebswirtschaftslehre bei Prof. Christof<br />
Weinhardt, Fakultät für Wirtschaftswissenschaften Universität<br />
Karlsruhe (TH)<br />
2000 – 2001 Wissenschaftlicher Mitarbeiter (BAT IIa) am Lehrstuhl<br />
für BWL-Wirtschaftsinformatik bei Prof. Christof Weinhardt,<br />
Fakultät für Wirtschaftswissenschaften Justus Liebig Universität<br />
Giessen<br />
1996 – 2000 Wissenschaftliche Hilfskraft<br />
1999 – 2000 Stud. Hilfskraft Tutor <strong>und</strong> Übungsgruppenleiter für<br />
Wirtschaftsinformatik (Programmierung Visual Basic), Prof.<br />
Weinhardt, Universität Gießen<br />
1996 – 1998 Stud. Hilfskraft Tutor <strong>und</strong> Übungsgruppenleiter für<br />
Wirtschaftsinformatik (Programmierung Visual Basic), Prof.<br />
Weinhardt, Universität Gießen<br />
1993 – 1994 Gr<strong>und</strong>wehrdienst in Gotha <strong>und</strong> Stadtallendorf<br />
Panzeraufklärungseinheit<br />
318
3. Eingeladene Vorträge<br />
• Entwicklungsperspektiven des Grid Computing, im Rahmen des 7.@kit-Kongresses<br />
"Vernetztes Rechnen – Softwarepatente – Web 2.0" 2007, Potsdam, 21.06.2007.<br />
• A Decision Support System for Designing Auctions, im Rahmen des Research Symposiums<br />
"Hot Topics in Negotiation Support Systems" an der HICSS Conference 2006,<br />
Po’ipu, Hawaii, 04.01.2006.<br />
• A Framework for Assessment of e-Negotiation Systems – The Case of Online Auctions<br />
and E-negotiations, im Rahmen des Research Symposiums "Hot Topics in Negotiation<br />
Support Systems" an der HICSS Conference 2006, Po’ipu, Hawaii, 04.01.2006.<br />
• Computer Aided Market Engineering, im Rahmen der Vortragsreihe der John Molson<br />
School of Business in Montreal 2005, Montreal, Kanada, 12.10.2005.<br />
• P2P – Technical and Economic Fo<strong>und</strong>ations, Tutorial zusammen mit Prof. Dr. Klemens<br />
Böhm im Rahmen der 7th International IEEE Conference on E-Commerce Technology,<br />
München 21.07.2005.<br />
• Market Engineering – Wissensbasierte Unterstützung der Konzeptphase, E-Commerce<br />
Kompetenzzentrum Abschluss-Kolloquium, Dresden, 18.11.2004.<br />
• A Market Engineering – Knowledge Based Auction Design Markets, negotiations and<br />
dispute resolution in the new economy, Montreal, Canada, October 12-13, 2004, Montreal,<br />
Kanada, 12.10.2004.<br />
• Market Engineering and Auction Design: Pricing Per Column (PPC), im Rahmen der<br />
CORS/INFORMS Tagung in Banff 2004, Banff, Kanada, 16.05.2004.<br />
• eNegotiation –Taxonomy and Design, im Rahmen des IFIP Workshops “B2B Markets –<br />
Beyond MRO” in Zürich 2001, Zürich, Schweiz, 03.10.2001.<br />
• The London Classification Scheme – An Application on Commercial eNegotiations, im<br />
Rahmen des eNegotiation Workshops in Montreal 2001, Montreal, Kanada, 05.08.2001.<br />
4. Lehre<br />
Vorlesungen<br />
• Vorlesung “Gr<strong>und</strong>züge der <strong>Informationswirtschaft</strong>”, im Wintersemester 2005/2006<br />
Pflichtvorlesung für Informationswirte, ca. 80 Studierende.<br />
• Vorlesung “Programmieren in Visual Basic”, im Wintersemester 2001/2002 (6 Termine),<br />
Justus-Liebig-Universität Gießen<br />
Pflichtvorlesung im Gr<strong>und</strong>studium für Wirtschaftswissenschaftler, ca. 300 Studierende.<br />
• Vorlesung “Information and Market Engineering” (engl.), im Sommersemeter 2006<br />
– Vorlesung im Rahmen des Executive Master Programs „Information Engineering“ der<br />
Hector School<br />
• Vorlesung “Management of Business Networks” (engl.), im Wintersemester<br />
2004/2005 – Vorlesung enthält eine Fallstudie, die mit den kanadischen Partnern der<br />
Concordia University abgehalten wird.<br />
Wahlpflichtvorlesung, ca. 15 Studenten.<br />
• Vorlesung “Market Engineering” (engl.), im Sommersemester 2005<br />
Wahlpflichtvorlesung, ca. 20 Studenten.
• Vorlesung “Market Engineering” (engl.), im Sommersemester 2006<br />
Wahlpflichtvorlesung, ca. 25 Studenten.<br />
• Vorlesung “Market Engineering” (engl.), im Sommersemester 2007<br />
Wahlpflichtvorlesung, ca. 25 Studenten.<br />
Seminare<br />
• eNegotiation I, SS 2001<br />
• eNegotiation II, WS 2001/2002<br />
• Management of Business Networks, SS 2005<br />
• Peer-to-Peer Economics, SS 2005<br />
• Grid Economics, WS 2005/2006<br />
• Mechanismen in verteilten Netzen (Mechanisms in Distributed Networks), in Kooperation<br />
mit Prof. Dr. Klemens Böhm, WS 2005/2006<br />
• Information Pricing, SS 2006<br />
• Supply Chain Management & Auktionen, SS 2006<br />
• Service Science. Management and Engineering, WS 2006/2007<br />
• Computer Online Game in Finance – Entwicklung eines browserbasierten Wirtschaftssimulationsspiels,<br />
WS 2006/2007<br />
• Automatisierte Verhandlungen <strong>und</strong> Algorithmen, SS 2007<br />
• Energie <strong>und</strong> Märkte, WS 2007/2008<br />
• Mobilfunk <strong>und</strong> Telekommunikation, WS 2007/2008<br />
Tutorien Vorlesungen<br />
• Tutorium (50 Teilnehmer) zur Vorlesung „Gr<strong>und</strong>züge der <strong>Informationswirtschaft</strong>“,<br />
Universität Karlsruhe (TH), Prof. Weinhardt, Wintersemester 2000/2001<br />
• Tutorium (50 Teilnehmer) zur Vorlesung „Gr<strong>und</strong>züge der <strong>Informationswirtschaft</strong>“<br />
(Hälfte der Termine), Universität Karlsruhe (TH), Prof. Weinhardt, Wintersemester<br />
2001/2002<br />
• Tutorium (50 Teilnehmer) zur Vorlesung „Gr<strong>und</strong>züge der <strong>Informationswirtschaft</strong>“<br />
(Hälfte der Termine), Universität Karlsruhe (TH), Prof. Weinhardt, Wintersemester<br />
2002/2003<br />
• Tutorium (15 Teilnehmer) zur Vorlesung „Koordination <strong>und</strong> Netzwerkeffekte im<br />
eBusiness“, Universität Karlsruhe (TH), Prof. Weinhardt, Sommersemester 2002<br />
• Tutorium (15 Teilnehmer) zur Vorlesung „Koordination <strong>und</strong> Netzwerkeffekte im<br />
eBusiness“, Universität Karlsruhe (TH), (Hälfte der Termine) Prof. Weinhardt, Sommersemester<br />
2003<br />
• Zwei Tutorien (je 25 Teilnehmer) zur Vorlesung „Wirtschaftsinformatik I“, Justus-<br />
Liebig-Universität Gießen, Prof. Weinhardt, Wintersemester 1996/1997<br />
• Drei Tutorien (je 25 Teilnehmer) zur Vorlesung „Wirtschaftsinformatik I“, Justus-<br />
Liebig-Universität Gießen, Prof. Weinhardt, Wintersemester 1997/1998<br />
• Zwei Tutorien (je 25 Teilnehmer) zur Vorlesung „Wirtschaftsinformatik I“, Justus-<br />
Liebig-Universität Gießen, Prof. Weinhardt, Wintersemester 1999/2000<br />
320
• Ein Tutorium (19 Teilnehmer) zur Vorlesung „Einführung in die BWL“, Justus-<br />
Liebig-Universität Gießen, Prof. Glaum, (3 Termine) Wintersemester 2000/2001<br />
Betreute Diplomarbeiten<br />
• Dr. Bjoern Schnizler, Iterative Kombinatorische Auktionen – iB<strong>und</strong>le vs. SAAPB. Abgabe<br />
im Dezember 2003 (Von 2004-2007 Assistent am Lehrstuhl Prof. Weinhardt), Note:<br />
1,3.<br />
• Thorsten Bendrich, Computational Mechanism Design im Umfeld kombinatorischer<br />
Auktionen. Abgabe im Januar 2004, Note: 2,0.<br />
• Dr. Henner Gimpel, Auktionsverfahren im Aktienprimärmarkt – Experimentelle Evaluierung<br />
der Informationseffizienz. Abgabe im Mai 2003. Ausgezeichnet mit dem 1.<br />
Hochschulpreis 2003 des Deutschen Aktieninstituts (DAI) in der Kategorie Diplomarbeiten.<br />
(Von 2003-2007 Assistent am Lehrstuhl Prof. Weinhardt), Note: 1,0.<br />
• Dr. Daniel Rolli, Eine dynamische Workflow-Standard-Erweiterung im Hinblick auf die<br />
Steuerung elektronischer kombinierter Verhandlungen. Abgabe im März 2003 (von<br />
2003-2006 Assistent am Lehrstuhl Prof. Weinhardt), Note: 1,3.<br />
• Tunc Berdan, Auctions versus Negotiations in Procurement., an der Concordia University,<br />
Montreal in Kooperation mit Gregory Kersten. Abgabe im März 2006, Note: 1,3.<br />
• Christoph Goebel, Securing Interdomain Routing: An Incentive Analysis, an der Carnegie<br />
Mellon University, Pittsburg in Kooperation mit Ramayya Krishnan. Abgabe im<br />
Juli 2006. (Seit 10/2006 Assistent am Lehrstuhl von Prof. Dr. Günther, Humboldt Universität<br />
Berlin), Note: 1,0.<br />
• Jochen Stoesser, Generic Market Platforms: Adaption and Application of meet2trade to<br />
Emissions Trading, an der University of Newsouth Wales, Sydney in Kooperation mit<br />
Fethi Rabhi. Abgabe im Juli 2006 (Seit 09/2006 Assistent am Lehrstuhl Prof. Weinhardt),<br />
Note: 1,3.<br />
• Kai Beckhaus, Innovation in Value Chain Management, in Kooperation mit AT Kearney,<br />
Abgabe im März 2007, Note 1,0.<br />
• Sibylle Stacklies, Innovation in Value Chain Management, in Kooperation mit AT<br />
Kearney, Abgabe im April 2007, Note 1,0.<br />
• Christian Wiedenhoff, Einflussgrößen innerhalb der Transportplanung auf eine gesamtkostenminimale<br />
Gestaltung des Logistiksystems Audi Neckarsulm, in Kooperation<br />
mit Audi AG. Abgabe im Januar 2007, Note 1,3.<br />
• Thorsten Hau, An Allocation Scheme for the Retail Business – How Auctions Can Ensure<br />
Allocative Efficiency, in Kooperation mit Lux International. Abgabe im Dezember<br />
2006 (Seit 01/2007 Assistent am Lehrstuhl von Prof. Dr. Brenner, St. Gallen), Note:<br />
1,0.<br />
• Lutz Jansen, Information Management Design Decomposition, in Kooperation mit ETH<br />
Zürich <strong>und</strong> Forschungszentrum Informatik (FZI), Abgabe im Dezember 2006, Note:<br />
1,3.<br />
• Christine Rössle, Decentralized Online Resource Allocation for Dynamic Web Service<br />
Applications, bei SUN Singapur <strong>und</strong> Nanyang Technological University in Kooperation<br />
mit Simon See, Abgabe im Juni 2007, Note: 1,3.<br />
• Philipp Bodenbenner, Pay-your-Bid Mechanisms for Resource Allocation in Grids, bei<br />
SUN Singapur <strong>und</strong> Nanyang Technological University in Kooperation mit Simon<br />
See, Abgabe im Juni 2007, Note: 1,0.<br />
321
• Marius Romanescu, Product delivery to Customers using company-internal Markets,<br />
Abgabe im April 2007, Note: 1,0.<br />
• Benno Schwaiger, Das Similarity-based-Cost-Estimation Verfahren zur Schätzung der<br />
IT-Kosten bei Kooperationen, in Kooperation mit BMW, Abgabe im August 2007.<br />
• Christian Schaub, Efficient Management of Information Resources Over Ad-Hoc Data<br />
Grids, laufend, an der University of Newsouth Wales, Sydney in Kooperation mit Fethi<br />
Rabhi.<br />
• Philip Caune, Evaluating different Market Mechanisms in Grids, laufend, an der Singapure<br />
Management University (SMU), Singapore in Kooperation mit Steven Miller.<br />
• Tim Püschel, Economically Enhanced Resource Management, laufend, am Barcelona<br />
Supercomputing Center (BSC), Barcelona in Kooperation mit Jordi Torres.<br />
• Michael Becker, Risk Management in Grids, laufend, an der Cardiff University, Wales<br />
in Kooperation mit Omer Rana.<br />
• Melanie Mossmann, Combinatorial Auctions in Grids, laufend, bei France Telecom‚<br />
Paris, Frankreich in Kooperation mit Ruby Krishnaswamy.<br />
• Matthias Deindl, Agent-Based Demand Management in Micro Energy Grids, laufend,<br />
an der Concordia University, Montreal in Kooperation mit Rustam Vahidov.<br />
• Tobias Heitfeld, Situated Decision Support Systems for Complex Services, laufend, an<br />
der Concordia University, Montreal in Kooperation mit Rustam Vahidov.<br />
5. Forschungsschwerpunkt <strong>und</strong> Drittmittel<br />
Aktuelle Forschungsschwerpunkte<br />
• Grid Computing <strong>und</strong> Internet Economics<br />
• Marktmechanismen in verteilten Netzen<br />
• Peer-to-Peer Economics<br />
• IT-Service-Management<br />
• Business Service Management<br />
• Autonomic Computing<br />
• Virtuelle Währungssysteme<br />
• Incentive Engineering <strong>und</strong> Service Science<br />
• Anreizgestaltung in kooperativen Systemen <strong>und</strong> Netzwerken<br />
• Geschäftsmodelle<br />
• Prozess<strong>management</strong>/ Kontinuierliche Verbesserungsprozesse<br />
• Situated Negotiation Support Systeme<br />
• Supply Chain Management <strong>und</strong> Market Engineering<br />
• Koordination in Wertschöpfungsnetzen<br />
• eProcurement Mechanismen<br />
• Anwendungen des Operation Research<br />
322
Laufende Forschungsprojekte<br />
1/8 Self-Organizing ICT Resource Management (SORMA)<br />
Förderung<br />
EU FP 6 IST 5th Call als STREP<br />
Förderzeitraum 01.08.2006 – 31.07.2008<br />
Beantragtes 2.700.000,00 €<br />
Gesamtvolumen<br />
Beantragter 609.034,63 €<br />
eigener Anteil<br />
Persönliche Rolle Koordinator/Projektmanager<br />
Projektpartner Forschungszentrum Informatik, Karlsruhe<br />
Cardiff University, Großbritannien<br />
Swedish <strong>Institut</strong>e of Computer Science, Schweden<br />
Research Center at the BFM Bayreuth<br />
Universitat Politècnica de Catalunya, Spanien<br />
University of Reading, Großbritannien<br />
Hebrew University, Israel<br />
Sun Microsystems, Singapur<br />
Correlation Systems, Israel<br />
TXT e-Solutions, Italien<br />
Barcelona Supercomputing Center, Spanien<br />
Kurzbeschreibung Konzeption, Implementierung <strong>und</strong> Einführung eines Open Grid<br />
Marktes für Grid Ressourcen.<br />
Webseite: http://www.sorma-project.eu/<br />
2/8 Billing the Grid<br />
Förderung<br />
Gefördert von der Landesstiftung Baden-Württemberg als Landesschwerpunktprogramm<br />
Förderzeitraum 01.11.2005 – 31.10.2007<br />
Gesamtvolumen 280.000,00 €<br />
Eigener Anteil 70.000,00 €<br />
Persönliche Rolle<br />
Projektpartner<br />
Kurzbeschreibung<br />
Projektleiter<br />
Universität Karlsruhe (TH)<br />
Konzeption <strong>und</strong> Implementierung von Zahlungs- <strong>und</strong> Abrechnungsfunktionalitäten<br />
für Grids im universitären Bereich.<br />
Webseite: http://www.billing-the-grid.org/<br />
3/8 Efficient Management of Information Resources Over Ad-Hoc Data Grids<br />
(ADAGE)<br />
Förderung<br />
DEST-ISL Competitive Grants Programme – Australien<br />
Ergänzungsantrag zu SORMA<br />
Förderzeitraum 01.01.2007 – 31.12.2008<br />
Gesamtvolumen 812.863,00 $<br />
Eigener Anteil 16.000,00 $<br />
Persönliche Rolle<br />
Projektpartner<br />
Workpackage Leader<br />
University of New South Wales, Sydney, Australien<br />
University of Sydney, Australien<br />
SIRCA, Australien<br />
Cardiff University, Großbritannien<br />
University of Lyon, Frankreich<br />
323
Kurzbeschreibung<br />
Vienna University of Technology, Österreich<br />
Das Ergänzungsprojekt zu SORMA zielt darauf ab, effiziente<br />
Methoden <strong>und</strong> Technologien für ad-hoc Data Grids zu entwickeln.<br />
Webseite: http://cgi.cse.unsw.edu.au/~soc/adage/<br />
4/8 Selbstorganisation <strong>und</strong> Spontaneität in liberalisierten <strong>und</strong> harmonisierten Märkten<br />
(SESAM)<br />
Förderung<br />
BMBF-Forschungsschwerpunktprogramm "Internetökonomie"<br />
Förderzeitraum 01.07.2003 – 31.09.2007<br />
Gesamtvolumen 3.063.927,00 €<br />
Eigener Anteil 63.170,34 € (p.a)<br />
Persönliche Rolle Projektverantwortlicher am <strong>Institut</strong><br />
Projektpartner ---<br />
Kurzbeschreibung Das interdisziplinäre Projekt “SESAM” bietet Infrastruktur <strong>und</strong><br />
ökonomische Mechanismen, um einen dezentralen Energiehandel<br />
zu ermöglichen.<br />
Webseite: http://www.sesam.uni-karlsruhe.de/<br />
5/8 Electronic Negotiations, Media and Transactions for Socio-Economic Interactions<br />
Förderung<br />
Social Sciences and Humanities Research Council of Canada<br />
(SSHRC)<br />
Förderzeitraum 01.04.2002 – 30.10.2007<br />
Eigener Anteil ca. 12.000,00 €<br />
Persönliche Rolle Projektverantwortlicher am <strong>Institut</strong><br />
Projektpartner Concordia University Montreal, Kanada<br />
University of Ottawa, Kanada<br />
McMasters University, Hamilton, Kanada<br />
University of Hawaii, USA<br />
TU München<br />
Universität Wien, Österreich<br />
University of Taiwan, Taiwan<br />
Kurzbeschreibung Analyse der Auswirkung von e-Negotiation-Protokollen auf das<br />
Verhalten menschlicher Verhandlungsführer. Design geeigneter<br />
e-Negotiation-Protokolle in verschiedenen Szenarien.<br />
Webseite: http://interneg.concordia.ca/enegotiation/index.html<br />
6/8 Georgia Tech Austauschprogramm<br />
Förderung<br />
ISAP Programm, DAAD<br />
Förderzeitraum 21.08.2006 – 03.08.2007<br />
Gesamtvolumen ca. 30.650,00 €<br />
Eigener Anteil ca. 30.650,00 €<br />
Persönliche Rolle Projektleiter<br />
Projektpartner Georgia <strong>Institut</strong>e of Technology, Atlanta<br />
Kurzbeschreibung Studentenaustausch mit dem Computer Science Department der<br />
Georgia Tech. Jährlicher Austausch von 3 Karlsruher Studenten<br />
im Master Programm.<br />
Webseite:<br />
http://www.wiwi.uni-<br />
324
karlsruhe.de/studium/auslandstudium/bewerbung.html<br />
7/8 CatNets<br />
Förderung<br />
EU IST FET Open als STREP<br />
Förderzeitraum 01.09.2004 – 30.08.2007<br />
Gesamtvolumen 1.414.000,00 €<br />
Eigener Anteil 284.275,00 €<br />
Persönliche Rolle Projektmitarbeiter<br />
Projektpartner Universität Bayreuth<br />
Universitat Politecnica de Catalunia, Spanien<br />
Universita delle merche Ancona, Italien<br />
University of Cardiff, Großbritannien<br />
ITC-irst Treno, Italien<br />
Universität Mannheim<br />
Kurzbeschreibung Untersuchung von dezentralen versus zentralen Marktmechanismen<br />
in Grid Märkten.<br />
Webseite: http://www.catnets.org/<br />
8/8 Moving Business to the Grid – An Application for the Automotive Industry<br />
(Biz2Grid)<br />
Förderung<br />
BMBF Projekt<br />
Förderzeitraum 01.06.2007 – 31.05.2010<br />
Gesamtvolumen 1.180.000,00 €<br />
Eigener Anteil 401.052,00 €<br />
Persönliche Rolle Projektmitarbeiter/Antragsteller<br />
Projektpartner Forschungszentrum Informatik, Karlsruhe<br />
BMW München<br />
IBM Deutschland GmbH<br />
Philipps-Universität Marburg<br />
Webseite: http://www.biz2grid.de/<br />
6. Gutachtertätikeiten<br />
Begutachtung von Beiträgen der Zeitschriften:<br />
• Concurrency and Computation: Practice and Expertise<br />
• International Journal of Electronic Markets<br />
• Journal of Grid Computing<br />
• Journal of Group Decision and Negotiation<br />
• Journal of Managament Information Systems (JMIS)<br />
• Journal of Organizational Computing and Electronic Commerce<br />
• Wirtschaftsinformatik<br />
Begutachtung von Beiträgen der Konferenzen <strong>und</strong> Workshops:<br />
• Agent-based Grid Computing Workshop<br />
• Cooperative Information Systems<br />
• EC-Web Conference<br />
325
• ECIS<br />
• EuroSys<br />
• Gecon Workshop<br />
• Grid Economics Workshop<br />
• HICSS<br />
• Multi-Konferenz Wirtschaftsinformatik<br />
• Wirtschaftsinformatik Tagung<br />
• Workshop on eBusiness – ICIS Pre-Conference<br />
Enagagement in internationalen Veranstaltungen<br />
• Chair des Workshops "Economic Models for Distributed System", im Rahmen der Mardigras<br />
Conference 2008, Baton Rouge, Louisiana.<br />
• Mitglied im Programm-Kommittee an der Konferenz "EC Web Conference", 2008<br />
• Mitglied im Programm-Kommittee der Teilkonferenz "Betriebswirtschaftliche Anwendungen<br />
des Grid Computing" im Rahmen der Multikonferenz Wirtschaftsinformatik<br />
2008 (MKWI '08), in München, 26.-28. Februar 2008.<br />
• Mitglied im Programm-Kommittee der Teilkonferenz "Märkte <strong>und</strong> Geschäftsmodelle<br />
für Service-orientiertes Computing <strong>und</strong> Grid-Netzwerke" im Rahmen der Multikonferenz<br />
Wirtschaftsinformatik 2008 (MKWI '08), in München, 26.-28. Februar 2008.<br />
• Mitglied im Programm-Kommittee an der Konferenz “Group Decision and Negotiation”,<br />
Coimbra, Portugal, 2008.<br />
• Chair des Workshops "Economic Models and Algorithms in Grid System", im Rahmen<br />
der Grid 2007, Austin, Texas.<br />
• Tutorial Chair an der Wirtschaftsinformatik Konferenz WI 2007 “eOrganisation: Service-,<br />
Process-, Market-Engineering” Karlsruhe, 2007.<br />
• Track Chair der Session “Autonomous Bidding Agents for Auctions and Negotiations”<br />
im Rahmen der Group Decision and Negotiation Konferenz (GDN07) in Montreal.<br />
• Mitglied im Programm-Kommittee am Workshop "Grid Economics and Business Models"<br />
(Gecon 2007), Rennes, Frankreich, 2007.<br />
• Mitglied im Programm-Kommittee an der Konferenz “Group Decision and Negotiation”,<br />
Montreal, Kanada, 2007.<br />
• Mitglied im Programm-Kommittee am Workshop "FinanceCom" im Rahmen der ICIS,<br />
Montreal, Kanada 2007.<br />
• Invited Speaker für das Tutorial “P2P – Technical and Economic Fo<strong>und</strong>ations” zusammen<br />
mit Prof. Dr. Klemens Böhm im Rahmen der 7th International IEEE Conference<br />
on E-Commerce Technology, München 21. Juli 2005.<br />
• Mitglied des CT1 Technical Collaboration Steering Committee der Europäischen<br />
Kommission für den Bereich Grid Computing.<br />
• Mitglied im Programm-Kommittee an der Konferenz "EC Web Conference", 2007<br />
326
• Mitglied im Programm-Kommittee am Workshop "Agent-based Grid Computing" for<br />
7 th IEEE International Symposium on Cluster Computing and the Grid (CCGrid07) in<br />
Rio de Janeiro, Mai 2007.<br />
• Mitglied im Programm-Kommittee der Konferenz "Group Decision and Negotiation“<br />
(GDN07) in Montreal.<br />
• Mitglied im Programm-Kommittee am Workshop "Agent-based Grid Computing" im<br />
Rahmen der 6 th IEEE International Symposium on Cluster Computing and the Grid<br />
(CCGrid06) in Singapore, 16. – 19. Mai 2006.<br />
• Mitglied im Programm-Kommittee am Track "Business Applications of P2P and Grid<br />
Computing" for Multikonferenz Wirtschaftsinformatik 2006 (MKWI '06), in Passau, 20.<br />
– 22. Februar 2006.<br />
• Mitglied im Programm-Kommittee am "4th Workshop on Grid Economics & Business<br />
Models for Grid” im Rahmen der Grid Asia Konferenz 2006, in Singapore, 16. Mai<br />
2006.<br />
• Mitglied im Programm-Kommittee der Konferenz "EC Web Conference", 2006<br />
• Mitglied im Programm-Kommittee am Workshop "Smart Grid Technologies“ (SGT06),<br />
Dublin, Ireland,<br />
• Eingeladener Teilnehmer für das Dagstuhl Seminar “Electronic Market Design” im Juli<br />
2002. These workshops consist of gatherings of leading experts in one research area for<br />
a week at the Dagstuhl Castle in Germany. Participation is by invitation only.<br />
• Eingeladener Teilnehmer für das Dagstuhl Seminar “Market and Negotiation Engineering”<br />
im November 2006. These workshops consist of gatherings of leading experts in<br />
one research area for a week at the Dagstuhl Castle in Germany. Participation is by invitation<br />
only<br />
• Organisation des Habilitanden Workshop im November 2005, in Schloss Rauischholzhausen<br />
Schriftenverzeichnis<br />
Beiträge in referierten Fachzeitschriften<br />
• D. Neumann, J. Stoesser, C. Weinhardt (2008): Briding the Adoption Gap – Developing<br />
a Roadmap for Trading Grids, Electronic Markets – The International Journal, 18(1),<br />
(akzeptiert am 26. August 2007), forthcoming.<br />
• B. Schnizler, D. Neumann, D. Veit, Ch. Weinhardt: A Multi-attribute Combinatorial<br />
Exchange for Trading Grid Services, European Journal of Operational Research<br />
(EJOR), (akzeptiert am 04. November 2006), forthcoming<br />
• C. Weinhardt, D. Neumann, C. Holtmann (2006) Computer Aided Market Engineering,<br />
Communications of the ACM, 49(7): 79.<br />
• D. Neumann, C. Orwat, C. Holtmann (2006) Grid Economics, Wirtschaftsinformatik,<br />
48(3): 206-209<br />
• Ch. Weinhardt, B. Weber, D. Neumann, T. Hendershott, R. Schwartz (2006) Financial<br />
Market Engineering, Electronic Markets – The International Journal, 16(2), 98-100<br />
327
• D. Rolli, M. Conrad, D. Neumann, C. Sorge (2006) Distributed Ascending Proxy Auction<br />
– A Cryptographic Approach, Wirtschaftsinformatik, 48(1): 7-15.<br />
• T. Eymann, O. Ardaiz, M. Catalano, P. Chacin, I. Chao, F. Freitag, M. Gallegati, G.<br />
Giulioni, L. Joita, L. Navarro, D. Neumann, O. Rana, M. Reinicke, R. Schiaffino, B.<br />
Schnizler, W. Streitberger, D. Veit, F. Zini (2006), Catallaxy-based Grid Markets, Multi-Agent<br />
Systems and Grid, Special Issue on Smart Grid Technologies & Market Models,<br />
1(4): 297 - 307<br />
• Ch. Weinhardt, C. Holtmann, D. Neumann (2003) Market Engineering, Wirtschaftsinformatik<br />
45(6): 635-640<br />
• D. Neumann, M. Benyoucef, S. Bassil, J. Vachon (2003) Applying the Montreal Taxonomy<br />
to State of the Art E-Negotiation Systems, Group Decision and Negotiation<br />
12(4): 287-310<br />
• C. Lattemann, D. Neumann (2002) Clearing <strong>und</strong> Settlement im Wandel – Eine Perspektive<br />
für den europäischen Wertpapierhandel, Zeitschrift für die gesamte Kreditwirtschaft:<br />
1159-1164.<br />
• M. Budimir, C. Holtmann, D. Neumann (2001) Design of a Best Execution Market,<br />
Revue Bancaire et Financière/Bank en Financiewezen: 66(2-3): 138-146.<br />
Monographien<br />
• D. Neumann (2007) Market Engineering - A Structured Design Process for Electronic<br />
Markets, in Universitätsverlag Karlsruhe<br />
Referierte Konferenzbeiträge<br />
1. J. Stoesser, D. Neumann, C. Weinhardt, Market-Based Pricing in Grids: On Strategic<br />
Manipulation and Computational Cost, Journal of AIS Sponsored Theory Development<br />
Workshop, Montreal, Canada.<br />
2. G. Kersten, E. Chen, D. Neumann, R. Vahidov, Technology Assessment and Comparison:<br />
The Case of Auction and E-negotiation Systems, Journal of AIS Sponsored Theory<br />
Development Workshop, Montreal, Canada.<br />
3. R. Vahidov, D. Neumann, Situated Decision Support for Service Level Agreement Negotiations,<br />
Hawaii International Conference on System Sciences (HICSS 2008), 7-10<br />
Januar 2008, Waikoloa, Hawaii.<br />
4. J. Stoesser, P. Bodenbenner, S. See, D. Neumann, A Discriminatory Pay-as-Bid Mechanism<br />
for Efficient Scheduling in the Sun N1 Grid Engine, Hawaii International Conference<br />
on System Sciences (HICSS 2008), 7-10 Januar 2008, Waikoloa, Hawaii.<br />
5. C. Block, D. Neumann, C. Weinhardt, A Market Mechanism for Energy Allocation in<br />
Micro-CHP Grids, Hawaii International Conference on System Sciences (HICSS 2008),<br />
7-10 Januar 2008, Waikoloa, Hawaii.<br />
6. M. Becker, N. Borissov, V. Deora, O. Rana, D. Neumann, Using k-pricing for Penalty<br />
Calculation in Grid Markets, , Hawaii International Conference on System Sciences<br />
(HICSS 2008), 7-10 Januar 2008, Waikoloa, Hawaii.<br />
328
7. R. Riordan, B. Blau, C. Weinhardt, D. Neumann, Collaborative Continuous Service<br />
Engineering, Hawaii International Conference on System Sciences (HICSS 2008), 7-10<br />
Januar 2008, Waikoloa, Hawaii.<br />
8. T. Püschel, N. Borissov, M. Macias, D. Neumann, J. Guitart, J. Torres, Economically<br />
Enhanced Resource Management for Internet Service Utilities, The 8th International<br />
Conference on Web Information Systems Engineering (WISE 2007), 3-7 Dezember<br />
2007, Nancy, France.<br />
• J. Stoesser, D. Neumann, GreedEx – A Scalable Clearing Mechanism for Utility Computing,<br />
Networking and Electronic Commerce Research Conference (NAEC) 2007, 18-<br />
21 October 2007, Riva Del Garda, Italy.<br />
• L. Amar, J. Stoesser, A. Barak, D. Neumann, Economically Enhanced MOSIX for Market-based<br />
Scheduling in Grid OS, Workshop on Economic Models and Algorithms for<br />
Grid Systems (EMAGS) 2007, 19 - 21 September 2007, Austin, USA.<br />
• T. Püschel, N. Borissov, D. Neumann, M. Macías, J. Guitart, J. Torres, Extended Resource<br />
Management Using Client Classification and Economic Enhancements, eChallenges<br />
e-2007 Conference, 24-26 October 2007, The Hague, Netherlands.<br />
• J. Stoesser, C. Roessle, D. Neumann, Decentralized Online Ressource Allocation for<br />
Dynamic Web Service Applications, IEEE Joint Conference on E-Commerce Technology<br />
(CEC'07) and Enterprise Computing, E-Commerce and E-Services (EEE '07), 23-26<br />
July 2007, Tokyo, Japan.<br />
• D. Neumann, J. Stoesser, C. Weinhardt, Bridging the Grid Adoption Gap – Developing<br />
a Roadmap for Trading Grids, 20th Bled eConference, Merging and Emerging Technologies,<br />
Processes, and <strong>Institut</strong>ions, June 4-6, 2007, Bled, Slovenia – Best Theme Paper<br />
Award.<br />
• A. Anandasivam, D. Neumann, C. Weinhardt, Rightsizing of Incentives for collaborative<br />
e-Science Grid Applications with TES, 20th Bled eConference, Merging and<br />
Emerging Technologies, Processes, and <strong>Institut</strong>ions, June 4-6, 2007, Bled, Slovenia.<br />
• P. Bodenbenner, J. Stoesser, D. Neumann, A Pay-as-Bid Mechanism for Pricing Utility<br />
Computing, 20th Bled eConference, Merging and Emerging Technologies, Processes,<br />
and <strong>Institut</strong>ions, June 4-6, 2007, Bled, Slovenia.<br />
• D. Neumann, C. Block, C. Weinhardt, Y. Karabulut, Knowledge-Driven Selection of<br />
Market Mechanisms in e-Procurement, European Conference on Information Systems<br />
(ECIS) 2007.<br />
• J. Stoesser, D. Neumann, A. Anandasivam, A Truthful Heuristic for Efficient Scheduling<br />
in Network-centric Grid OS, European Conference on Information Systems (ECIS)<br />
2007.<br />
• D. Neumann, J. Stoesser, A. Anandasivam, N. Borissov, SORMA – Building an Open<br />
Grid Market for Grid Resource Allocation, The 4th International Workshop on Grid<br />
Economics and Business Models (GECON 2007), 28 August 2007, Rennes, France.<br />
• D. Neumann, B. Schnizler, I. Weber, Ch. Weinhardt, Second Best Combinatorial Auctions<br />
– The Pricing-Per-Column Auction, Hawaiian International Conference of System<br />
Sciences (HICSS) 2007.<br />
• A. Anandasivam, D. Neumann, Token Exchange System as incentive mechanism for<br />
the e-Science Grid community, Workshop on Agents for Grid Computing 2007<br />
(AGC2007), Rio de Janeiro, Brasilien.<br />
329
• E. Chen, G. Kersten, D. Neumann, R. Vahidov, C. Weinhardt, B. Yu, A Framework for<br />
E-market System Assessment and Design, Group Decision and Negotiation, 2007,<br />
Montreal, Kanada.<br />
• K. Kolitz, D. Neumann, The Impact of Information System Design on eMarketplaces –<br />
An Experimental Analysis, Group Decision and Negotiation, 2007, Montreal, Kanada.<br />
• D. Neumann, Engineering Grid Markets, N. Jennings, G. Kersten, A. Ockenfels, C.<br />
Weinhardt (Eds.): Dagstuhl Seminar Proceedings 06461 Negotiation and Market Engineering,<br />
2006, Internationales Begegnungs- <strong>und</strong> Forschungszentrum (IBFI), Schloss<br />
Dagstuhl, Germany.<br />
• G. Kersten, E. Chen, D. Neumann, R. Vahidov, C. Weinhardt, On Comparison of Mechanisms<br />
of Economic and Social Exchanges: The Times Model N. Jennings, G. Kersten,<br />
A. Ockenfels, C. Weinhardt (Eds.): Dagstuhl Seminar Proceedings 06461 Negotiation<br />
and Market Engineering, Internationales Begegnungs- <strong>und</strong> Forschungszentrum<br />
(IBFI), Schloss Dagstuhl, Germany, 2006.<br />
• D. Neumann, Self-Organizing ICT Resource Management – Policy-based Automated<br />
Bidding, eChallenges e-2006 Conference, Barcelona.<br />
• J. Stoesser, A. Anandasivam, N. Borissov, D. Neumann, Economic Virtualization of<br />
ICT Infrastructures, Cracow Grid Workshop 2006.<br />
• D. Neumann, C. Holtmann, Ch. Weinhardt, Using Blueprints for Engineering Electronic<br />
Market Services, The First International Workshop on Service Intelligence and Service<br />
Science (SISS 2006), October 17, 2006, Hong Kong.<br />
• D. Neumann, S. Lamparter, B. Schnizler, Automated Bidding for Trading Grid Resources,<br />
European Conference on Information Systems ECIS 2006, Göteborg.<br />
• B. Schnizler, D. Neumann, D. Veit, Ch. Weinhardt, Moving Markets to the Grid, 68.<br />
wissenschaftlichen Jahrestagung des Verbandes der Hochschullehrer für Betriebswirtschaft,<br />
2006, Dresden.<br />
• T. Eymann, D. Neumann, M. Reinicke, B. Schnizler, W. Streitberger, D. Veit, On the<br />
Design of a Two-Tiered Grid Market Structure, Multi-Konferenz Wirtschaftsinformatik,<br />
2006, Passau.<br />
9. M. Kunzelmann, D. Neumann and C. Weinhardt, Zwischen Limit <strong>und</strong> Market Order,<br />
10th Symposium on Finance, Banking, and Insurance, 2005, Karlsruhe.<br />
10. D. Rolli, M. Conrad, D. Neumann, C. Sorge, An Asynchronous and Secure Ascending<br />
Peer-to-Peer Auction, ACM SIGCOMM'05 Workshop on the Economics of Peer-to-Peer<br />
Systems, 2005, Philadelphia, PA.<br />
• D. Neumann, B. Schnizler, Transforming Auction Mechanisms into Protocols, Group<br />
Decision and Negotiation, 2005, Vienna, Austria.<br />
11. B. Schnizler, D. Neumann, D. Veit, C. Weinhardt, A Multiattribute Combinatorial Exchange<br />
for Trading Grid Resources Proceedings of the 12th Research Symposium on<br />
Emerging Electronic Markets (RSEEM), 2005, Amsterdam, Netherlands.<br />
12. D. Neumann, J. Maekioe, C. Weinhardt, CAME – A Toolset for Configuring Electronic<br />
Markets, European Conference on Information Systems (ECIS), 2005, Regensburg.<br />
• B. Schnizler, D. Neumann, C.Weinhardt, Resource Allocation in Computational Grids –<br />
A Market Engineering Approach, Proceedings of the Workshop on eBusiness (WeB<br />
2004) – ICIS Pre-conference, Washington.<br />
330
• D. Rolli, D. Neumann, C. Weinhardt, A Minimal Market Model in Ephemeral Markets,<br />
Proceedings of 1st International Workshop on Theory Building and Formal Methods in<br />
Electronic/Mobile Commerce (TheFormEMC), Toledo, 2004, Spain.<br />
• D. Neumann, C. Holtmann, Embodiment Design in Market Engineering, Proceedings of<br />
the Research Symposium on Emerging Electronic Markets (RSEEM 2004), Dublin, Irland,<br />
S. 85-96.<br />
• D. Neumann, Market Engineering - A Structured Design Process for Electronic Markets<br />
Dissertation, Fakultät für Wirtschaftswissenschaften, 2004, Universität Karlsruhe.<br />
• C. Holtmann, D. Neumann, Market and Firm - Two Sides of a Coin, Proceedings of the<br />
10th Research Symposium on Emerging Electronic Markets (RSEEM 2003), Bremen.<br />
• C. Holtmann, D. Neumann, Konzeption eines Bezugsrahmens für die interdisziplinäre<br />
Analyse <strong>und</strong> Gestaltung elektronischer Märkte, 65. wissenschaftlichen Jahrestagung des<br />
Verbandes der Hochschullehrer für Betriebswirtschaft, 2003, Zürich.<br />
• D. Neumann, C. Holtmann, T. Honekamp, Market Integration and Metamediation:<br />
Perspectives for Neutral B2B E-Commerce Hubs in: Ch. Weinhardt, C. Holtmann<br />
(Hrsg.) E-Commerce - Netze, Märkte, Technologien, 2002, Physica, Heidelberg, S. 67-<br />
86.<br />
• D. Neumann, C. Holtmann, H. Gimpel, Ch. Weinhardt, Agent-based bidding in Electronic<br />
Markets – A Prototypical Approach J. Monteiro, P. M. C. Swatman <strong>und</strong> L. V.<br />
Tavares (Hrsg.) Towards the Knowledge Society: e-Commerce, e-Business and e-<br />
Government, 2002, Kluwer Academic Publishers, S.553-567.<br />
• D. Neumann, C. Holtmann, H. Weltzien, C. Lattemann, Ch. Weinhardt, Towards a Generic<br />
E-Market Design J. Monteiro, P. M. C. Swatman <strong>und</strong> L. V. Tavares (Hrsg.) Towards<br />
the Knowledge Society: e-Commerce, e-Business and e-Government, 2002,<br />
Kluwer Academic Publishers, S. 289-305.<br />
• D. Neumann, Ch. Weinhardt, Domain-independent eNegotiation Design: Prospects,<br />
Methods, and Challenges In Proceedings of the 3rd DEXA Workshop e-Negotiations,<br />
2002, Aix-en-Provence, France, S.680-684.<br />
Sonstige Beiträge<br />
1. D. Neumann, Engineering Grid Markets, in H. Gimpel, N.R. Jennings, G. Kersten, A.<br />
Ockenfels, C. Weinhardt (Hrsg), 2008 Negotiation and Market Engineering, Lecture<br />
Notes in Business Information Processing (LNBIP), Springer.<br />
2. G. Kersten, R. Kowalczyk, H. Lai, D. Neumann, M. Chhetri, Shaman: Software and<br />
Human Agents in Multiattribute Auctions and Negotiations, in H. Gimpel, N.R.<br />
Jennings, G. Kersten, A. Ockenfels, C. Weinhardt (Hrsg), 2008 Negotiation and Market<br />
Engineering, Lecture Notes in Business Information Processing (LNBIP), Springer.<br />
3. G. Kersten, E. Chen, D. Neumann, R. Vahidov, C. Weinhardt, On Comparison of<br />
Mechanisms of Economic and Social Exchanges: The Times Model, in H. Gimpel, N.R.<br />
Jennings, G. Kersten, A. Ockenfels, C. Weinhardt (Hrsg), 2008 Negotiation and Market<br />
Engineering, Lecture Notes in Business Information Processing (LNBIP), Springer.<br />
4. C. Block, D. Neumann, A Decision Support System for choosing Market Mechanisms<br />
in e-Procurement, in H. Gimpel, N.R. Jennings, G. Kersten, A. Ockenfels, C. Weinhardt<br />
331
(Hrsg), 2008 Negotiation and Market Engineering, Lecture Notes in Business Information<br />
Processing (LNBIP), Springer.<br />
5. D. Neumann, D. Veit, C. Weinhardt, Grid Economics: Market Mechanisms for Grid<br />
Markets. Grid Computing: Konzepte, Technologien, Anwendungen. T. Barth and A.<br />
Schüll ‘(‘Hrsg), 2006, Viehweg Verlag: 64-83.<br />
6. Weinhardt, C., D. Neumann, M. Kunzelmann, Design <strong>und</strong> Evaluierung von Relative<br />
Orders zur Reduktion impliziter Transaktionskosten, in W. Bessler (Hrsg), 2006 Börsen,<br />
Banken <strong>und</strong> Kapitalmärkte, 205-226.<br />
7. D. Neumann, I. Weber, Market Engineering and the PPC-Auction, CORS/INFORMS<br />
Joint International Meeting, May 19-22 2004, The Banff Center, Banff, Alberta, Kanada.<br />
8. C. Holtmann, D. Neumann, Ch. Weinhardt, e-Commerce im Wertpapierhandel – Der<br />
private Investor im Fokus, Branchengutachten im Auftrag des Deutschen B<strong>und</strong>estags,<br />
2001, im Rahmen des TAB-Projekts "e-Commerce", 2. Phase, vorgelegt dem Büro für<br />
Technikfolgenabschätzung (TAB) beim Deutschen B<strong>und</strong>estag, Berlin.<br />
332