Forecast: Mostly Cloudy - Ziti - Ruprecht-Karls-UniversitÃ¤t Heidelberg

(Weather) Forecast: Mostly Cloudy 

Dr. Holger Fröning 

Juniorprofessor für Technische Informatik 

Ruprecht-Karls-Universität Heidelberg

Popular Web Services 

 

Holger Fröning, Informatiktag der Universität Heidelberg, 22.06.2012 2

Cloud Computing – What is it 

• John McCarthy (MIT/Stanford), MIT Centennial, 1961: 

„If computers of the kind I have advocated become the computers of 

the future, then computing may someday be organized as a public 

utility just as the telephone system is a public utility... The computer 

utility could become the basis of a new and important industry.” 

• Larry Ellison (Oracle), Wall Street Journal 2008: 

„The interesting thing about Cloud Computing is that we‘ve redefined 

Cloud Computing to include everything that we already do...“ 

• Andy Isherwood (HP), ZDnet News, 2008: 

„... I have not heard two people say the same thing about it [the 

Cloud].“ 

• Richard Stallman (Free Software Advocat), 2008: 

„It‘s a marketing hype campaign...“ 

[M. Armbrust et al. 2009. Above the Clouds: A Berkeley View of Cloud Computing, Technical Report No. UCB/EECS-2009-28, University of California, Berkeley] 


Cloud Computing 

• Pervasive internet access 

enables utility computing 

• Computing is provided as a 

service 

• Elasticity: no need to 

overprovision resources 

Virtualization 

• Cloud computing: 

Software 

applications & hardware & as a 

system software delivered Service 

as a service 

• Cloud: datacenter hardware and 

software 

Connectivity 

Cloud 

Computing 

Grid 

Computing 

Distributed 

Systems 

Utility 

Computing 


Analogies 

Electricity 

Supply 

• Past: companies 

produced their 

own electricity 

(steam, water) 

• Today: electricity 

as a service, 

enabled by the 

electrical grid 

Chip 

Manufacturing 

• Past: most big 

companies had 

their own fabs 

• Today: fab-less 

manufacturing, 

only few fabs left 

(Intel, Samsung, 

TSMC) 

Cloud 

Computing 

• Past/today: run 

your own computing 

facilities 

• Today/tomorrow: 

rely on cloud 

computing 


Different Levels of Service 

• Software as a Service 

• Targets: end users 

• Examples: Web search, E-Business, 

Web Mail 

• Platform as a Service 

• Targets: application developers 

• Examples: Google AppEngine, 

Microsoft Azure 

• Infrastructure as a Service 

• Targets: network architects 

• Examples: Amazon Elastic 

Computing Cloud, GoGrid 


Typical Questions 

• Who needs it Can it be really be successful 

• Definitely, look at the analogies 

• Disruptive technology 

• Is it faster than a high-performance cluster 

• Edward Walker (TACC): NO (absolute performance) 

• Ian Foster (ANL): YES (response time) 

• How many datacenters are there 

• 500k in total, but only a fraction public available 

• Major players: Google, Microsoft, Facebook, Amazon, Apple, ... 

• Is it interesting for research 

• David Patterson, 2008: „[...] the most interesting computers of the future are at 

the extremes in scale: the {datacenter,cell phone} is the computer“ 

• Power wall 


Hardware Perspectives of Cloud Computing 

• Illusion of infinite on-demand computing resources 

• No need for users to plan ahead for provisioning 

• No up-front commitment by cloud users, use on a shortterm 

basis as needed 

• Allowing companies to start small and increase resources only if 

required 

• Keys behind these perspectives: 

Extreme scalability & cost-effectiveness 


Rest of this Talk 

• Backbone of Cloud Computing: Datacenters 

• Short Analysis 

• Research in our labs 

• Conclusion 


Backbone of Cloud Computing: Datacenters 


Google Datacenters 

First Production 

System (1999) 

Google Datacenter 

today 

[Flickr.com] 

[Wikipedia.org] 


Microsoft Datacenters 

[blogs.msdn.com] 


Inside a Datacenter 

[CNet News, 2009] 

[A. Fox, Science 331, 406 (2011)] 


Facebook Datacenter 

• Facebook‘s 

OpenCompute Initiative 

[building43.com] 


Apple Datacenter 

• One of three Apple DCs 

• Used for iCloud and Siri 

[EarthTechling.com, 2012] 


[D. Dyer, Current trends/challenges in datacenter 

thermal management, ITHERM, San Diego, CA, 2006.] 

Datacenter Architecture 

• 10,000 – 50,000+ servers ( 320k - 1.6M+ cores) 

• Powerful server CPUs 

• 10-30 MW 

• 60,000 sqm 


Energy Consumption 

• Cloud computing allows for (almost) any location 

+ Cooling advantages, lower electricity costs 

- Costs for data movement, response latency increases 

• Greenpeace: “Dirty data triangle” 

• Datacenter Hub: Apple, Google and Facebook in North Carolina 

• „Facebook will receive about $17 million in local subsidies and tax 

breaks over 10 years” [NY Times, Nov.11, 2010] 

• Main reasons for NC are cheap energy and tax savings 

• Typical electricity mix: coal and nuclear 

• Increasing interest to use renewable sources like solar, wind and 

water 


Energy Consumption 

• Power Usage Efficiency (PUE) 

• PUE = total power for facility / IT equipment power 

• Losses due to cooling and 

power suppy 

• Facebook reported a PUE 

of 1.07 

• Nothing about total 

energy consumption 

• Fact: about 1% of 

world-wide energy 

is consumed by 

datacenters 

[J. G. Koomey. Worldwide electricity used in data centers. 

Environmental Research Letters, 3(3):034008, 2008] 

[Hennessy/Patterson, Computer Architecture: A quantitative 

approach, Morgan Kaufmann, 5. Ed., 2011] 


Industrialization of IT 

• Huge efforts to 

minimize PUE 

• Cooling and power supply 

• Warehouse-Scale 

Computers (WSC) 

• Composed of commodity 

parts 

• Is this the solution 

[Hennessy/Patterson, Computer Architecture: A 

quantitative approach, Morgan Kaufmann, 5. Ed., 2011] 


Short Analysis 


New Degree of Parallelism 

Instruction-level parallelism 

• One instruction stream, many dependencies 

• Limited amount (2-6) 

ILP 

• Pipelined architectures 

• Superscalar architectures 

Thread-level parallelism 

• Multiple instruction streams, less dependencies 

• Requires functional decomposition, still limited 

TLP 

• Multi-Core 

• Multi-Threading 

Data-level parallelism 

• Even less dependencies 

• Depends almost completely on problem size 

DLP 

• GPUs and Clusters 

• MMX/SSE/AVX 

New: Request-level parallelism 

• Vast amount of users, no dependencies! 

• Allows for relaxed consistency models 

RLP 

• Datacenters 


System Architecture 

• Oversubscription of the network 

• Local to Rack to Array 

• Huge locality effects 




Access Cost Disparities 

Local Rack Array 

DRAM Latency 0.1 usec 100 usec 500 usec 

Disk Latency 10,000 usec 10,000 usec 10,000 usec 

DRAM Bandwidth 20,000 MB/s 100 MB/s 10 MB/s 

Disk Bandwidth 200 MB/s 100 MB/s 10 MB/s 

[Barroso and Hölzle 2009] 

DRAM Capacity 16 GB 1,040 GB 31,200 GB 

Disk Capacity 2,000 GB 160,000 GB 4,800,000 GB 

• Performance degradation, but huge amount of aggregated 

resources 

• Data movement is expensive 

• MapReduce 

• Static Resource Partitioning – no shared use! 


CPU Utilization and (non-)Energy-proportional Computing 

[Barroso and Hölzle, The Case for 

Energy-Proportional Computing, IEEE 

Computer, 2007] 




Datacenters compared to HPC 

• Datacenter 

• Highly dynamic use 

• No structured execution of online tasks 

• Energy consumption (costs for energy exceed acquisition costs) 

• Low-power CPUs likely not an option 

• High Performance Computing (HPC) 

• E.g.: clusters, massively parallel processors 

• Extremely optimized for selected use models 

• Similar energy consumption problems 

• Low-power CPUs likely not an option 

• Lessons learned in HPC 

• Interconnection Networks 

• Accelerators – absolute Performance and Performance/Watt 

• Commodity parts + custom parts 


Research in our labs 


One Example: Memory as a Scarce Resource 

• Performance disparity for 

different technologies 

• Memory hierarchy only helps if 

enough locality is present 

• Effective memory capacity 

• Memory capacity per core is 

not keeping the pace of core 

count increase 

500x 

Memory capacity per computing core 

for 4P servers at highest memory speed 


Current Computer Architectures 

Shared Memory Computers 

(shared-everything) 

Message Passing Systems 

(shared-nothing) 

• Single address space 

• Up to 64 Cores and 2TB RAM 

• Shared Memory programming model 

• Global coherency among all processors 

• Limited scalability! 

• Resource aggregation 

• Multiple address spaces 

• Over-provisioning of single nodes 

• Message Passing programming model 

• No coherency among nodes 

• Unlimited scalabilty 

• Resource partitioning 

Main Memory 

Main Memory 

Main Memory 

Main Memory 

Main Memory 

Main Memory 

Address 

Space 

Address 

Space 1 

Address 

Space 2 

Address 

Space 3 

Processors/Caches 







A New Approach: MEMSCALE 

• Dynamic, selective aggregation of resources 

• Shared use of scalable resources (memory) 

• Exclusive use of resources with limited scalability (cores/caches) 

• Memory regions can expand to other nodes 

• Overhead of global coherency is avoided 

Spanning up global address spaces 

Decoupled resource aggregation, no resource partitioning 

Main Memory 

Main Memory 

Main Memory 

Main Memory 

Main Memory 

Memory 

Region 1 

Memory 

Region 2 

Memory 

Region 3 

Memory 

Region 4 

Memory 

Region 5 







Proof of Concept – In-Memory-Database 

MySQL Cluster 

• Cluster-level execution 

• MEMSCALE vs. MySQL cluster 

• MySQL cluster 

• 16 nodes, Gigabit Ethernet 

• 450 queries per second 

• Saturation starts at 20-30 flows 

• MEMSCALE 

• 16 nodes, EXTOLL R1 w/t SME 

• 128GB memory pool 

• 35k queries per second, 77x 

• Linear scalability up to 4/5 flows per 

node (64-80 total), then saturation 

• Limited by 

• Number of outstanding loads 

• Access latency 

MEMSCALE 

2012/04/11 - CERCS Systems Seminar 30

Impact of Dynamic Resource Aggregation 

• Dynamic Aggregation and 

Disaggregation of 

Resources 

• Shared and exclusive use 

models 

• Overcome capacity 

limitations 

• Scarce resources like memory 

• No need for overprovisioning 

• Provision for average case, not 

for worst case 

• Maximize utilization 

[P. Ranganathan and N. Jouppi, Enterprise IT Trends and 

Implications for Architecture Research, HPCA2005] 


Future Project: MEERkAT 

•MEERkAT – Improving 

datacenter utilization 

and energy efficiency 

• System-level virtualization 

• Dynamic resource 

aggregation of 

hetereogeneous resources 

• Various migration levels 

•Analogy – meerkats are 

highly social 

• Compatible with big cats 

Red Fox and Ocelot Project at Georgia Tech 


General research methodology 

• Optimize the common case 

• Effective specialized hardware 

components, completed by 

software stacks 

• Parallel Computing and 

Computer Architecture 

• Parallel programming, communication 

libraries, synchronization methods 

• High Performance Computing, 

Accelerated Computing 


Prototype cluster, Valencia 

• Collaborations 

• Technical University of Valencia, Spain – Prof. 

Jose Duato 

• Georgia Tech, US – Prof. Sudhakar Yalamanchili 

• Simula Labs, Norway – Prof. Olav Lysne 

• EXTOLL & Computer Architecture Group – Prof. 

Ulrich Brüning 


Conclusion 

• Cloud Computing is the future of IT, based on datacenters 

• Limited by power consumption with economic, ecologic and technical 

implications 

• Utilization far from optimum 

• Sole use of commodity parts is 

hitting a wall - Learn from HPC! 


• Accelerated Computing 

• Dynamic resource aggregation 

• Only aggregated selected resources 

• Avoid over-provisioning 

• Overcome the static partitioning 


Danke für die Aufmerksamkeit! 

froening@uni-hd.de 

http://ce.uni-hd.de

Backup Slides 


Motivation – In-Memory Computing 

• DRAM for storage In-Memory Databases 

• Jim Gray: „Memory is the new disk and disk is the new tape” 

• HDD bad for random access, but pretty good for linear access 

• Natural for logging and journaling an in-memory database 

• Google, Yahoo!: entire indices are stored in DRAM 

• Bigtable, Memcached 

• Litte or no locality for many new applications 

• “[…] new Web applications such as Facebook appear to have little or 

no locality, due to complex linkages between data (e.g., friendships in 

Facebook). As of August 2009 about 25% of all the online data for 

Facebook is kept in main memory on memcached servers at any given 

point in time, providing a hit rate of 96.5%”. [Ousterhout2009] 

• Typical computer: 2-8MB cache, 2-8GB DRAM. At most 0.1% of DRAM 

capacity can be held in caches 


Setting up global address spaces 

• Distributed shared memory 

• Local address space 

is split up: 

1. Private partition 

2. Shared partition 

3. Mapping to global 

address space 

• Virtually unlimited 

• Only limited by physical 

address sizes 

• Currently: 2^48 bytes or 256 TB 


Remote memory access 

• Software-transparent access 

to remote memory 

• Loads/stores to local mapping 

of the global address space 

• Parts of the global address will 

identify target node 

• Forward request over the 

network 

• Request hits shared local 

partition on target node 

• If appropriate, send response 

back 

origin 

address space 

0 

private 

local 

shared 

local 

global 

• Direct, low-latency path to remote memory 

• Shared Memory Engine (SME) 

2 40 

2 64 -1 

global 

address space 

node 0 

node 1 

node n-1 

target 

address space 

0 

private 

local 

shared 

local 

2 40 

global 

oAddr gAddr tAddr 

2 64 -1 


Remote memory access in detail 

Node #0 (Source) 

Issuing loads/stores on 

remote memory 

Memory 

CPU 

(MCT) 

Memory 

CPU 

(MCT) 

Remote load latency: 

1.89usec (R1, Virtex-4) 

1.44usec (R2, Virtex-6) 

DRAM 

Controller 

Memory 

Controller 

Core 

Core 

Node #1 (Target) 

Serving as memory host 

Memory 

CPU 

(MCT) 

Memory 

CPU 

(MCT) 

HT 

HT 

Coherency 

Domain 

SME 

EXTOLL Packet 

SME 

Coherency 

Domain 

Source-local address 

Target node determination 

Address calculation 

Global address 

Loss-less and in-order 

packet forwarding 

Target-local address 

Source tag management 

Address calculation 


Excursion: EXTOLL 

• High performance interconnection network 

• Designed from scratch for HPC demands 

• Optimized for low latency and high message rate 

• Virtex-4 (156MHz, HT400, 6.24Gbps) 

HOST 

NETWORK 

NETWORK 

• 40% faster for INTERFACE WRF (compared INTERFACE to IBDDR) 

ATU 

Linkport 

Host Interface 

(HT3 or PCIe) 

On Chip 

Network 

VELO 

RMA 

EXTOLL 

Network 

Switch 

Networkport 

Networkport 

Networkport 

Linkport 

Linkport 

Linkport 

Linkport 

Control 

& Status 

Linkport 


Behind a Google Web Search 

• Data Acquisition – Web Crawling 

• Offline processing 

• MapReduce 

• Developed and optimized for distributed, loosely-coupled systems 

like datacenters 

• Online processing (Googling) 

• Serving user request 

• Delivery of previously prepared results 

• Goal: minimal request latency, exploit vast amount of RLP 


Abstract 

Als Cloud Computing bezeichnet man die Bereitstellung von Rechen- und Speicherkapazitäten über 

das Internet als ein Service, mit hochgradig unterschiedlichen Nutzerprofilen von privaten Nutzern 

über kleine mittelständige bis hin zu großen internationalen Unternehmen. Aktueller Trend ist eine 

starke Zunahme der Nutzung des Cloud Computings, und es ist absehbar das dies sich auch in der 

Zukunft fortsetzen wird. Während das Cloud Computing schon relativ bekannt ist, ist der wahre Kern 

dieses Nutzungsmodells eher unbekannt: die sogenannten Datacenters, welche große Mengen an 

Rechenleistung an einem Ort bündeln und als Rückgrat des Internets gelten. Alle modernen 

Internetdienste wie Web Search, Content Delivery und soziale Netzwerke nutzen diese Datacenters 

um Anwendern möglichst kurze Antwortzeiten zu bieten. Da die Nutzung dieser Dienste immens 

zunimmt, steigen auch die Anforderungen an die Datacenters rapide an. Im Rahmen dieses Vortrages 

wird die Architektur und Funktionsweise von Datacenters erläutert, insbesondere wie aktuelle 

Datacenters konstruiert und organisiert sind und wie Dienste auf diesen Datacenters realisiert werden. 

Obwohl ein hoher Bedarf an immer größeren Installationen besteht, ist die Skalierbarkeit aber 

aufgrund von technischen, ökonomischen und ökologischen Faktoren limitiert, insbesondere was die 

immense Leistungsaufnahme dieser Systeme betrifft. So wird unter anderem eine ausschliessliche 

Nutzung von Standardtechnologien über kurz oder lang zu einem prinzipiellen Problem werden. Diese 

einschränkenden Faktoren bieten jedoch der Forschung die Möglichkeit mit neuartigen Methoden und 

Architekturen aktiv zu der Entwicklung der Datacenters beizutragen.

Forecast: Mostly Cloudy - Ziti - Ruprecht-Karls-UniversitÃ¤t Heidelberg

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?