Werner Vogels Keynote
Werner Vogels Keynote
Werner Vogels Keynote
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
7/22/2008<br />
Bandwidth Consumed by<br />
Amazon’s Global Websites<br />
Bandwidth Consumed by<br />
Amazon Web Services<br />
What is Amazon?<br />
• 7 Countries<br />
• More than 76 million active customer accounts<br />
• Approximately 1.3 million active seller accounts<br />
• Hundreds of thousand of registered associates<br />
• Over 370,000 registered web services developers<br />
• Over 17,000 employees World Wide<br />
• More than 30 Fulfillment Centers worldwide<br />
• 28% of units sold by 3rd party sellers<br />
• 3.9M shipments on peak day in 2007<br />
2001 1996 2002 1997 2003 1998 1999 2004 2000 2005 2001 2006 2002 2007<br />
2008<br />
Amazon’s History in 30 Seconds<br />
• Transformations<br />
Retail<br />
Customers<br />
– From single seller to 1M+ sellers<br />
– From technology consumer to technology provider<br />
– From single application to platform<br />
– From web site & database to a massively distributed system<br />
– From web site to web services<br />
– From enterprise scale to web scale<br />
Infrastructure<br />
Web Service<br />
Developers<br />
Amazon<br />
Technology<br />
Platform<br />
Retail<br />
Marketplace<br />
Sellers<br />
Ecommerce<br />
Web Service<br />
Customers<br />
Enterprise<br />
Ecommerce<br />
Customers<br />
1
7/22/2008<br />
App Server &<br />
Database<br />
Service<br />
Orientation<br />
Massively<br />
Scalable<br />
Services<br />
Amazon<br />
Retail<br />
Amazon<br />
E-Commerce<br />
Web Services<br />
Amazon<br />
Web Services<br />
Innovation<br />
Amazon<br />
Retail<br />
Partners<br />
Amazon<br />
Enterprise<br />
Partners<br />
……<br />
Amazon<br />
Retail<br />
Amazon<br />
E-Commerce<br />
Web Services<br />
Amazon<br />
Web Services<br />
Innovation<br />
Amazon<br />
Retail<br />
Partners<br />
Amazon<br />
Enterprise<br />
Partners<br />
……<br />
Foundation<br />
Aggregators<br />
2
7/22/2008<br />
Develop Test Release Install Configure Operate<br />
Develop Test Operate http://www....<br />
Develop Test Operate Develop Test<br />
Undifferentiated<br />
heavy lifting<br />
• Hardware costs<br />
• Software costs<br />
• Maintenance<br />
• Load balancing<br />
• Scaling<br />
• Utilization<br />
• Idle machines<br />
• Bandwidth<br />
management<br />
• Server hosting<br />
• Storage<br />
Management<br />
• High availability<br />
Operate<br />
The 70/30 Switch<br />
3
7/22/2008<br />
Dream or Nightmare?<br />
• Scale to 100% - 1000% increase popularity?<br />
• Provide 4 nines uptime?<br />
• Survive a complete datacenter failure?<br />
• Survive a router flapping?<br />
• While keeping cost low at the same time?<br />
Amazon<br />
Retail<br />
Amazon<br />
E-Commerce<br />
Web Services<br />
Amazon<br />
Web Services<br />
Innovation<br />
Amazon<br />
Retail<br />
Partners<br />
Amazon<br />
Enterprise<br />
Partners<br />
……<br />
Aggregators<br />
Chapter 4: Priorities – Scale Later – It is too hard to get right<br />
Foundation<br />
4
7/22/2008<br />
Amazon<br />
S3<br />
Scalable Infrastructure that<br />
allows applications to meet<br />
infinite demand, cheaply and<br />
reliably<br />
Amazon<br />
FPS<br />
Amazon<br />
SQS<br />
Amazon<br />
SimpleDB<br />
Amazon<br />
EC2<br />
Amazon Infrastructure Services<br />
Scalable<br />
• Increase or<br />
decrease<br />
capacity in<br />
minutes<br />
Cost-<br />
Effective<br />
• Low rate,<br />
pay-as-yougo<br />
Reliable<br />
• Runs on<br />
Amazon’s<br />
proven<br />
infrastructure<br />
Simple<br />
• SOAP- and<br />
REST-based<br />
Web Service<br />
Calls .<br />
Scalability<br />
Availability<br />
Performance<br />
Cost-effectiveness<br />
Lower Cost<br />
Structure<br />
Sellers<br />
Selection<br />
Growth<br />
Lower<br />
Prices<br />
Customer<br />
Experience<br />
Traffic<br />
5
7/22/2008<br />
Incremental Scalability<br />
Incremental Scalability is Key<br />
• Service can be scaled up/down one node at a time.<br />
• Elastic: capable of growing and shrinking on<br />
demand.<br />
• Minimal disruption to customer performance.<br />
• Common, fully automated operation.<br />
• Addresses:<br />
– Different growth paths<br />
– Fault-tolerance<br />
– Heterogeneity<br />
– Operational efficiency<br />
Everything fails, All the Time<br />
Scalability<br />
Availability<br />
Performance<br />
Cost-effectiveness<br />
Failures are Highly Correlated<br />
Systems do not Fail by Stopping<br />
Let go<br />
of<br />
Control!<br />
6
7/22/2008<br />
Scalability<br />
Availability<br />
Performance<br />
Cost-effectiveness<br />
Scalability<br />
Availability<br />
Performance<br />
Cost-effectiveness<br />
Addressing Uncertainty<br />
• Acquire resources on demand<br />
• Release resources when no longer needed<br />
• Pay for what you use<br />
• Leverage other’s core competencies<br />
• Turn fixed cost into variable<br />
Scalability<br />
Availability<br />
Performance<br />
Cost-effectiveness<br />
Aggregators<br />
Amazon<br />
Retail<br />
Amazon<br />
E-Commerce<br />
Web Services<br />
Amazon<br />
Web Services<br />
Innovation<br />
Amazon<br />
Retail<br />
Partners<br />
Amazon<br />
Enterprise<br />
Partners<br />
……<br />
Are dominated by state<br />
management<br />
Foundation<br />
7
7/22/2008<br />
Categorization of Data Access Patterns<br />
• Primary Key Access<br />
– Eventual Consistent<br />
• High Read Volume<br />
• Always Writable<br />
– Strong Consistent<br />
• Query-based Access<br />
– Eventual Consistent<br />
• Relation-less<br />
• Relational<br />
– Strong Consistent<br />
Amazon Infrastructure Services for Storage<br />
• Primary Key Access<br />
– Amazon S3<br />
• Query-based Access<br />
– Amazon SimpleDB<br />
• Dedicated Solutions<br />
– Amazon EC2 with persistent storage<br />
Billions of Objects in Amazon S3<br />
20<br />
18<br />
16<br />
14<br />
12<br />
10<br />
8<br />
6<br />
4<br />
2<br />
0<br />
July 2006 April 2007 Oct 2007 Jan 2008 March 2008<br />
8
7/22/2008<br />
“Amazon on Monday announced persistent storage for its<br />
EC2 service and what’s notable is how quickly the e-tailer is<br />
running ahead of the competition. In fact, Amazon’s real<br />
business down the line will be its cloud services. Amazon<br />
will be like a book store that sells cocaine out the back<br />
door. Books will be just a front to sell storage and cloud<br />
computing.”<br />
“In a recent test, the Defense Information Systems Agency<br />
compared the cost of developing a simple application called<br />
the Tech Early Bird on $30,000 worth of in-house servers<br />
and software with the costs of developing the same<br />
application using the Amazon Elastic Compute Cloud from<br />
Amazon.com’s Web Services. Amazon charged 10 cents an<br />
hour for the service, and DISA paid a total of $5 to develop<br />
an application that matched the performance of the inhouse<br />
application.”<br />
9
Change language