SlideShare una empresa de Scribd logo
1 de 46
Descargar para leer sin conexión
Rethinking the Cloud
 - A View of Virtualization, Storage, Network, and Platform


Woohyun Kim
Cloud Platform Team
S-Core


2011-05-18
Cloud Success Stories
What is the Cloud?
 • A computing environment to elastically provide virtualized resources as a
   service over the Internet in a pay-as-you-go manner
Amazon’s Challenge and Paradigm Shift
Success Cases in Amazon
  SmugMug(http://www.smugmug.com/)
  • an online photo storage application that stores more than half a petabyte of data on S3
  • estimates cost savings on service and storage to be close to $1 million

  New York Times(http://www.nytimes.com)
  • use EC2 to process terabytes of archival data using hundreds of EC2 instances within 36
    hours

  37Signals(http://37signals.com/)
  • maker of popular online project-management software Basecamp, uses S3 for storage
    needs.
  Animoto(http://animoto.com/)
  • an online presentation video generator that needs gobs of computing power for video
    processing
  • recently successfully withstood a surge in Web traffic that would kill most companies’
    systems by scaling up their processing power quickly using EC2 with RightScale
       • Animoto ramped from 25,000 users to 250,000 users in three days, signing up
          20,000 new users per hour at peak
       • Using RightScale, EC2 instances automatically scaled out 40 to 4000 at that time
       • For more detail, refer to http://blog.rightscale.com/2008/04/23/animoto-facebook-
          scale-up/
Start-up Company: Powerset
  Powerset had a great idea, “Natural Language Search”
  It should index millions of pages of data and content
  They knew that this would require a massively large datacenter and
   extensive computing power
     CPUs, terminal switches, cable, racks, datacenters, hosting, power,
      maintenance, staffs
  But they needed to keep infrastructure costs at a minimum

   “By using Amazon EC2, Powerset is able to match the
    infrastructure of large scale search companies on a
        startup budget.” - Barney Pell, Founder and CEO of Powerset

   ““Amazon EC2 is a complete game-changer. EC2 and
 Amazon Web Services make it easy for start-ups to build
 a complete infrastructure without having to spend much
                  on capital .”- Paul Hammann
                                                                            6
Start-up Company: Powerset
  Powerset had a great idea, “Natural Language Search”
  It should index millions of pages of data and content
  They knew that this would require a massively large datacenter and
   extensive computing power
     CPUs, terminal switches, cable, racks, datacenters, hosting, power,
      maintenance, staffs
  But they needed to keep infrastructure costs at a minimum

   “By using Amazon EC2, Powerset is able to match the
                                             $100 millions
    infrastructure of large scale search companies on a
        startup budget.” - Barney Pell, Founder and CEO of Powerset

   ““Amazon EC2 is a complete game-changer. EC2 and
 Amazon Web Services make it easy for start-ups to build
 a complete infrastructure without having to spend much
                  on capital .”- Paul Hammann
                                                                            7
Temporary & Data-intensive : The New York Times
  The New York Times is a 150-year old company, and serves the largest
   newspaper Website, NYTimes.com
      1 billion page views per month
      20+ million monthly unique visitors
  They tried to convert TIFF images to PDFs
      TIFF images(405,000),
      Articles(3.3 million) in SGML                  PNG images(810,000)
      XML files(405,000) mapping articles to TIFFs   JavaScript files(405,000)

 “I had was this: upload 4TB of source data into S3, write some
  code that would run on numerous EC2 instances to read the
   source data, create PDFs, and store the results back into S3.
     S3 would then be used to serve the PDFs to the general
                        public.” - Derek Gottfrid

 “I got access to a few more machines and churned through all
     11 million articles in just under 24 hours using 100 EC2
 instances, and generated another 1.5TB of data to store in S3.
                  It just costs $3000.” - Derek Gottfrid

                                                                            8
Temporary & Data-intensive : The New York Times
  The New York Times is a 150-year old company, and serves the largest
   newspaper Website, NYTimes.com
      1 billion page views per month
      20+ million monthly unique visitors
  They tried to convert TIFF images to PDFs
      TIFF images(405,000),
      Articles(3.3 million) in SGML                  PNG images(810,000)
      XML files(405,000) mapping articles to TIFFs   JavaScript files(405,000)

 “I had was this: upload 4TB of source data into S3, write some
  code that would run on numerous EC2 instances to read the
   source data, create PDFs, and store the results back into S3.
     S3 would then be used to serve the PDFs to the general
                             public.”

 “I got access to a few more machines and churned through all
     11 million articles in just under 24 hours using 100 EC2
 instances, and generated another 1.5TB of data to store in S3.
                         It just costs $3000.”

                                                                            9
64,865,381,400
Cloud Skepticism
The Cloud is Falling
  Amazon’s cloud outages receive a lot of exposure …

   April 21 ~ 22, 2011   A networking glitch made its storage volumes automatically create
                         back-ups of themselves, filling up storage capacity and causing
                         connectivity issues, lasts two days
                         Amazon’s customers include start-ups like the social networking site
                         Foursquare but also big companies like Pfizer, Netflix and Nasdaq
                         dkagh
                         Affected web sites included Quora.com, Reddit.com, GroupMe.com
                         and Scvngr.com
      July 20, 2008      Failure due to stranded zombies, lasts 5 hours

      Feb 15, 2008       Authentication overload leads to two-hour service outage

      October 2007       Service failure lasts two days

      October 2006       Security breach where users could see other users data

 … and their current SLAs don’t match those(99.99%) of enterprises

      Amazon EC2                99.95%                    Amazon S3               99.9%
Cloud Is NOT A Brand-New Technology
                                                               Utility
                                                             Computing




                                         Google App Engine
                          Amazon EC2        (April 2008)
                         (August 2006)             Microsoft Azure
                                                     (Oct 2008)
                    Amazon S3
                   (March 2006)




 GFS                     BigTable
       MapReduce    Hadoop
Cloud is just Buzz, and Marketing Hype Campaign
 •   Cloud computing is simply a buzzword used to repackage grid
     computing and utility computing, both of which have existed for
     decades
                                               -       Definition of Cloud Computing, whatis.com
 •   What is it? What is it? ... Is it - 'Oh, I am going to access data on a
     server on the Internet.' That is cloud computing?
 •   The interesting thing about cloud computing is that we’ve
     redefined cloud computing to include everything that we already
     do
                                                   -      During Oracle’s Analyst Day, Larry Ellison
 •   .. cloud computing was simply a trap aimed at forcing more people
     to buy into locked, proprietary systems that would cost them more
     and more over time
 •   It's stupidity. It's worse than stupidity: it's a marketing hype
     campaign
                                                              -    GNU founder, Richard Stallman
 •   Server revenue for public cloud computing will grow from $582
     million in 2009 to $718 million in 2014
 •   Server revenue for the much larger private cloud market will grow
     from $7.3 billion to $11.8 billion in the same time period
                     -   Worldwide Enterprise Server Cloud Computing 2010-2014 Forecast, IDC
Cloud Wars and Strategies
The Cloud Wars
                                                          JBoss              Xen
                                                   420M
   KVM         Qumranet
   SPICE                       107M
                                      RedHat               NetScaler       XenSource
   iSilon   GreenPlum                                         300M

                                       OpenShift
                                                                  Citrix
        EMC                   CloudFoundry   OpenStack

            650M
                                                                  Rackspace
               VMware                  VMForce       Oracle
                       450M                                             BtrFS
GemStone
RabbitMQ           SpringSource       Force.com
                                                                                Ceph
 Hyperic                                             Sun             ZFS


     Heroku    212M     SalesForce.com                     Java        NexentaStor
                                                          MySQL
Infra is getting more Programmable




 rPath rBuilder




                  Quest vControl




                                   3Tera Applogic
Cloud Disruptors

 • RightScale & enStratus
Cloud Disruptors (cont’d)

 • Cloud Mgmt. Functionality in enStratus
Cloudburst and Hybrid Cloud

 • CloudSwitch
Who Moved My Cheese?
 Disrupting or Disrupted??!
Virtualization
Virtualization on x86 Architecture
 •   VMM(Virtual Machine Monitor) or Hypervisor
      – Since VMM must perform in the privileged level(0) , OS is moved to non-privileged
        level(1 or 3)

                             app    app             app      app            app     app


                        Operating System          Operating System        Operating System
                               #1                        #2                      #3
                           (Win-XP)                  (Mac-OS)                 (Linux)

                         Virtual Machine           Virtual Machine        Virtual Machine


                                               Virtual Machine Manager



                                   CPU          Memory          NIC          Disk



 •   Problems on x86 Architecture
      – Privileged Instruction
           •   Trap when called from CPU user mode, and Emulate its effect by VMM
      – Sensitive Non-privileged Instruction
           •   Cause physical state of CPU to leak
                  –   smsw    %eax             # reads CR0 into EAX
                  –   mov     %cr0, %edx       # reads CR0 into EDX
                  –   sub     %eax, %edx       # what’s the difference?                      No Trap, No Emulation
                  –   jnz     emulation_flaw   # it ought to be zero!!                       => VMM is finally crashed
CPU Virtualization on x86 Architecture
 • How to handle nonvirtualizable instructions
    – Full virtualization using binary translation
    – Paravirtualization using hypercalls
    – Hardware assisted virtualization using root/non-root mode
       •   VT-x : Virtualization Technology for 32bit CPU
       •   VT-i : Virtualization Technology for 64bit CPU
       •   VT-d : Virtualization Technology for Directed I/O
       •   VT-c : Virtualization Technology for Connectivity
Virtualization on x86 Architecture
Hurdles in Server Virtualization
 •   Storage Allocation & Interfacing
      – On-demand, Pre-allocation
      – NAS, iSCSI, Local Storage
 •   VM Management
      – Snapshot, Fast Cloning, Thin Provisioning, Live Migration, DRS
 •   Virtual Network
      – L2/L3 Network Design, Directed, Bridged, NAT, VLAN, Load-Balance, Firewall
 •   Resource Sharing
      – Resource Pool, High Availability, Scheduling, Workload Mgmt.
 •   Migration
      – P2V, V2V
 •   Hardware-Assisted Support
      – Privileged instruction virtualization
           •   De-privileging or ring compression to handle privileged instructuions
      – Memory virtualization
           •   Memory partitioning and allocation of physical memory
      – Device and I/O virtualization
           •   Routing I/O requests between virtual devices and physical hardware
Storage
File System with Shared Storage



                                    Cluster File Systems
                             • GFS2 – DLM, scaling to 100
                             • GlusterFS – fuse, poor performance
                             • Lustre - dfs
Unified Storage using Virtual Block Pool

  • NexentaStor based on ZFS
Cluster File System using Virtual Block Pool
  • GlusterFS
A Feasible SAN File System

  • IBM TotalStorage SAN File System




                                                        • GlusterFS
                                                        • ZFS
                                       Pooled Storage   • Openfiler
                                                        • iSCSI+GNBD+DRBD
Network
Basic Virtual Network

Tap vs. Tun
   • Tap – simulate an Ethernet device and operate with layer 2
     packets such as Ethernet frames
   • Tun(nel) - simulate a network layer device and operate with layer
     3 packets such as IP packets
   • TAP is used to create a network bridge, while TUN is used with
     routing.
IPTables vs. Bridging
   • IPTables - let the host forward packets between each tap on its
     own subnets
   • Bridging – let all the taps connect into a specific bridge to put
     them on the same subnet
VDE(Virtual Distributed Ethernet) and VDE Switch
OpenStack Nova Network Virtualization

                                                                                        ① Flat Mode
                                Physical Machine
                                                              eth   VM                  • manual config. of bridge
                                                                                        • get fixed public IP from the
                                                                                          pool
         S/W       NIC          bridge         public IP      eth   VM
                                 (br100)

                            manual config

                                                              eth   VM



                                                                                        ② Flat DHCP Mode
                                Physical Machine
                                                              eth   VM   dhcpdiscover   • auto config. of bridge
                                                                                        • get fixed public IP
                         auto
                         eth0
         S/W       NIC          bridge          public IP
                                 (br100)                      eth   VM   dhcpdiscover



                                dnsmasq
                                                              eth   VM   dhcpdiscover




                                                                                        ③ VLAN DHCP Mode (default)
                                Physical Machine
        supports                                 private IP   eth   VM   dhcpdiscover   • auto config. of bridge,
          VLAN                  bridge        from dnsmasq                              • auto config. of VLAN: range
         tagging                 (br100)
                                                                                          of private IPs for project
                                                  VLAN-1
         S/W       NIC          VPN                           eth   VM                    VLAN
                                                                         dhcpdiscover
                                VM          dnsmasq                                     • get fixed private IP: iptables
                                                                                          + NAT (private/public)
     Nova                       bridge                                                  • VLAN: cloudpipe (=openVPN
                                                              eth   VM   dhcpdiscover
     users                       (br101)                                                  VM template  TAP/TUN)
CloudStack Network Virtualization

Virtual Private Network for Each Account
CloudStack Network Virtualization

Detail Virtual Private Network in Node A and Node B
Hurdles in Network Virtualization
 • L2 Network
    –   Problem: Scalability, Performance, Security
    –   Solution: VLAN(for Scalability and Security), RBridge(for Scalability and STP Limitation), L2 over
        L3
 • Multi-tier Networking Design vs. Migration Limitation
    – Limitation of Spanning Tree Protocol
          • Keep Layer 2 networks relatively small and join them together via Layer 3
            segments
          • But VM migration cannot be live across the multi-tier networks
    – Port Consistency
          • Map the settings such as VLAN, ACL, QoS, and security profiles to all the
            network ports
          • But some VMs are not able to meet required service levels
 • L2(Switching) and L3(Routing) Networking Design
    – Scalability and Efficiency on the service provider side
          • Amazon EC2 using L3
                – 500,000 VM on 60,000 PM
    – Legacy Support on the service consumer side
          • Amazon VPC, 3Tera AppLogic
                –   Define virtual network topology
                –   Select IP address range
                –   Create public subnets and private subnets
                –   Configure route table and network gateway
Cloud Platform
Cloud Technologies
Anatomy of Cloud Tehcnologies
OpenStack Nova Architecture


                                                                                 Volume Controller
                                                                                 • provide fast and permanent block-level
                                                                                 storage for the compute servers



                                                                                 Network Controller
                                                                                 • provide virtual networks to enable
                                                                                 compute servers to interact with each
                                                                                 other and with the public network

                                                                                 Scheduler
                                                                                 • select the most suitable compute
                                                                                 controller to host an instance


                                                                                 Compute Controller
                                                                                 • compute server resources

 API Server                                                 Object Store
 • act as the Web services front end for                    • provide storage services
 the cloud controller
                                                            Auth Manager
 Cloud Controller                                           • provide authentication and authorization services
 • represent the global state and interact with all other
 components
CloudStack Architecture

                                Load Balancer


                 Host A                             Host B
                                                     Mgmt.
              Mgmt. Server VM
                                                     Server
                      Management Servers


                                                                       Computing Nodes

                                                Host X                        Host Y                         Host Z
                                                                                                                   dynamically
                                                                                                                   adding            Pod
                                                              Computing Nodes
                                             Guest VM                Guest VM          Guest VM             Guest VM
                                                                                                                            Computing Nodes should be
                                    Host X                           Host Y                        Host Z                   in the same subnet,
     Templates                                                                                                              and have no limit to
                                                    attach                                         live migration
                                                                                                  Guest VM                  number of nodes
                      copy,        Guest VM              Guest VM             Guest VM
         ISO          create,,
       images                                                 Max(6*Volumes)
                      boot,                  Volumes                                             VM Image                 Cluster
                                                              perattach VM
                                                                  Guest                  live migration
                      attach
                                                                                                                      Max(16*Computing Nodes)
                                                                                                                      per Cluster
                                                                 Primary Shared Storage
       VM                             VM                               Max(6*Volumes) VM
                                                           Volumes     per Guest VM                                   Computing Nodes should be
      Image                          Image                                              Image
                      snapshot                                                                                        in the same subnet,
                                                                                                                      and homogeneous
     Secondary                                           Primary Shared Storage
   Shared Storage



                                                                   Zone
Conclusion
Top 10 Cloud Obstacles and Opportunities
• A View of Cloud Computing, ACM, April, 2010
2011 Predictions of IaaS, PaaS, and NoSQL
• IaaS Prediction
     • Hybrid is the way to go: The public-private cloud discussion isn’t relevant anymore
     • Openstack will dominate the open IaaS offering
• PaaS Prediction
     • 2011 is the year of PaaS
          • CloudFoundry – Vmware
          • OpenShift - Redhat
     • A new PaaS category will emerge – Building your own PaaS
          • CEAP(Cloud Enabled Application Platform) is being specifically designed to handle
             multi-tenancy, scalability, and on-demand provisioning, but not higher degree of
             flexibility and control
     • Application servers will change their name to PaaS – But won’t change their stripes
     • VMForce will fail to deliver on its promise => Already open Cloud
• NoSQL (+Big Data) predictions
     • NoSQL will become compatible with SQL
     • More applications will run entirely In-Memory
     • Real-time /Stream-based analytics big will replace majority of the MapReduce batch
        processing
          • i.e., Yahoo S4, Google’s Percolator

                               written by Nati Shalom at Gigaspaces
                               http://natishalom.typepad.com/nati_shaloms_blog/2010/12/2011-cloud-paas-nosql-predictions.html
Rethinking the cloud_-_limitations_and_oppotunities_-_2011_nexcom
Thank you.

Más contenido relacionado

La actualidad más candente

Getting Started with Big Data and HPC in the Cloud - August 2015
Getting Started with Big Data and HPC in the Cloud - August 2015Getting Started with Big Data and HPC in the Cloud - August 2015
Getting Started with Big Data and HPC in the Cloud - August 2015Amazon Web Services
 
AWS Partner Presentation - PetaByte Scale Computing on Amazon EC2 with BigDat...
AWS Partner Presentation - PetaByte Scale Computing on Amazon EC2 with BigDat...AWS Partner Presentation - PetaByte Scale Computing on Amazon EC2 with BigDat...
AWS Partner Presentation - PetaByte Scale Computing on Amazon EC2 with BigDat...Amazon Web Services
 
Intro to High Performance Computing in the AWS Cloud
Intro to High Performance Computing in the AWS CloudIntro to High Performance Computing in the AWS Cloud
Intro to High Performance Computing in the AWS CloudAmazon Web Services
 
High Performance Computing on AWS
High Performance Computing on AWSHigh Performance Computing on AWS
High Performance Computing on AWSAmazon Web Services
 
High Performance Computing on AWS: Accelerating Innovation with virtually unl...
High Performance Computing on AWS: Accelerating Innovation with virtually unl...High Performance Computing on AWS: Accelerating Innovation with virtually unl...
High Performance Computing on AWS: Accelerating Innovation with virtually unl...Amazon Web Services
 
AWS Webcast - An Introduction to High Performance Computing on AWS
AWS Webcast - An Introduction to High Performance Computing on AWSAWS Webcast - An Introduction to High Performance Computing on AWS
AWS Webcast - An Introduction to High Performance Computing on AWSAmazon Web Services
 
Unlocking the Value of your Data Featuring AWS Enterprise Use Cases
Unlocking the Value of your Data Featuring AWS Enterprise Use CasesUnlocking the Value of your Data Featuring AWS Enterprise Use Cases
Unlocking the Value of your Data Featuring AWS Enterprise Use CasesAmazon Web Services
 
AWS re:Invent 2016: Deep Learning at Cloud Scale: Improving Video Discoverabi...
AWS re:Invent 2016: Deep Learning at Cloud Scale: Improving Video Discoverabi...AWS re:Invent 2016: Deep Learning at Cloud Scale: Improving Video Discoverabi...
AWS re:Invent 2016: Deep Learning at Cloud Scale: Improving Video Discoverabi...Amazon Web Services
 
(CMP202) Engineering Simulation and Analysis in the Cloud
(CMP202) Engineering Simulation and Analysis in the Cloud(CMP202) Engineering Simulation and Analysis in the Cloud
(CMP202) Engineering Simulation and Analysis in the CloudAmazon Web Services
 
Workload-Aware: Auto-Scaling A new paradigm for Big Data Workloads
Workload-Aware: Auto-Scaling A new paradigm for Big Data WorkloadsWorkload-Aware: Auto-Scaling A new paradigm for Big Data Workloads
Workload-Aware: Auto-Scaling A new paradigm for Big Data WorkloadsVasu S
 
Masterworks talk on Big Data and the implications of petascale science
Masterworks talk on Big Data and the implications of petascale scienceMasterworks talk on Big Data and the implications of petascale science
Masterworks talk on Big Data and the implications of petascale scienceDeepak Singh
 
洞悉未來運算:量子與5G、混合雲架構與EC2新應用
洞悉未來運算:量子與5G、混合雲架構與EC2新應用洞悉未來運算:量子與5G、混合雲架構與EC2新應用
洞悉未來運算:量子與5G、混合雲架構與EC2新應用Amazon Web Services
 
AWS re:Invent 2016: Zillow Group: Developing Classification and Recommendatio...
AWS re:Invent 2016: Zillow Group: Developing Classification and Recommendatio...AWS re:Invent 2016: Zillow Group: Developing Classification and Recommendatio...
AWS re:Invent 2016: Zillow Group: Developing Classification and Recommendatio...Amazon Web Services
 
Architectures for HPC and HTC Workloads on AWS | AWS Public Sector Summit 2017
Architectures for HPC and HTC Workloads on AWS | AWS Public Sector Summit 2017Architectures for HPC and HTC Workloads on AWS | AWS Public Sector Summit 2017
Architectures for HPC and HTC Workloads on AWS | AWS Public Sector Summit 2017Amazon Web Services
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon RedshiftAmazon Web Services
 
大數據運算媒體業案例分享 (Big Data Compute Case Sharing for Media Industry)
大數據運算媒體業案例分享 (Big Data Compute Case Sharing for Media Industry)大數據運算媒體業案例分享 (Big Data Compute Case Sharing for Media Industry)
大數據運算媒體業案例分享 (Big Data Compute Case Sharing for Media Industry)Amazon Web Services
 

La actualidad más candente (20)

Getting Started with Big Data and HPC in the Cloud - August 2015
Getting Started with Big Data and HPC in the Cloud - August 2015Getting Started with Big Data and HPC in the Cloud - August 2015
Getting Started with Big Data and HPC in the Cloud - August 2015
 
HPC in the Cloud
HPC in the CloudHPC in the Cloud
HPC in the Cloud
 
AWS Partner Presentation - PetaByte Scale Computing on Amazon EC2 with BigDat...
AWS Partner Presentation - PetaByte Scale Computing on Amazon EC2 with BigDat...AWS Partner Presentation - PetaByte Scale Computing on Amazon EC2 with BigDat...
AWS Partner Presentation - PetaByte Scale Computing on Amazon EC2 with BigDat...
 
Intro to High Performance Computing in the AWS Cloud
Intro to High Performance Computing in the AWS CloudIntro to High Performance Computing in the AWS Cloud
Intro to High Performance Computing in the AWS Cloud
 
High Performance Computing on AWS
High Performance Computing on AWSHigh Performance Computing on AWS
High Performance Computing on AWS
 
High Performance Computing on AWS: Accelerating Innovation with virtually unl...
High Performance Computing on AWS: Accelerating Innovation with virtually unl...High Performance Computing on AWS: Accelerating Innovation with virtually unl...
High Performance Computing on AWS: Accelerating Innovation with virtually unl...
 
AWS Webcast - An Introduction to High Performance Computing on AWS
AWS Webcast - An Introduction to High Performance Computing on AWSAWS Webcast - An Introduction to High Performance Computing on AWS
AWS Webcast - An Introduction to High Performance Computing on AWS
 
Unlocking the Value of your Data Featuring AWS Enterprise Use Cases
Unlocking the Value of your Data Featuring AWS Enterprise Use CasesUnlocking the Value of your Data Featuring AWS Enterprise Use Cases
Unlocking the Value of your Data Featuring AWS Enterprise Use Cases
 
AWS re:Invent 2016: Deep Learning at Cloud Scale: Improving Video Discoverabi...
AWS re:Invent 2016: Deep Learning at Cloud Scale: Improving Video Discoverabi...AWS re:Invent 2016: Deep Learning at Cloud Scale: Improving Video Discoverabi...
AWS re:Invent 2016: Deep Learning at Cloud Scale: Improving Video Discoverabi...
 
Keynote AWS Summit 2013 Sao Paulo
Keynote AWS Summit 2013 Sao PauloKeynote AWS Summit 2013 Sao Paulo
Keynote AWS Summit 2013 Sao Paulo
 
Intro to HPC on AWS
Intro to HPC on AWSIntro to HPC on AWS
Intro to HPC on AWS
 
(CMP202) Engineering Simulation and Analysis in the Cloud
(CMP202) Engineering Simulation and Analysis in the Cloud(CMP202) Engineering Simulation and Analysis in the Cloud
(CMP202) Engineering Simulation and Analysis in the Cloud
 
Workload-Aware: Auto-Scaling A new paradigm for Big Data Workloads
Workload-Aware: Auto-Scaling A new paradigm for Big Data WorkloadsWorkload-Aware: Auto-Scaling A new paradigm for Big Data Workloads
Workload-Aware: Auto-Scaling A new paradigm for Big Data Workloads
 
Masterworks talk on Big Data and the implications of petascale science
Masterworks talk on Big Data and the implications of petascale scienceMasterworks talk on Big Data and the implications of petascale science
Masterworks talk on Big Data and the implications of petascale science
 
Amazon EC2 Foundations
Amazon EC2 FoundationsAmazon EC2 Foundations
Amazon EC2 Foundations
 
洞悉未來運算:量子與5G、混合雲架構與EC2新應用
洞悉未來運算:量子與5G、混合雲架構與EC2新應用洞悉未來運算:量子與5G、混合雲架構與EC2新應用
洞悉未來運算:量子與5G、混合雲架構與EC2新應用
 
AWS re:Invent 2016: Zillow Group: Developing Classification and Recommendatio...
AWS re:Invent 2016: Zillow Group: Developing Classification and Recommendatio...AWS re:Invent 2016: Zillow Group: Developing Classification and Recommendatio...
AWS re:Invent 2016: Zillow Group: Developing Classification and Recommendatio...
 
Architectures for HPC and HTC Workloads on AWS | AWS Public Sector Summit 2017
Architectures for HPC and HTC Workloads on AWS | AWS Public Sector Summit 2017Architectures for HPC and HTC Workloads on AWS | AWS Public Sector Summit 2017
Architectures for HPC and HTC Workloads on AWS | AWS Public Sector Summit 2017
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
 
大數據運算媒體業案例分享 (Big Data Compute Case Sharing for Media Industry)
大數據運算媒體業案例分享 (Big Data Compute Case Sharing for Media Industry)大數據運算媒體業案例分享 (Big Data Compute Case Sharing for Media Industry)
大數據運算媒體業案例分享 (Big Data Compute Case Sharing for Media Industry)
 

Destacado

Tu delft realiseert extra capaciteit door inbreiding
Tu delft realiseert extra capaciteit door inbreidingTu delft realiseert extra capaciteit door inbreiding
Tu delft realiseert extra capaciteit door inbreidingRogier den Boer
 
NER & Emerson Infrastructure Optimization Capabilties Storyboard
NER & Emerson   Infrastructure Optimization Capabilties StoryboardNER & Emerson   Infrastructure Optimization Capabilties Storyboard
NER & Emerson Infrastructure Optimization Capabilties StoryboardGreg Stover
 
Case Study: Datotel Extended the Power of Infrastructure Management to the Ph...
Case Study: Datotel Extended the Power of Infrastructure Management to the Ph...Case Study: Datotel Extended the Power of Infrastructure Management to the Ph...
Case Study: Datotel Extended the Power of Infrastructure Management to the Ph...CA Technologies
 
Trellis DCIM Platform
Trellis DCIM PlatformTrellis DCIM Platform
Trellis DCIM PlatformGreg Stover
 
Data Center Infrastructure Management(DCIM)
Data Center Infrastructure Management(DCIM)Data Center Infrastructure Management(DCIM)
Data Center Infrastructure Management(DCIM)MD. IFTEKARUL ALAM
 
Best practices in het datacenter: samenwerking tussen facilitair beheer en IT
Best practices in het datacenter: samenwerking tussen facilitair beheer en ITBest practices in het datacenter: samenwerking tussen facilitair beheer en IT
Best practices in het datacenter: samenwerking tussen facilitair beheer en ITPeter Vermeulen
 

Destacado (9)

Tu delft realiseert extra capaciteit door inbreiding
Tu delft realiseert extra capaciteit door inbreidingTu delft realiseert extra capaciteit door inbreiding
Tu delft realiseert extra capaciteit door inbreiding
 
NER & Emerson Infrastructure Optimization Capabilties Storyboard
NER & Emerson   Infrastructure Optimization Capabilties StoryboardNER & Emerson   Infrastructure Optimization Capabilties Storyboard
NER & Emerson Infrastructure Optimization Capabilties Storyboard
 
Case Study: Datotel Extended the Power of Infrastructure Management to the Ph...
Case Study: Datotel Extended the Power of Infrastructure Management to the Ph...Case Study: Datotel Extended the Power of Infrastructure Management to the Ph...
Case Study: Datotel Extended the Power of Infrastructure Management to the Ph...
 
DCIM: ERP for the Data Center Manager
DCIM: ERP for the Data Center ManagerDCIM: ERP for the Data Center Manager
DCIM: ERP for the Data Center Manager
 
Trellis DCIM Platform
Trellis DCIM PlatformTrellis DCIM Platform
Trellis DCIM Platform
 
DCIM
DCIMDCIM
DCIM
 
Data Center Infrastructure Management(DCIM)
Data Center Infrastructure Management(DCIM)Data Center Infrastructure Management(DCIM)
Data Center Infrastructure Management(DCIM)
 
Executive DCIM
Executive DCIMExecutive DCIM
Executive DCIM
 
Best practices in het datacenter: samenwerking tussen facilitair beheer en IT
Best practices in het datacenter: samenwerking tussen facilitair beheer en ITBest practices in het datacenter: samenwerking tussen facilitair beheer en IT
Best practices in het datacenter: samenwerking tussen facilitair beheer en IT
 

Similar a Rethinking the cloud_-_limitations_and_oppotunities_-_2011_nexcom

Cloud Architectures - Jinesh Varia - GrepTheWeb
Cloud Architectures - Jinesh Varia - GrepTheWebCloud Architectures - Jinesh Varia - GrepTheWeb
Cloud Architectures - Jinesh Varia - GrepTheWebjineshvaria
 
Keynote: Your Future With Cloud Computing - Dr. Werner Vogels - AWS Summit 2...
Keynote: Your Future With Cloud Computing - Dr. Werner Vogels  - AWS Summit 2...Keynote: Your Future With Cloud Computing - Dr. Werner Vogels  - AWS Summit 2...
Keynote: Your Future With Cloud Computing - Dr. Werner Vogels - AWS Summit 2...Amazon Web Services
 
Introduction to Amazon Web Services
Introduction to Amazon Web ServicesIntroduction to Amazon Web Services
Introduction to Amazon Web ServicesAmazon Web Services
 
Kb12012011 amitava cloud_computing
Kb12012011 amitava cloud_computingKb12012011 amitava cloud_computing
Kb12012011 amitava cloud_computingAmitava Kumar
 
AWSSummit NYC- KeyNote by Werner Vogels
AWSSummit NYC- KeyNote by Werner VogelsAWSSummit NYC- KeyNote by Werner Vogels
AWSSummit NYC- KeyNote by Werner VogelsAmazon Web Services
 
Cloud Computing With Amazon Web Services, Part 1: Introduction - When It's Sm...
Cloud Computing With Amazon Web Services, Part 1: Introduction - When It's Sm...Cloud Computing With Amazon Web Services, Part 1: Introduction - When It's Sm...
Cloud Computing With Amazon Web Services, Part 1: Introduction - When It's Sm...white paper
 
Serving Media From The Edge - Miles Ward - AWS Summit 2012 Australia
Serving Media From The Edge - Miles Ward - AWS Summit 2012 AustraliaServing Media From The Edge - Miles Ward - AWS Summit 2012 Australia
Serving Media From The Edge - Miles Ward - AWS Summit 2012 AustraliaAmazon Web Services
 
Introduction to Clouds (Cloud Camp Columbus)
Introduction to Clouds (Cloud Camp Columbus)Introduction to Clouds (Cloud Camp Columbus)
Introduction to Clouds (Cloud Camp Columbus)John Willis
 
Cloud introduction2.ppt
Cloud introduction2.pptCloud introduction2.ppt
Cloud introduction2.pptBala Anand
 
O'Reilly Webcast: Architecting Applications For The Cloud
O'Reilly Webcast: Architecting Applications For The CloudO'Reilly Webcast: Architecting Applications For The Cloud
O'Reilly Webcast: Architecting Applications For The CloudO'Reilly Media
 
Workshop cloud computing basics (1)
Workshop cloud computing basics (1)Workshop cloud computing basics (1)
Workshop cloud computing basics (1)Infraright
 
Cloud Computing ...changes everything
Cloud Computing ...changes everythingCloud Computing ...changes everything
Cloud Computing ...changes everythingLew Tucker
 
Cloud Computing
Cloud ComputingCloud Computing
Cloud Computingwebscale
 
Cloud computing – An Overview
Cloud computing – An OverviewCloud computing – An Overview
Cloud computing – An OverviewKannan Subbiah
 

Similar a Rethinking the cloud_-_limitations_and_oppotunities_-_2011_nexcom (20)

Cloud Architectures - Jinesh Varia - GrepTheWeb
Cloud Architectures - Jinesh Varia - GrepTheWebCloud Architectures - Jinesh Varia - GrepTheWeb
Cloud Architectures - Jinesh Varia - GrepTheWeb
 
Keynote: Your Future With Cloud Computing - Dr. Werner Vogels - AWS Summit 2...
Keynote: Your Future With Cloud Computing - Dr. Werner Vogels  - AWS Summit 2...Keynote: Your Future With Cloud Computing - Dr. Werner Vogels  - AWS Summit 2...
Keynote: Your Future With Cloud Computing - Dr. Werner Vogels - AWS Summit 2...
 
Introduction to Amazon Web Services
Introduction to Amazon Web ServicesIntroduction to Amazon Web Services
Introduction to Amazon Web Services
 
Jeff barr Seattle_interactive_2011_q4
Jeff barr Seattle_interactive_2011_q4Jeff barr Seattle_interactive_2011_q4
Jeff barr Seattle_interactive_2011_q4
 
Kb12012011 amitava cloud_computing
Kb12012011 amitava cloud_computingKb12012011 amitava cloud_computing
Kb12012011 amitava cloud_computing
 
AWSSummit NYC- KeyNote by Werner Vogels
AWSSummit NYC- KeyNote by Werner VogelsAWSSummit NYC- KeyNote by Werner Vogels
AWSSummit NYC- KeyNote by Werner Vogels
 
Cloud Computing With Amazon Web Services, Part 1: Introduction - When It's Sm...
Cloud Computing With Amazon Web Services, Part 1: Introduction - When It's Sm...Cloud Computing With Amazon Web Services, Part 1: Introduction - When It's Sm...
Cloud Computing With Amazon Web Services, Part 1: Introduction - When It's Sm...
 
Serving Media From The Edge - Miles Ward - AWS Summit 2012 Australia
Serving Media From The Edge - Miles Ward - AWS Summit 2012 AustraliaServing Media From The Edge - Miles Ward - AWS Summit 2012 Australia
Serving Media From The Edge - Miles Ward - AWS Summit 2012 Australia
 
Self-Service Supercomputing
Self-Service SupercomputingSelf-Service Supercomputing
Self-Service Supercomputing
 
Cloud Talk
Cloud TalkCloud Talk
Cloud Talk
 
Introduction to Clouds (Cloud Camp Columbus)
Introduction to Clouds (Cloud Camp Columbus)Introduction to Clouds (Cloud Camp Columbus)
Introduction to Clouds (Cloud Camp Columbus)
 
Cloud introduction2.ppt
Cloud introduction2.pptCloud introduction2.ppt
Cloud introduction2.ppt
 
O'Reilly Webcast: Architecting Applications For The Cloud
O'Reilly Webcast: Architecting Applications For The CloudO'Reilly Webcast: Architecting Applications For The Cloud
O'Reilly Webcast: Architecting Applications For The Cloud
 
Workshop cloud computing basics (1)
Workshop cloud computing basics (1)Workshop cloud computing basics (1)
Workshop cloud computing basics (1)
 
Cloud Computing ...changes everything
Cloud Computing ...changes everythingCloud Computing ...changes everything
Cloud Computing ...changes everything
 
3 2011 aws mumbai2
3 2011 aws mumbai23 2011 aws mumbai2
3 2011 aws mumbai2
 
Cloud Computing
Cloud ComputingCloud Computing
Cloud Computing
 
Cloud computing – An Overview
Cloud computing – An OverviewCloud computing – An Overview
Cloud computing – An Overview
 
cloud.ppt
cloud.pptcloud.ppt
cloud.ppt
 
Analytics in the Cloud
Analytics in the CloudAnalytics in the Cloud
Analytics in the Cloud
 

Último

Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 

Último (20)

Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 

Rethinking the cloud_-_limitations_and_oppotunities_-_2011_nexcom

  • 1. Rethinking the Cloud - A View of Virtualization, Storage, Network, and Platform Woohyun Kim Cloud Platform Team S-Core 2011-05-18
  • 3. What is the Cloud? • A computing environment to elastically provide virtualized resources as a service over the Internet in a pay-as-you-go manner
  • 4. Amazon’s Challenge and Paradigm Shift
  • 5. Success Cases in Amazon SmugMug(http://www.smugmug.com/) • an online photo storage application that stores more than half a petabyte of data on S3 • estimates cost savings on service and storage to be close to $1 million New York Times(http://www.nytimes.com) • use EC2 to process terabytes of archival data using hundreds of EC2 instances within 36 hours 37Signals(http://37signals.com/) • maker of popular online project-management software Basecamp, uses S3 for storage needs. Animoto(http://animoto.com/) • an online presentation video generator that needs gobs of computing power for video processing • recently successfully withstood a surge in Web traffic that would kill most companies’ systems by scaling up their processing power quickly using EC2 with RightScale • Animoto ramped from 25,000 users to 250,000 users in three days, signing up 20,000 new users per hour at peak • Using RightScale, EC2 instances automatically scaled out 40 to 4000 at that time • For more detail, refer to http://blog.rightscale.com/2008/04/23/animoto-facebook- scale-up/
  • 6. Start-up Company: Powerset  Powerset had a great idea, “Natural Language Search”  It should index millions of pages of data and content  They knew that this would require a massively large datacenter and extensive computing power  CPUs, terminal switches, cable, racks, datacenters, hosting, power, maintenance, staffs  But they needed to keep infrastructure costs at a minimum “By using Amazon EC2, Powerset is able to match the infrastructure of large scale search companies on a startup budget.” - Barney Pell, Founder and CEO of Powerset ““Amazon EC2 is a complete game-changer. EC2 and Amazon Web Services make it easy for start-ups to build a complete infrastructure without having to spend much on capital .”- Paul Hammann 6
  • 7. Start-up Company: Powerset  Powerset had a great idea, “Natural Language Search”  It should index millions of pages of data and content  They knew that this would require a massively large datacenter and extensive computing power  CPUs, terminal switches, cable, racks, datacenters, hosting, power, maintenance, staffs  But they needed to keep infrastructure costs at a minimum “By using Amazon EC2, Powerset is able to match the $100 millions infrastructure of large scale search companies on a startup budget.” - Barney Pell, Founder and CEO of Powerset ““Amazon EC2 is a complete game-changer. EC2 and Amazon Web Services make it easy for start-ups to build a complete infrastructure without having to spend much on capital .”- Paul Hammann 7
  • 8. Temporary & Data-intensive : The New York Times  The New York Times is a 150-year old company, and serves the largest newspaper Website, NYTimes.com  1 billion page views per month  20+ million monthly unique visitors  They tried to convert TIFF images to PDFs  TIFF images(405,000),  Articles(3.3 million) in SGML PNG images(810,000)  XML files(405,000) mapping articles to TIFFs JavaScript files(405,000) “I had was this: upload 4TB of source data into S3, write some code that would run on numerous EC2 instances to read the source data, create PDFs, and store the results back into S3. S3 would then be used to serve the PDFs to the general public.” - Derek Gottfrid “I got access to a few more machines and churned through all 11 million articles in just under 24 hours using 100 EC2 instances, and generated another 1.5TB of data to store in S3. It just costs $3000.” - Derek Gottfrid 8
  • 9. Temporary & Data-intensive : The New York Times  The New York Times is a 150-year old company, and serves the largest newspaper Website, NYTimes.com  1 billion page views per month  20+ million monthly unique visitors  They tried to convert TIFF images to PDFs  TIFF images(405,000),  Articles(3.3 million) in SGML PNG images(810,000)  XML files(405,000) mapping articles to TIFFs JavaScript files(405,000) “I had was this: upload 4TB of source data into S3, write some code that would run on numerous EC2 instances to read the source data, create PDFs, and store the results back into S3. S3 would then be used to serve the PDFs to the general public.” “I got access to a few more machines and churned through all 11 million articles in just under 24 hours using 100 EC2 instances, and generated another 1.5TB of data to store in S3. It just costs $3000.” 9
  • 12. The Cloud is Falling Amazon’s cloud outages receive a lot of exposure … April 21 ~ 22, 2011 A networking glitch made its storage volumes automatically create back-ups of themselves, filling up storage capacity and causing connectivity issues, lasts two days Amazon’s customers include start-ups like the social networking site Foursquare but also big companies like Pfizer, Netflix and Nasdaq dkagh Affected web sites included Quora.com, Reddit.com, GroupMe.com and Scvngr.com July 20, 2008 Failure due to stranded zombies, lasts 5 hours Feb 15, 2008 Authentication overload leads to two-hour service outage October 2007 Service failure lasts two days October 2006 Security breach where users could see other users data … and their current SLAs don’t match those(99.99%) of enterprises Amazon EC2 99.95% Amazon S3 99.9%
  • 13. Cloud Is NOT A Brand-New Technology Utility Computing Google App Engine Amazon EC2 (April 2008) (August 2006) Microsoft Azure (Oct 2008) Amazon S3 (March 2006) GFS BigTable MapReduce Hadoop
  • 14. Cloud is just Buzz, and Marketing Hype Campaign • Cloud computing is simply a buzzword used to repackage grid computing and utility computing, both of which have existed for decades - Definition of Cloud Computing, whatis.com • What is it? What is it? ... Is it - 'Oh, I am going to access data on a server on the Internet.' That is cloud computing? • The interesting thing about cloud computing is that we’ve redefined cloud computing to include everything that we already do - During Oracle’s Analyst Day, Larry Ellison • .. cloud computing was simply a trap aimed at forcing more people to buy into locked, proprietary systems that would cost them more and more over time • It's stupidity. It's worse than stupidity: it's a marketing hype campaign - GNU founder, Richard Stallman • Server revenue for public cloud computing will grow from $582 million in 2009 to $718 million in 2014 • Server revenue for the much larger private cloud market will grow from $7.3 billion to $11.8 billion in the same time period - Worldwide Enterprise Server Cloud Computing 2010-2014 Forecast, IDC
  • 15. Cloud Wars and Strategies
  • 16. The Cloud Wars JBoss Xen 420M KVM Qumranet SPICE 107M RedHat NetScaler XenSource iSilon GreenPlum 300M OpenShift Citrix EMC CloudFoundry OpenStack 650M Rackspace VMware VMForce Oracle 450M BtrFS GemStone RabbitMQ SpringSource Force.com Ceph Hyperic Sun ZFS Heroku 212M SalesForce.com Java NexentaStor MySQL
  • 17. Infra is getting more Programmable rPath rBuilder Quest vControl 3Tera Applogic
  • 18. Cloud Disruptors • RightScale & enStratus
  • 19. Cloud Disruptors (cont’d) • Cloud Mgmt. Functionality in enStratus
  • 20. Cloudburst and Hybrid Cloud • CloudSwitch
  • 21. Who Moved My Cheese? Disrupting or Disrupted??!
  • 23. Virtualization on x86 Architecture • VMM(Virtual Machine Monitor) or Hypervisor – Since VMM must perform in the privileged level(0) , OS is moved to non-privileged level(1 or 3) app app app app app app Operating System Operating System Operating System #1 #2 #3 (Win-XP) (Mac-OS) (Linux) Virtual Machine Virtual Machine Virtual Machine Virtual Machine Manager CPU Memory NIC Disk • Problems on x86 Architecture – Privileged Instruction • Trap when called from CPU user mode, and Emulate its effect by VMM – Sensitive Non-privileged Instruction • Cause physical state of CPU to leak – smsw %eax # reads CR0 into EAX – mov %cr0, %edx # reads CR0 into EDX – sub %eax, %edx # what’s the difference? No Trap, No Emulation – jnz emulation_flaw # it ought to be zero!! => VMM is finally crashed
  • 24. CPU Virtualization on x86 Architecture • How to handle nonvirtualizable instructions – Full virtualization using binary translation – Paravirtualization using hypercalls – Hardware assisted virtualization using root/non-root mode • VT-x : Virtualization Technology for 32bit CPU • VT-i : Virtualization Technology for 64bit CPU • VT-d : Virtualization Technology for Directed I/O • VT-c : Virtualization Technology for Connectivity
  • 25. Virtualization on x86 Architecture
  • 26. Hurdles in Server Virtualization • Storage Allocation & Interfacing – On-demand, Pre-allocation – NAS, iSCSI, Local Storage • VM Management – Snapshot, Fast Cloning, Thin Provisioning, Live Migration, DRS • Virtual Network – L2/L3 Network Design, Directed, Bridged, NAT, VLAN, Load-Balance, Firewall • Resource Sharing – Resource Pool, High Availability, Scheduling, Workload Mgmt. • Migration – P2V, V2V • Hardware-Assisted Support – Privileged instruction virtualization • De-privileging or ring compression to handle privileged instructuions – Memory virtualization • Memory partitioning and allocation of physical memory – Device and I/O virtualization • Routing I/O requests between virtual devices and physical hardware
  • 28. File System with Shared Storage Cluster File Systems • GFS2 – DLM, scaling to 100 • GlusterFS – fuse, poor performance • Lustre - dfs
  • 29. Unified Storage using Virtual Block Pool • NexentaStor based on ZFS
  • 30. Cluster File System using Virtual Block Pool • GlusterFS
  • 31. A Feasible SAN File System • IBM TotalStorage SAN File System • GlusterFS • ZFS Pooled Storage • Openfiler • iSCSI+GNBD+DRBD
  • 33. Basic Virtual Network Tap vs. Tun • Tap – simulate an Ethernet device and operate with layer 2 packets such as Ethernet frames • Tun(nel) - simulate a network layer device and operate with layer 3 packets such as IP packets • TAP is used to create a network bridge, while TUN is used with routing. IPTables vs. Bridging • IPTables - let the host forward packets between each tap on its own subnets • Bridging – let all the taps connect into a specific bridge to put them on the same subnet VDE(Virtual Distributed Ethernet) and VDE Switch
  • 34. OpenStack Nova Network Virtualization ① Flat Mode Physical Machine eth VM • manual config. of bridge • get fixed public IP from the pool S/W NIC bridge public IP eth VM (br100) manual config eth VM ② Flat DHCP Mode Physical Machine eth VM dhcpdiscover • auto config. of bridge • get fixed public IP auto eth0 S/W NIC bridge public IP (br100) eth VM dhcpdiscover dnsmasq eth VM dhcpdiscover ③ VLAN DHCP Mode (default) Physical Machine supports private IP eth VM dhcpdiscover • auto config. of bridge, VLAN bridge from dnsmasq • auto config. of VLAN: range tagging (br100) of private IPs for project VLAN-1 S/W NIC VPN eth VM VLAN dhcpdiscover VM dnsmasq • get fixed private IP: iptables + NAT (private/public) Nova bridge • VLAN: cloudpipe (=openVPN eth VM dhcpdiscover users (br101) VM template  TAP/TUN)
  • 35. CloudStack Network Virtualization Virtual Private Network for Each Account
  • 36. CloudStack Network Virtualization Detail Virtual Private Network in Node A and Node B
  • 37. Hurdles in Network Virtualization • L2 Network – Problem: Scalability, Performance, Security – Solution: VLAN(for Scalability and Security), RBridge(for Scalability and STP Limitation), L2 over L3 • Multi-tier Networking Design vs. Migration Limitation – Limitation of Spanning Tree Protocol • Keep Layer 2 networks relatively small and join them together via Layer 3 segments • But VM migration cannot be live across the multi-tier networks – Port Consistency • Map the settings such as VLAN, ACL, QoS, and security profiles to all the network ports • But some VMs are not able to meet required service levels • L2(Switching) and L3(Routing) Networking Design – Scalability and Efficiency on the service provider side • Amazon EC2 using L3 – 500,000 VM on 60,000 PM – Legacy Support on the service consumer side • Amazon VPC, 3Tera AppLogic – Define virtual network topology – Select IP address range – Create public subnets and private subnets – Configure route table and network gateway
  • 39. Cloud Technologies Anatomy of Cloud Tehcnologies
  • 40. OpenStack Nova Architecture Volume Controller • provide fast and permanent block-level storage for the compute servers Network Controller • provide virtual networks to enable compute servers to interact with each other and with the public network Scheduler • select the most suitable compute controller to host an instance Compute Controller • compute server resources API Server Object Store • act as the Web services front end for • provide storage services the cloud controller Auth Manager Cloud Controller • provide authentication and authorization services • represent the global state and interact with all other components
  • 41. CloudStack Architecture Load Balancer Host A Host B Mgmt. Mgmt. Server VM Server Management Servers Computing Nodes Host X Host Y Host Z dynamically adding Pod Computing Nodes Guest VM Guest VM Guest VM Guest VM Computing Nodes should be Host X Host Y Host Z in the same subnet, Templates and have no limit to attach live migration Guest VM number of nodes copy, Guest VM Guest VM Guest VM ISO create,, images Max(6*Volumes) boot, Volumes VM Image Cluster perattach VM Guest live migration attach Max(16*Computing Nodes) per Cluster Primary Shared Storage VM VM Max(6*Volumes) VM Volumes per Guest VM Computing Nodes should be Image Image Image snapshot in the same subnet, and homogeneous Secondary Primary Shared Storage Shared Storage Zone
  • 43. Top 10 Cloud Obstacles and Opportunities • A View of Cloud Computing, ACM, April, 2010
  • 44. 2011 Predictions of IaaS, PaaS, and NoSQL • IaaS Prediction • Hybrid is the way to go: The public-private cloud discussion isn’t relevant anymore • Openstack will dominate the open IaaS offering • PaaS Prediction • 2011 is the year of PaaS • CloudFoundry – Vmware • OpenShift - Redhat • A new PaaS category will emerge – Building your own PaaS • CEAP(Cloud Enabled Application Platform) is being specifically designed to handle multi-tenancy, scalability, and on-demand provisioning, but not higher degree of flexibility and control • Application servers will change their name to PaaS – But won’t change their stripes • VMForce will fail to deliver on its promise => Already open Cloud • NoSQL (+Big Data) predictions • NoSQL will become compatible with SQL • More applications will run entirely In-Memory • Real-time /Stream-based analytics big will replace majority of the MapReduce batch processing • i.e., Yahoo S4, Google’s Percolator written by Nati Shalom at Gigaspaces http://natishalom.typepad.com/nati_shaloms_blog/2010/12/2011-cloud-paas-nosql-predictions.html