SlideShare a Scribd company logo
1 of 23
Download to read offline
Cloud Bursting 101: What to Do
When Computing Demand
Exceeds Capacity
WEBINAR | February 16, 2017
Housekeeping
2
●Recording
●Today’s Slides
●Attachments
● Questions
●Feedback
SubscribetoourBrightTalkChannel
Note to those new to BrightTalk: Audio
access to webinars is only via computer.
Phone numbers are not available.
Our Presenters
3
Damian Hasak
Solutions Architect Lead
Avere Systems
Bernhard Behn
Principal Technical Marketing Engineer
Avere Systems
4
And When Should You Use It
What is Cloud Bursting?
Cloud Bursting
• Extend existing filesystem(s) into public compute
environment
• Retain application compatibility
• Leverage on-demand compute at massive scale
• R/W caching to eliminate WAN latency
Cloud Bursting Benefits
Use as Much (Or As Little) Compute as is Needed
• Simple to scale up and down, bursting to the cloud with Avere brings balance and efficiency to
an infrastructure by eliminating the need to house and maintain idle IT resources but still gain
the ability to get the job done quickly.
No Need to Rewrite Enterprise Applications
• With the Avere vFXT, enterprise apps do not have to be rewritten to run in the compute cloud.
By creating a NAS-based performance storage tier in cloud computing, applications gain access
to the data they need, while overcoming inherent NAS verses object API incompatibilities.
Data is Easy to Move, Easy to Manage
• Expansion of compute resources no longer involves hardware ordering, shipping, installation,
networking, configuring, or testing.
7
Avere vFXT for Cloud Bursting
AvereOS Overview
• Components
• Clustered/Scale-out NAS system
• High-performance tier via R+W caching
• Global Namespace
• Built for parallel access: scale a working set across all nodes
• Scalable system
• 3 to 50 nodes within a single cluster
• Add nodes for additional performance and/or data locality
• High-Availability
• Single management interface, regardless of cluster size
• Easy expansion
Avere Overview: Public Compute
• Complete port to public compute environments
• All features + functionality intact: same look/fit/finish/feel as physical product
• Instantiated via dedicated management micro-instance
• Or, can be configured via command line interface
• Valuable for orchestration workflows
• One command, one cluster (no matter the size)
10
Performance and Efficiencies
Cloud Bursting for HPC
Cloud Bursting - Networking Challenges
Quick Deployment
SSH Tunneling
Flexible Network
VPN over Internet
Best Performance
Dedicated 10gig Link
-- Performance ++
++ Deployment Ease/Speed --
11
10Gbit
• Compute Cloud Components
• Compute Instances
• Disks/SSDs
• Network Interfaces
• AvereOS Software Components
• Install OS onto instance disks
• Bootstrap a configuration
• Cluster the system together
• vFXT Cluster
• IP addresses for clients to mount
• Core filer storage configuration
• Filesystem namespace access to on-premises data
Cloud Bursting – The Cloud Puzzle Pieces
12
SW/OS
SSD
NET
Cloud Bursting – Avere OS Orchestration Options
• Create/Monitor/Destroy vFXT clusters
• Interaction with Cloud Provider APIs
• Assemble the cloud components
• Interactions with Avere APIs
• Configure the cluster for file access
• Avere vFXT Cluster Manager
• Cloud Compute Instance with UI
• Avere provides access to disk image
• Graphical User Interface
• API (XMLRPC based)
• Python Library Orchestration
• Provide your own Python environment
• Avere provides python package
• Command-line orchestration
• Programmatic Python calls directly into library
13
AvereOS
SSD
Cloud Bursting – I/O Performance Capabilities
• GCE Instance Types
• n1-highmem-8: capable of 8Gbit/sec raw network throughput
• n1-highmem-32: capable of 16Gbit/sec raw network throughput
• GCE Disk Types
• PersistentSSD: capable of 240MB/sec read, 240MB/sec write
• LocalSSD: capable of 1560MB/sec read, 1090MB/sec write
• vFXT Scale-Out Clustering
• Adding vFXT nodes to the cluster grows disk cache and disk/network throughput
• Balancing client activity across the cluster vFXT nodes yields optimal performance
• vFXT File/Directory Caching
• Read or Read+Write caching
• Performance vs. Consistency can be dialed-in
14
15
Target Deployment / Use Cases
Use-case: Media and Entertainment
10.1.1.191:/vfx
NFS / SMB
VPN
Scenario
- Media company requires extra compute to complete a project, but cannot acquire new
hardware within a reasonable timeframe. Beg/borrow/steal isn’t an option.
- Avere vFXT cluster in EC2/GCE/Azure
- Render assets lifted from Local NAS to public compute environment
- Jobs run/data is processed, output written down to Local NAS
- vFXT cluster (and compute) powered down to reduce cost
- Data optionally resident in compute environment for next run
Local NAS /
Datacenter
Use-case: Financial Services and Technology
10.1.1.192:/marketdata
NFS / SMB
VPN
Scenario
- Financial Services company would like to process new models at massive scale for 1-2
hours per day, without changing researcher workflow
- Avere vFXT cluster in EC2/GCE/Azure
- Researcher generates new model, writing directly to Local NAS as always
- vFXT and 10,000-50,000 compute cores spun up to begin work
- Output written directly to Local NAS for additional interpretation
- Compute and vFXT assets powered down or removed to minimize cost
Local NAS /
Datacenter
Use-case: Life Sciences NGS
10.1.1.193:/genomes
NFS / SMB
VPN
Scenario
- A Drug Discovery company would like to process genetic data from drug trials at
massive scale in the compute cloud, without growing in-house compute infrastructure
- Avere vFXT cluster in EC2/GCE/Azure
- Researcher obtains new genomic models, writing directly to Local NAS as always
- vFXT and tens of thousands of compute cores spun up to begin work
- Output written directly via vFXT funnles back to Local NAS for additional analysis
- Compute and vFXT assets powered down or removed to minimize cost
Local NAS /
Datacenter
19
Live Demonstration
The Demo Environment
20
Infrastructure
• Network Connectivity
• Orchestrator
• vFXT cluster
• Compute Client Instances
Application
• Linux NFS grid application
• CPU intensive computation
• Read intensive I/O workload
On-Prem Datacenter
NAS
/on-prem-data
/on-prem-archive
Compute Cloud Clients have
real-time cached access to on-
premises data via GCE-hosted
vFXT Cluster
Clients on-premises create
the content and consume the
results, consisting of desktop
users, NAS storage and
compute grid
Google Cloud Project
vFXT
/ (root)
|--- /on-prem-data
|--- /on-prem-archive
compute-node##
GCE vFXT.py
Orchestrator
Pre-Flight Checklist
• Can the orchestrator reach the GCP Project network?
• Does the GCP Project network have a network path back to on-premises NAS
core filer?
• Can the GCE vfxt.py instance reach the googleapis.com endpoint?
• Is the vfxt.py python library installed on the GCE instance?
• Does the vfxt.py GCE instance have the proper permissions/credentials to
orchestrate the vFXT cluster creation?
• Determine the region/zone to build the cloud infrastructure
• Are the project quotas set up for the instances/storage we plan to deploy?
OK! Time for liftoff!!!
21
22
April 24-27, 2017
Las Vegas, Nevada
Booth SL9627
May 15-17, 2017
Hilton Midtown
New York, New York
May 17-18, 2017
Navy Pier
Chicago, Illinois
Questions?
23
Damian Hasak
Solutions Architect Lead
Avere Systems
dhasak@averesystems.com
Bernhard Behn
Principal Technical Marketing Engineer
Avere Systems
bbehn@averesystems.com
www.averesystems.
com

More Related Content

What's hot

Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...
Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...
Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...
Simplilearn
 
Red hat linux 9 ppt2003
Red hat linux 9 ppt2003Red hat linux 9 ppt2003
Red hat linux 9 ppt2003
ashishsjcit
 

What's hot (20)

Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...
Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...
Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop...
 
Multi-tenancy in Private Clouds
Multi-tenancy in Private CloudsMulti-tenancy in Private Clouds
Multi-tenancy in Private Clouds
 
Trust Frameworks Explained
Trust Frameworks ExplainedTrust Frameworks Explained
Trust Frameworks Explained
 
GitLab과 Kubernetes를 통한 CI/CD 구축
GitLab과 Kubernetes를 통한 CI/CD 구축GitLab과 Kubernetes를 통한 CI/CD 구축
GitLab과 Kubernetes를 통한 CI/CD 구축
 
Kubernetes architecture
Kubernetes architectureKubernetes architecture
Kubernetes architecture
 
Streaming, Database & Distributed Systems Bridging the Divide
Streaming, Database & Distributed Systems Bridging the DivideStreaming, Database & Distributed Systems Bridging the Divide
Streaming, Database & Distributed Systems Bridging the Divide
 
Cloud Computing: Virtualization
Cloud Computing: VirtualizationCloud Computing: Virtualization
Cloud Computing: Virtualization
 
Getting Started with Qlik Sense® Cloud: Understanding the Basics
Getting Started with Qlik Sense® Cloud: Understanding the BasicsGetting Started with Qlik Sense® Cloud: Understanding the Basics
Getting Started with Qlik Sense® Cloud: Understanding the Basics
 
Introduction to virtualization
Introduction to virtualizationIntroduction to virtualization
Introduction to virtualization
 
Hive(ppt)
Hive(ppt)Hive(ppt)
Hive(ppt)
 
Introduction to docker
Introduction to dockerIntroduction to docker
Introduction to docker
 
NoSQL Architecture Overview
NoSQL Architecture OverviewNoSQL Architecture Overview
NoSQL Architecture Overview
 
DevOps 3 - Docker.pdf
DevOps 3 - Docker.pdfDevOps 3 - Docker.pdf
DevOps 3 - Docker.pdf
 
Grid computing ppt
Grid computing pptGrid computing ppt
Grid computing ppt
 
Introduction to docker
Introduction to dockerIntroduction to docker
Introduction to docker
 
Docker by Example - Basics
Docker by Example - Basics Docker by Example - Basics
Docker by Example - Basics
 
Red hat linux 9 ppt2003
Red hat linux 9 ppt2003Red hat linux 9 ppt2003
Red hat linux 9 ppt2003
 
Advantages of Cloud Computing for Business
Advantages of Cloud Computing for BusinessAdvantages of Cloud Computing for Business
Advantages of Cloud Computing for Business
 
Caching Strategies
Caching StrategiesCaching Strategies
Caching Strategies
 
Serverless Computing
Serverless ComputingServerless Computing
Serverless Computing
 

Viewers also liked

Progress towards security in the Cloud-Héctor Sánchez, Microsoft
Progress towards security in the Cloud-Héctor Sánchez, MicrosoftProgress towards security in the Cloud-Héctor Sánchez, Microsoft
Progress towards security in the Cloud-Héctor Sánchez, Microsoft
Mind the Byte
 
Why Enterprise Architecture Must drive Cloud Strategy and Planning
Why Enterprise Architecture Must drive Cloud Strategy and PlanningWhy Enterprise Architecture Must drive Cloud Strategy and Planning
Why Enterprise Architecture Must drive Cloud Strategy and Planning
Rateb Abu Hawieleh
 
Security in Cloud Computing
Security in Cloud ComputingSecurity in Cloud Computing
Security in Cloud Computing
Rohit Buddabathina
 

Viewers also liked (20)

Top 10 benefits of cloud computing - By Techvedic
Top 10 benefits of cloud computing - By TechvedicTop 10 benefits of cloud computing - By Techvedic
Top 10 benefits of cloud computing - By Techvedic
 
Cloud 2.0: Containers, Microservices and Cloud Hybridization
Cloud 2.0: Containers, Microservices and Cloud HybridizationCloud 2.0: Containers, Microservices and Cloud Hybridization
Cloud 2.0: Containers, Microservices and Cloud Hybridization
 
Cloud Computing and Free Software: the eyeOS Project
Cloud Computing and Free Software: the eyeOS ProjectCloud Computing and Free Software: the eyeOS Project
Cloud Computing and Free Software: the eyeOS Project
 
Cloud 2.0 - How Containers, Microservices and Open Source Software are Redefi...
Cloud 2.0 - How Containers, Microservices and Open Source Software are Redefi...Cloud 2.0 - How Containers, Microservices and Open Source Software are Redefi...
Cloud 2.0 - How Containers, Microservices and Open Source Software are Redefi...
 
Overcoming Operational & Financial Barriers to Cloud
Overcoming Operational & Financial Barriers to CloudOvercoming Operational & Financial Barriers to Cloud
Overcoming Operational & Financial Barriers to Cloud
 
Cloud Computing: Helping Financial Institutions Leverage the Cloud to Improve...
Cloud Computing: Helping Financial Institutions Leverage the Cloud to Improve...Cloud Computing: Helping Financial Institutions Leverage the Cloud to Improve...
Cloud Computing: Helping Financial Institutions Leverage the Cloud to Improve...
 
Progress towards security in the Cloud-Héctor Sánchez, Microsoft
Progress towards security in the Cloud-Héctor Sánchez, MicrosoftProgress towards security in the Cloud-Héctor Sánchez, Microsoft
Progress towards security in the Cloud-Héctor Sánchez, Microsoft
 
6 trends that will shape Cloud computing in 2017 - An Infograph
6 trends that will shape Cloud computing in 2017 - An Infograph6 trends that will shape Cloud computing in 2017 - An Infograph
6 trends that will shape Cloud computing in 2017 - An Infograph
 
Information Technology in the Cloud
Information Technology in the CloudInformation Technology in the Cloud
Information Technology in the Cloud
 
Cloud Computing for Startups
Cloud Computing for StartupsCloud Computing for Startups
Cloud Computing for Startups
 
Embracing Cybersecurity on Cloud Computing
Embracing Cybersecurity on Cloud ComputingEmbracing Cybersecurity on Cloud Computing
Embracing Cybersecurity on Cloud Computing
 
Protecting Yourself from the Heartbleed Bug
Protecting Yourself from the Heartbleed BugProtecting Yourself from the Heartbleed Bug
Protecting Yourself from the Heartbleed Bug
 
Cloud Computing & Cybersecurity
Cloud Computing & CybersecurityCloud Computing & Cybersecurity
Cloud Computing & Cybersecurity
 
Why Enterprise Architecture Must drive Cloud Strategy and Planning
Why Enterprise Architecture Must drive Cloud Strategy and PlanningWhy Enterprise Architecture Must drive Cloud Strategy and Planning
Why Enterprise Architecture Must drive Cloud Strategy and Planning
 
Cloud computing
Cloud computingCloud computing
Cloud computing
 
Advantages of Cloud Computing
Advantages of Cloud ComputingAdvantages of Cloud Computing
Advantages of Cloud Computing
 
Cloud Computing #Manoj_Rockstar
Cloud Computing #Manoj_RockstarCloud Computing #Manoj_Rockstar
Cloud Computing #Manoj_Rockstar
 
Security in Cloud Computing
Security in Cloud ComputingSecurity in Cloud Computing
Security in Cloud Computing
 
Security Trend Report, 2017
Security Trend Report, 2017Security Trend Report, 2017
Security Trend Report, 2017
 
The Benefits of Cloud Computing
The Benefits of Cloud ComputingThe Benefits of Cloud Computing
The Benefits of Cloud Computing
 

Similar to Cloud Bursting 101: What to do When Cloud Computing Demand Exceeds Capacity

Building a PaaS Platform like Bluemix on OpenStack
Building a PaaS Platform like Bluemix on OpenStackBuilding a PaaS Platform like Bluemix on OpenStack
Building a PaaS Platform like Bluemix on OpenStack
Animesh Singh
 

Similar to Cloud Bursting 101: What to do When Cloud Computing Demand Exceeds Capacity (20)

Building a Just-in-Time Application Stack for Analysts
Building a Just-in-Time Application Stack for AnalystsBuilding a Just-in-Time Application Stack for Analysts
Building a Just-in-Time Application Stack for Analysts
 
SharePoint on Azure
SharePoint on Azure SharePoint on Azure
SharePoint on Azure
 
Data Center for Cloud Computing - DC3X
Data Center for Cloud Computing - DC3XData Center for Cloud Computing - DC3X
Data Center for Cloud Computing - DC3X
 
Deploying couchbaseserverazure cihanbiyikoglu_microsoft
Deploying couchbaseserverazure cihanbiyikoglu_microsoftDeploying couchbaseserverazure cihanbiyikoglu_microsoft
Deploying couchbaseserverazure cihanbiyikoglu_microsoft
 
2689 - Exploring IBM PureApplication System and IBM Workload Deployer Best Pr...
2689 - Exploring IBM PureApplication System and IBM Workload Deployer Best Pr...2689 - Exploring IBM PureApplication System and IBM Workload Deployer Best Pr...
2689 - Exploring IBM PureApplication System and IBM Workload Deployer Best Pr...
 
Throughwave Day 2015 - Throughwave Next Generation Enterprise IT Infrastructure
Throughwave Day 2015 - Throughwave Next Generation Enterprise IT InfrastructureThroughwave Day 2015 - Throughwave Next Generation Enterprise IT Infrastructure
Throughwave Day 2015 - Throughwave Next Generation Enterprise IT Infrastructure
 
VMworld 2014: How to Build a Hybrid Cloud
VMworld 2014: How to Build a Hybrid CloudVMworld 2014: How to Build a Hybrid Cloud
VMworld 2014: How to Build a Hybrid Cloud
 
What's new in informix v11.70
What's new in informix v11.70What's new in informix v11.70
What's new in informix v11.70
 
Nutanix and microsoft_webinar_oct_28
Nutanix and microsoft_webinar_oct_28Nutanix and microsoft_webinar_oct_28
Nutanix and microsoft_webinar_oct_28
 
Microsoft Server Virtualization and Private Cloud
Microsoft Server Virtualization and Private CloudMicrosoft Server Virtualization and Private Cloud
Microsoft Server Virtualization and Private Cloud
 
Presentation cisco unified fabric
Presentation   cisco unified fabricPresentation   cisco unified fabric
Presentation cisco unified fabric
 
Server Virtualization using Hyper-V
Server Virtualization using Hyper-VServer Virtualization using Hyper-V
Server Virtualization using Hyper-V
 
AWS Webcast - Datacenter Migration to AWS
AWS Webcast - Datacenter Migration to AWSAWS Webcast - Datacenter Migration to AWS
AWS Webcast - Datacenter Migration to AWS
 
Cloud ppt
Cloud pptCloud ppt
Cloud ppt
 
Unleash the Power of Open Networking
Unleash the Power of Open NetworkingUnleash the Power of Open Networking
Unleash the Power of Open Networking
 
What's New in Windows Server 2012 R2
What's New in Windows Server 2012 R2What's New in Windows Server 2012 R2
What's New in Windows Server 2012 R2
 
Boyan Krosnov - Building a software-defined cloud - our experience
Boyan Krosnov - Building a software-defined cloud - our experienceBoyan Krosnov - Building a software-defined cloud - our experience
Boyan Krosnov - Building a software-defined cloud - our experience
 
Building a PaaS Platform like Bluemix on OpenStack
Building a PaaS Platform like Bluemix on OpenStackBuilding a PaaS Platform like Bluemix on OpenStack
Building a PaaS Platform like Bluemix on OpenStack
 
Desktop Virtualization made Simpler
Desktop Virtualization made SimplerDesktop Virtualization made Simpler
Desktop Virtualization made Simpler
 
Pivotal cloud cache for .net microservices
Pivotal cloud cache for .net microservicesPivotal cloud cache for .net microservices
Pivotal cloud cache for .net microservices
 

More from Avere Systems

More from Avere Systems (19)

Scaling Security Workflows in Government Agencies
Scaling Security Workflows in Government AgenciesScaling Security Workflows in Government Agencies
Scaling Security Workflows in Government Agencies
 
Hedge Fund IT Challenges Financial Survey
Hedge Fund IT Challenges Financial SurveyHedge Fund IT Challenges Financial Survey
Hedge Fund IT Challenges Financial Survey
 
Solving enterprise challenges through scale out storage & big compute final
Solving enterprise challenges through scale out storage & big compute finalSolving enterprise challenges through scale out storage & big compute final
Solving enterprise challenges through scale out storage & big compute final
 
Deliver Best-in-Class HPC Cloud Solutions Without Losing Your Mind
Deliver Best-in-Class HPC Cloud Solutions Without Losing Your MindDeliver Best-in-Class HPC Cloud Solutions Without Losing Your Mind
Deliver Best-in-Class HPC Cloud Solutions Without Losing Your Mind
 
Compute Cloud for Rendering
Compute Cloud for RenderingCompute Cloud for Rendering
Compute Cloud for Rendering
 
Rendering Takes Flight
Rendering Takes FlightRendering Takes Flight
Rendering Takes Flight
 
Three Steps to Modern Media Asset Management with Active Archive
Three Steps to Modern Media Asset Management with Active ArchiveThree Steps to Modern Media Asset Management with Active Archive
Three Steps to Modern Media Asset Management with Active Archive
 
Cloud Computing Gets Put to the Test
Cloud Computing Gets Put to the TestCloud Computing Gets Put to the Test
Cloud Computing Gets Put to the Test
 
Scientific Computing in the Cloud: Speeding Access for Drug Discovery
Scientific Computing in the Cloud: Speeding Access for Drug DiscoveryScientific Computing in the Cloud: Speeding Access for Drug Discovery
Scientific Computing in the Cloud: Speeding Access for Drug Discovery
 
Build a Cloud Render-Ready Infrastructure
Build a Cloud Render-Ready InfrastructureBuild a Cloud Render-Ready Infrastructure
Build a Cloud Render-Ready Infrastructure
 
4 C’s for Using Cloud to Support Scientific Research
4 C’s for Using Cloud to Support Scientific Research4 C’s for Using Cloud to Support Scientific Research
4 C’s for Using Cloud to Support Scientific Research
 
Share on LinkedIn Share on Twitter Share on Facebook Share on Google+ Share b...
Share on LinkedIn Share on Twitter Share on Facebook Share on Google+ Share b...Share on LinkedIn Share on Twitter Share on Facebook Share on Google+ Share b...
Share on LinkedIn Share on Twitter Share on Facebook Share on Google+ Share b...
 
Avere & AWS Enterprise Solution with Special Bundle Pricing Offer
Avere & AWS Enterprise Solution with Special Bundle Pricing OfferAvere & AWS Enterprise Solution with Special Bundle Pricing Offer
Avere & AWS Enterprise Solution with Special Bundle Pricing Offer
 
Enable Enterprise Hybrid Cloud NAS
Enable Enterprise Hybrid Cloud NASEnable Enterprise Hybrid Cloud NAS
Enable Enterprise Hybrid Cloud NAS
 
Avere Cloud NAS
Avere Cloud NASAvere Cloud NAS
Avere Cloud NAS
 
Clouds in Your Coffee Session with Cleversafe & Avere
Clouds in Your Coffee Session with Cleversafe & AvereClouds in Your Coffee Session with Cleversafe & Avere
Clouds in Your Coffee Session with Cleversafe & Avere
 
Are you ready for Avere Cloud NAS?
Are you ready for Avere Cloud NAS?Are you ready for Avere Cloud NAS?
Are you ready for Avere Cloud NAS?
 
Optimizing the Upstreaming Workflow: Flexibly Scale Storage for Seismic Proce...
Optimizing the Upstreaming Workflow: Flexibly Scale Storage for Seismic Proce...Optimizing the Upstreaming Workflow: Flexibly Scale Storage for Seismic Proce...
Optimizing the Upstreaming Workflow: Flexibly Scale Storage for Seismic Proce...
 
Webinar: Untethering Compute from Storage
Webinar: Untethering Compute from StorageWebinar: Untethering Compute from Storage
Webinar: Untethering Compute from Storage
 

Recently uploaded

Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Recently uploaded (20)

Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelMcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Vector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptxVector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptx
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 

Cloud Bursting 101: What to do When Cloud Computing Demand Exceeds Capacity

  • 1. Cloud Bursting 101: What to Do When Computing Demand Exceeds Capacity WEBINAR | February 16, 2017
  • 2. Housekeeping 2 ●Recording ●Today’s Slides ●Attachments ● Questions ●Feedback SubscribetoourBrightTalkChannel Note to those new to BrightTalk: Audio access to webinars is only via computer. Phone numbers are not available.
  • 3. Our Presenters 3 Damian Hasak Solutions Architect Lead Avere Systems Bernhard Behn Principal Technical Marketing Engineer Avere Systems
  • 4. 4 And When Should You Use It What is Cloud Bursting?
  • 5. Cloud Bursting • Extend existing filesystem(s) into public compute environment • Retain application compatibility • Leverage on-demand compute at massive scale • R/W caching to eliminate WAN latency
  • 6. Cloud Bursting Benefits Use as Much (Or As Little) Compute as is Needed • Simple to scale up and down, bursting to the cloud with Avere brings balance and efficiency to an infrastructure by eliminating the need to house and maintain idle IT resources but still gain the ability to get the job done quickly. No Need to Rewrite Enterprise Applications • With the Avere vFXT, enterprise apps do not have to be rewritten to run in the compute cloud. By creating a NAS-based performance storage tier in cloud computing, applications gain access to the data they need, while overcoming inherent NAS verses object API incompatibilities. Data is Easy to Move, Easy to Manage • Expansion of compute resources no longer involves hardware ordering, shipping, installation, networking, configuring, or testing.
  • 7. 7 Avere vFXT for Cloud Bursting
  • 8. AvereOS Overview • Components • Clustered/Scale-out NAS system • High-performance tier via R+W caching • Global Namespace • Built for parallel access: scale a working set across all nodes • Scalable system • 3 to 50 nodes within a single cluster • Add nodes for additional performance and/or data locality • High-Availability • Single management interface, regardless of cluster size • Easy expansion
  • 9. Avere Overview: Public Compute • Complete port to public compute environments • All features + functionality intact: same look/fit/finish/feel as physical product • Instantiated via dedicated management micro-instance • Or, can be configured via command line interface • Valuable for orchestration workflows • One command, one cluster (no matter the size)
  • 11. Cloud Bursting - Networking Challenges Quick Deployment SSH Tunneling Flexible Network VPN over Internet Best Performance Dedicated 10gig Link -- Performance ++ ++ Deployment Ease/Speed -- 11 10Gbit
  • 12. • Compute Cloud Components • Compute Instances • Disks/SSDs • Network Interfaces • AvereOS Software Components • Install OS onto instance disks • Bootstrap a configuration • Cluster the system together • vFXT Cluster • IP addresses for clients to mount • Core filer storage configuration • Filesystem namespace access to on-premises data Cloud Bursting – The Cloud Puzzle Pieces 12 SW/OS SSD NET
  • 13. Cloud Bursting – Avere OS Orchestration Options • Create/Monitor/Destroy vFXT clusters • Interaction with Cloud Provider APIs • Assemble the cloud components • Interactions with Avere APIs • Configure the cluster for file access • Avere vFXT Cluster Manager • Cloud Compute Instance with UI • Avere provides access to disk image • Graphical User Interface • API (XMLRPC based) • Python Library Orchestration • Provide your own Python environment • Avere provides python package • Command-line orchestration • Programmatic Python calls directly into library 13 AvereOS SSD
  • 14. Cloud Bursting – I/O Performance Capabilities • GCE Instance Types • n1-highmem-8: capable of 8Gbit/sec raw network throughput • n1-highmem-32: capable of 16Gbit/sec raw network throughput • GCE Disk Types • PersistentSSD: capable of 240MB/sec read, 240MB/sec write • LocalSSD: capable of 1560MB/sec read, 1090MB/sec write • vFXT Scale-Out Clustering • Adding vFXT nodes to the cluster grows disk cache and disk/network throughput • Balancing client activity across the cluster vFXT nodes yields optimal performance • vFXT File/Directory Caching • Read or Read+Write caching • Performance vs. Consistency can be dialed-in 14
  • 16. Use-case: Media and Entertainment 10.1.1.191:/vfx NFS / SMB VPN Scenario - Media company requires extra compute to complete a project, but cannot acquire new hardware within a reasonable timeframe. Beg/borrow/steal isn’t an option. - Avere vFXT cluster in EC2/GCE/Azure - Render assets lifted from Local NAS to public compute environment - Jobs run/data is processed, output written down to Local NAS - vFXT cluster (and compute) powered down to reduce cost - Data optionally resident in compute environment for next run Local NAS / Datacenter
  • 17. Use-case: Financial Services and Technology 10.1.1.192:/marketdata NFS / SMB VPN Scenario - Financial Services company would like to process new models at massive scale for 1-2 hours per day, without changing researcher workflow - Avere vFXT cluster in EC2/GCE/Azure - Researcher generates new model, writing directly to Local NAS as always - vFXT and 10,000-50,000 compute cores spun up to begin work - Output written directly to Local NAS for additional interpretation - Compute and vFXT assets powered down or removed to minimize cost Local NAS / Datacenter
  • 18. Use-case: Life Sciences NGS 10.1.1.193:/genomes NFS / SMB VPN Scenario - A Drug Discovery company would like to process genetic data from drug trials at massive scale in the compute cloud, without growing in-house compute infrastructure - Avere vFXT cluster in EC2/GCE/Azure - Researcher obtains new genomic models, writing directly to Local NAS as always - vFXT and tens of thousands of compute cores spun up to begin work - Output written directly via vFXT funnles back to Local NAS for additional analysis - Compute and vFXT assets powered down or removed to minimize cost Local NAS / Datacenter
  • 20. The Demo Environment 20 Infrastructure • Network Connectivity • Orchestrator • vFXT cluster • Compute Client Instances Application • Linux NFS grid application • CPU intensive computation • Read intensive I/O workload On-Prem Datacenter NAS /on-prem-data /on-prem-archive Compute Cloud Clients have real-time cached access to on- premises data via GCE-hosted vFXT Cluster Clients on-premises create the content and consume the results, consisting of desktop users, NAS storage and compute grid Google Cloud Project vFXT / (root) |--- /on-prem-data |--- /on-prem-archive compute-node## GCE vFXT.py Orchestrator
  • 21. Pre-Flight Checklist • Can the orchestrator reach the GCP Project network? • Does the GCP Project network have a network path back to on-premises NAS core filer? • Can the GCE vfxt.py instance reach the googleapis.com endpoint? • Is the vfxt.py python library installed on the GCE instance? • Does the vfxt.py GCE instance have the proper permissions/credentials to orchestrate the vFXT cluster creation? • Determine the region/zone to build the cloud infrastructure • Are the project quotas set up for the instances/storage we plan to deploy? OK! Time for liftoff!!! 21
  • 22. 22 April 24-27, 2017 Las Vegas, Nevada Booth SL9627 May 15-17, 2017 Hilton Midtown New York, New York May 17-18, 2017 Navy Pier Chicago, Illinois
  • 23. Questions? 23 Damian Hasak Solutions Architect Lead Avere Systems dhasak@averesystems.com Bernhard Behn Principal Technical Marketing Engineer Avere Systems bbehn@averesystems.com www.averesystems. com