Managing Hybrid Cloud Infrastructure

1
Managing Heterogeneous Cloud Infrastructure
with OpenStack
Example:
Hosting ‘Hybrid’ Applications Comprising Bare-Metal and Virtualized Resources
OpenStack Israel, June 2nd, 2014, Hertzliya
IBM Research – Haifa
Alex Glikson
Ezra Silvera

2 © 2014 IBM Corporation
Outline
Heterogeneous Clouds
‒ Background & Motivation
‒ Opportunities and Challenges
Hosting ‘Hybrid’ (Bare-Metal + Virtualized) Applications
‒ Goals
‒ Scenarios
‒ Solution Approach
‒ Technical Challenges and Design Considerations
‒ Example
Summary, Q&A

Evolution of IaaS Usage Patterns and Offerings
Large variety of configurations
‒ E.g., AWS now offers >60
instance configurations
Rise of bare-metal providers
‒ IBM/SoftLayer, Internap, etc
Small set of
standardized (virtual)
hardware configurations
‒ e.g., 1 instance type
in AWS EC2 (2006)
Commonly
Offered
Hardware
Configurations
Growing spectrum of workloads
‒ E.g., ‘legacy’ business
applications, HPC, analytics, etc
Dev & test
Web applications
(‘cloud-native’)
Common
Usage Patterns
Agility, efficiency, elasticity, self-service, pay-per-useValue
proposition
NowEarly Days
Trend: Increased Cloud Infrastructure Heterogeneity (including bare-metal)
to Support Larger Variety of Applications with Special
Requirements

Embracing Infrastructure Heterogeneity
for the Benefit of the Application: Example
Virtualization
Network
Storage
Compute
CPU/mem/disk
balance
CPU-rich mem-rich disk-rich …
CPUs GPGPUs Accelerators
Local HDD SSD SAN NAS
…
…1 Gbit shared 10 Gbit shared 10 Gbit reserved
…
bare-metal KVM LXC VMware …
Tier-1: web app
Tier-2: 3D rendering
Tier-4: Designs DB Tier-3: Hadoop
balanced
Example: Application for Collaborative 3D Design

Embracing Infrastructure Heterogeneity
for the Benefit of the Application: Example
Virtualization
Network
Storage
Compute
CPU/mem/disk
balance
CPU-rich mem-rich disk-rich …
CPUs GPGPUs Accelerators
Local HDD SSD SAN NAS
…
…1 Gbit shared 10 Gbit shared 10 Gbit reserved
…
bare-metal KVM LXC VMware …
Tier-1: web app
Tier-2: 3D rendering
Tier-4: Designs DB Tier-3: Hadoop
balanced
Example: Application for Collaborative 3D Design
4
4
4
4 3
3
3
3
2
2
2
2
2
3
41
1
1
1
1

7
Opportunities and Challenges

Business opportunities enabled by infrastructure heterogeneity
‒ Service/application providers:
• Cloud adoption in ‘traditional’ market segments agility and efficiency
– E.g., manufacturing, finance, scientific, etc
– IDC: the (extended) HPC market alone is larger than IaaS ($15B vs $9B)
‒ Cloud/infrastructure providers:
• Ability to better fit to specific classes of applications differentiation
– Especially relevant given the overall commoditization of the IaaS market,
recent price wars, etc

Challenge: cloud infrastructure natively designed for:
‒ Underlying infrastructure heterogeneity
‒ Ability to seamlessly host applications with special requirements
‒ Still preserve the characteristics of the cloud model
• Agility, efficiency, elasticity, etc

Heterogeneous Cloud Requirements: Cloud User
Cloud user perspective
‒ Access to a wider range of offered HW/SW configurations
• Performance, security/isolation
• Compute, storage, network
‒ Same or better user experience as with ‘regular’ cloud, e.g.:
• Same: single/unified catalog of configuration ‘flavors’, images, etc
• Same: seamless network connectivity, storage access, etc
• Same: higher-level services (orchestration, LBaaS, DBaaS, etc)
• Better: higher-level application abstraction, hiding the details of the
underlying HW (critical with increased complexity of underlying HW)
– Specify goals/requirements, rather than specific resource types/models etc

Heterogeneous Cloud Requirements: Cloud Provider
Cloud provider perspective
‒ Minimize management cost by unified processes and APIs
• Provisioning, inventory, monitoring, life cycle operations, visualization, etc
– Unified resource model
‒ Secure multi-tenancy across heterogeneous resources
‒ Elasticity & efficiency
• Smart scheduling, to overcome resource fragmentation
• Dynamic balancing between HW-compatible pools by “repurposing”
(subject to compatibility)
– Capacity monitoring & prediction
– Automated evacuation & re-provisioning

Heterogeneous Cloud: OpenStack Perspective
OpenStack is following a similar pattern
‒ Started primarily with dev & test and cloud-native applications
• Simplistic approach to flavors, scheduling, performance optimization, etc
‒ Extensive efforts to enable adoption by additional applications, e.g.:
• Hadoop (Sahara)
• DBaaS (Trove)
• Bare-metal (Ironic, TripleO, Tuskar)
• Scheduling (Gantt)
• Reservations (Climate)
• Policies (Congress)
• Etc
‒ Growing community interest in supporting heterogeneous clouds, e.g.:
• Multi-Tenant Bare Metal Provisioning (Mon, 11:15, B206; Tue, 2:50pm, B301)
• Deploying OpenStack in a Multi-Hypervisor Environment (Wed, 11am, B101)
• IBM, SoftLayer and OpenStack - Present and Future (Wed, 11:50am, B312)
• Many more related sessions…

Heterogeneous Cloud: OpenStack to the Rescue!
Claim: OpenStack provides a good basis for
building heterogeneous clouds
• But few gaps yet to be addressed

Hosting ‘Hybrid’ (Bare-Metal + Virtualized)
Applications

Hybrid Applications – Background and Goals
Hybrid environment – includes both physical and virtual resources
‒ Pools: Resources are grouped into bare-metal and virtual pools
Hybrid application – a composite (e.g., multi-tier) application that
spans both virtual and physical server(s)
Goals
‒ Admin: Minimize management complexity and overhead. Unified
management across bare-metal and virtualized pools.
‒ User: Simplified ongoing management and application life cycle.
Design guideline: If possible try to use OpenStack native solution
‒ E.g., no external coordinators, orchestration, management

Pool 3: Virt Type 2 Pool 4: Bare metalPool 1: Spare (BM) Pool 2: Virt Type 1
Tenant ATenant B
Hybrid Applications – Background and Goals

Scenarios

Scenario 1: Node Repurposing
Support native policies that dynamically balance the virtual and
physical pools (e.g., moving servers from one to another)
Repurposing Virtualized node (Hypervisor) Physical node
1. Detect resource congestion in the physical pool
2. Identify a candidate host to be re-purposed into physical
3. Trigger “evacuate“ for all VMs on the selected host
4. Assign physical host to the BM pool
‒ Server is ready to be deployed as bare-metal application (later on)
Bare metal pool
Bare metal pool

Scenario 2: Runtime Decision on Target Pool
Scenario: decide on the server type (BM/VM) upon deployment
E.g., according to performance requirements
1. User deploys an application without specifying the server type
2. The system automatically choose server type
3. Automatically choose/construct actual image for deployment
‒ Option 1: dynamic adaptation
‒ Option 2: Maintain multiple ‘versions’
4. Deploy using corresponding provisioning mechanism
‒ leveraging Ironic
‒ Apply adaptation upon boot (e.g., leveraging Heat)

Solution Approach

Management Approaches
Possible solutions range from 'share nothing to 'share everything'
‒ 'share nothing’ - dedicated services for each pool
‒ ‘share everything’ - Use single copy of each service
‒ Some of the services are shared (e.g., regions, cells)
OpenStack can natively handle bare metal and virtual hosts, so
why not combining them together in a single OpenStack?
We focused on the 'shared everything' architecture
‒ Defined as 'Integrated Management‘

Integrated Management: Basic Architecture
Special resource types, mapped to specially designed resource pools
Use multiple host aggregates, including ‘bare-metal’
‒ Use aggregate filters for scheduling
Networking
‒ Admin network for provisioning and management of compute nodes
‒ Separate management network for BM machines hosting user applications
‒ Data network combining both VMs and BM nodes
Controller node
mysql-server
rabbit-server
neutron-server
glance
Network node
Neutron.*-plugin-agent
Neutron-l3-agent
Neutron-dhcp-agent
Neutron-l3-agent
BM computeCompute node
nova-compute
Compute node
nova-compute
Compute node
nova-compute
BM
node
BM
node
BM
node
External
Data/Guests
Management
API
Internet
BM Mgt
BM
node
BM
node
BM
spare

Advantages of Integrated Management approach
Native and simple OpenStack-based solution
‒ Doesn’t require external orchestration
‒ No need to coordinate between services of different pools
Simplified administration
‒ Controlling all pools/resources from one place in a unified manner
‒ Reduce configuration complexity and overhead
‒ Better diagnostics and root cause analysis
Improved user experience and manageability
‒ Maintain/expose combined topology view for different server types
‒ Hide the underlying infrastructure details from the user (if needed)

Technical Challenges and
Design Considerations

Challenges Examples
OpenStack has some gaps when managing bare-metal and
virtual machines simultaneously
Networking: Simultaneously manage both virtual and physical
networks
‒ Network isolation for tenants should combine both BM and VMs
‒ E.g., Use tunnels/overlays connecting physical nodes and VMs
‒ E.g., Use multiple plugins (e.g., OVS, openflow) simultaneously to
gain advantages for each pool (if possible)
Managing Bare-Metal workloads
‒ Security in multi-tenant environments (e.g., device firmware)

Challenges Examples (Cont.) - Compute
Enhanced scheduler to better support heterogeneity
‒ Use different placement policies for different pools (e.g., look at
GPU consumption rather then CPU/memory)
Support instance management for BM
‒ Actions for BM instances may differ from actions on VMs
‒ E.g., implement: Console, Explore usage of: Suspend, Resize
Add hierarchical relations between BM instance and the
compute node running on it
‒ E.g., Propagate BM delete command to the VM level
Support aggregates for specific BM nodes

Challenges Examples (Cont.)
Image Management
‒ Enhance access control for images in Glance
‒ Support run-time image selection
‒ Support dynamic adaptation of images
UI: extend UI to support Hybrid applications/instances
‒ Add bare-metal node management in Horizon
‒ Physical machines are shown as Hypervisors

Hybrid Application example

Example – Deploying Hybrid WordPress
Apache+WordPress on VM and MySQL on physical node
Setup and configuration
‒ A single HEAT template used for the two nodes
‒ Using host aggregates for scheduling
‒ Used diskimage-builder to build the baremetal images
• cloud-init element installed

Example – Deploying Hybrid WordPress
Example gaps reflected in Horizon:
‒ BM nodes are shown as Hypervisors
‒ Deployment images are exposed to users

Summary

Summary
Heterogeneous cloud environments are gaining momentum
‒ Critical in order to host a broad spectrum of applications
OpenStack is a promising solution to manage hybrid clouds
‒ By using integrated management we can have
• Simple native OpenStack solution
• Simplified administration and enhanced user experience
Need a careful design to ensure all requirements (e.g., security,
isolation) are maintained across the hybrid environment
Some gaps need to be addressed in OpenStack

Q & A

Managing Hybrid Cloud Infrastructure

Recommended

Recommended

More Related Content

What's hot

What's hot (18)

Viewers also liked

Viewers also liked (6)

Similar to Managing Hybrid Cloud Infrastructure

Similar to Managing Hybrid Cloud Infrastructure (20)

More from Cloud Native Day Tel Aviv

More from Cloud Native Day Tel Aviv (20)

Recently uploaded

Recently uploaded (20)

Managing Hybrid Cloud Infrastructure