SlideShare una empresa de Scribd logo
1 de 31
Descargar para leer sin conexión
Dell and CEPH 
Steve Smith: 
Steve_l_smith@dell.com 
@SteveSAtDell 
Paul Brook 
Paul_brook@dell.com 
Twitter @PaulBrookAtDell 
Ceph Day London 
October 22nd 2014
agenda 
• Why we are here. – we sell CEPH support 
• You need hardware to sit this on. Here are some ideas 
• Some best practice shared with CEPH colleagues this year 
• A concept – (Research Data – would like your input) 
Dell Corporation
Dell is a certified reseller of Red Hat-Inktank 
Services, Support and Training. 
• Need to Access and buy Red Hat Services & Support? 
15+ Years of Red Hat and Dell 
• Red Hat 1-year /3-year subscription packages 
– Inktank Pre-Production subscription 
– Gold (24*7) Subscription 
• Red Hat Professional Services 
– Ceph Pro services Starter Pack 
– Additional days services options 
– Ceph Training from Red Hat 
Or…you can download CEPH for Free 
Dell Corporation 
3Confidential
Components Involved 
http://docs.openstack.org/training-guides/content/module001-ch004-openstack-architecture.html 
Dell Corporation
Dell OpenStack Cloud Solution 
You Get 
Stuff 
Stuff 
Dell Corporation
Best Practices 
(well…….some) 
With acknowledgement and thanks to Kyle and Mark at InkTank 
Dell Corporation
Planning your Ceph Implementation 
• Business Requirements 
– Budget considerations, organisational commitment 
– Replacing Enterprise SAN/NAS for cost saving 
– xaaS use cases for massive-scale, cost-effective storage 
– Avoid lock-in – use open source and industry standards 
– Steady-state vs. Spike data usage 
• Sizing requirements 
– What is the initial storage capacity? 
– What is the expected growth rate? 
• Workload requirements 
– Does the workload need high performance or it is more capacity 
focused? 
– What are IOPS/Throughput requirements? 
– What applications will be running on Ceph cluster? 
– What type of data will be stored? 
Dell Corporation
Architectural considerations – Redundancy and 
replication considerations 
• Tradeoff between Cost vs. Reliability (use-case dependent) 
• How many node failures can be tolerated? 
• In a multi-rack scenario, should a whole rack failure be 
tolerated? 
• Is there a need for multi-site data replication? 
• Erasure coding (more capacity with the same raw disk. More 
CPU load) 
• Plan for redundancy of the monitor nodes – distribute across 
fault zones 
• 3 copies = 8 nines availability, less than 1 second downtime per 
year 
• Many many things affect performance - in Ceph, above Ceph 
and below Ceph. 
Dell Corporation
Understanding Your Workload 
Dell Corporation
CEPH Architecture Refresh 
Dell Corporation
Understanding Ceph (1) 
Dell Corporation
Understanding Ceph (2) 
Dell Corporation
Understanding The Storage Server 
Dell Corporation
Multi-Site Issues 
• Within a CEPH cluster RADOS enforces Strong Consistency 
• The Writer process will wait for the ACK, which happens after the 
primary copy, the replicated copies and the journals have all been 
written. 
• On a WAN this might extend latencies unacceptably. 
• Alternatives 
• For S3/Swift systems, federated gateways between CEPH clusters, 
RADOS uses Eventual Consistency. 
• For remote backup use RBD with sync agents and incremental 
snapshots. 
Dell Corporation
Recommended Storage Server Configurations 
CEPH and InkTank recommendations are a bit out of date. 
• CPU – 1 core GHz per OSD 
– so a 2 x 8-core Intel Haswell 2.0GHz server could support 32 OSDs 
– less for AMD 
• Memory – 2GB per OSD 
– Must be ECC 
• Disk Controller – SAS or SATA without extender for data and 
journal, RAID 1 for operating system disks 
• Data Disks – Size doesn’t matter! Rebuilds happen across 
hundreds of placement groups. 
– 12 disks seems a good number 
• Journal Disks – SSDs – write optimised 
Dell Corporation
Intel Processors 
Dell Corporation
Memory Considerations 
C0 C1 C2 C3 
C0 C1 C2 C3 
C4 C5 C6 C7 C4 C5 C6 C7 
• Always populate all channels – groups of 8 
• Anything less loses significant memory bandwidth 
• Speed drops with 3DPC (sometimes 2DPC) 
• Use Dual Rank RDIMMs for maximum performance and expandability 
• Important to PIN process and data to same NUMA node 
• But let OS processes float 
• Or try Hyperthreading 
• Sensible memory is now 64GB (8 x 8GB RDIMMs) 
Dell Corporation
STORAGE NODE LOAD BALANCER x2 
Dell PowerEdge R515 
6 core AMD CPU, 32GB RAM 
2x 300GB SAS drives (OS) 
12x 3TB SATA drives 
2x 10GbE, 1x 1GbE, IPMI 
M 
RADOS GATEWAY 
STORAGE NODE 
DreamObjects Hardware Specs 
STORAGE NODE 
STORAGE NODE 
STORAGE NODE 
STORAGE NODE 
STORAGE NODE 
x4 
x90 
MANAGEMENT NODE x3 
MANAGEMENT NODE 
Dell PowerEdge R415 
2x 1TB SATA 
1x 10GbE 
Dell Corporation
Ceph Gateway Server 
• Gateway does CRC32 and MD5 checksumming 
– Now included in Intel AVX2 on Haswell 
• 64GB memory (minimum sensible) 
• 2 separate 10GbE NICs, 1 for client comms, 1 for store/retrieve 
• Make sure you have enough file handles, default is 100 - you should 
start at 4096! 
• Load balancing with multiple gateways 
Dell Corporation
Ceph Cluster Monitors 
• Best practice to deploy monitor role on dedicated hardware 
– Not resource intensive but critical – Stewards of the cluster 
– Using separate hardware ensures no contention for resources 
• Make sure monitor processes are never starved for resources 
– If running monitor process on shared hardware, fence off resources 
• Deploy an odd number of monitors (3 or 5) 
– Need to have an odd number of monitors for quorum voting 
– Clusters < 200 nodes work well with 3 monitors 
– Larger clusters may benefit from 5 
– Main reason to go to 7 is to have redundancy in fault zones 
• Add redundancy to monitor nodes as appropriate 
– Make sure the monitor nodes are distributed across fault zones 
– Consider refactoring fault zones if needing more than 7 monitors 
– Build in redundant power, cooling, disk 
2 
0 
Dell Corporation
Networking Overview 
• Plan for low latency and high bandwidth 
• Use 10GbE switches within the rack 
• Use 40GbE uplinks between racks in the datacentre 
• Use more bandwidth at the backend compared to the front end 
• Enable Jumbo frames 
• Replication is done by the storage not the client 
• Client writes to primary and journal 
• Primary writes to replicas through back end network 
• Backend also does recovery and rebalancing 
2 
1 
Dell Corporation
Potential Dell Server Hardware Choices 
• Rackable Storage Node 
– Dell PowerEdge R720XD OR new 13g R730/R730xd 
• Bladed Storage Node 
– Dell PowerEdge C8000XD Disk 
and PowerEdge C8220 CPU 
– 2x Xeon E5-2687 CPU, 128GB RAM 
– 2x 400GB SSD drives 
(OS and optionally Journals) 
– 12x 3TB NL SAS drive 
– 2x 10GbE, 1x 1GbE, IPMI 
• Monitor Node 
– Dell PowerEdge R415 
– 2x 1TB SATA 
– 1x 10GbE 
Dell Corporation 
2Confidential 
2
Mixed Use Deployments 
• For simplicity, dedicate hardware to specific role 
– That may not always be practical (e.g., small clusters) 
– If needed, can combine multiple functions on same hardware 
• Multiple Ceph Roles (e.g., OSD+RGW, OSD+MDS, Mon+RGW) 
– Balance IO-intensive with CPU/memory intensive roles 
– If both roles are relatively light (e.g., Mon and RGW) can 
combine 
• Multiple Applications (e.g., OSD+Compute, Mon+Horizon) 
– In OpenStack environment, may need to mix components 
– Follow same logic of balancing IO-intensive with CPU intensive 
2 
3 
Dell Corporation
Super-size CEPH 
• Lots of Disk space 
• CEPH Rules apply 
• Great for cold dark storage 
• Surprisingly popular with 
Customers 
• 3PB raw in a rack! 
R730/R730XD or R720/R720XD 
PowerVault JBOD 
Dell Corporation
Other Design Guidelines 
• Use simple components, don't buy more than you 
need. 
–Save money on RAID, redundant NICs, PS 
and buy more disks 
• Keep networks as flat as possible (East-West) 
–VLANs don't scale 
– Use Software Defined Networking for multi-tenancy in 
cloud 
• Design the fault zones carefully for NoSPoF 
–Rack 
–Row 
–Datacentre 
2 
5 
Dell Corporation
Research Data: 
Beta Slides 
Dell Corporation
Concept: Get started? 
Keep, 
Search, 
Collaborate- 
Publish 
Research Data & Publications 
Digital - Pre-Publication 
(Any Format?) 
Digital -Other (Any Format?) 
Dell Corporation
Concept: Get started? 
Keep, 
Search, 
Collaborate- 
Publish 
Research Data & Publications 
Digital - Pre-Publication 
(Any Format?) 
Digital -Other (Any Format?) 
How tag metadata? 
How Search? 
Data Security? 
File types to store? 
How long to store? 
How Collaborate? 
Dell Corporation
Holding a tin cup below a Niagara Falls of data!" 
Data keeps on 
coming &……. 
..coming……& 
coming……….. 
Has anyone else had this problem and already solved it. ? 
Open Source is best protection/longevity. “Web 2.0/Social has already solved scale-storage 
problem” 
Dell Corporation
Solve problems one at a time 
OpenStack 
Layer 
(Access) 
CEPH Storage 
Identity 
Management 
Governance 
Policy & 
Control 
PUBLISH: 
Existing 
Publishing 
routes 
Dell Corporation
Solve problems one at a time 
OpenStack 
Layer 
(Access) 
CEPH Storage 
Identity 
Management 
Governance 
Policy & 
Control 
Start Here 
PUBLISH: 
Existing 
Publishing 
routes 
Dell Corporation

Más contenido relacionado

La actualidad más candente

ceph optimization on ssd ilsoo byun-short
ceph optimization on ssd ilsoo byun-shortceph optimization on ssd ilsoo byun-short
ceph optimization on ssd ilsoo byun-shortNAVER D2
 
Ceph Object Storage Reference Architecture Performance and Sizing Guide
Ceph Object Storage Reference Architecture Performance and Sizing GuideCeph Object Storage Reference Architecture Performance and Sizing Guide
Ceph Object Storage Reference Architecture Performance and Sizing GuideKaran Singh
 
Ceph Day Beijing - Ceph All-Flash Array Design Based on NUMA Architecture
Ceph Day Beijing - Ceph All-Flash Array Design Based on NUMA ArchitectureCeph Day Beijing - Ceph All-Flash Array Design Based on NUMA Architecture
Ceph Day Beijing - Ceph All-Flash Array Design Based on NUMA ArchitectureDanielle Womboldt
 
Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...
Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...
Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...Odinot Stanislas
 
What you need to know about ceph
What you need to know about cephWhat you need to know about ceph
What you need to know about cephEmma Haruka Iwao
 
Ceph Object Storage Performance Secrets and Ceph Data Lake Solution
Ceph Object Storage Performance Secrets and Ceph Data Lake SolutionCeph Object Storage Performance Secrets and Ceph Data Lake Solution
Ceph Object Storage Performance Secrets and Ceph Data Lake SolutionKaran Singh
 
Revisiting CephFS MDS and mClock QoS Scheduler
Revisiting CephFS MDS and mClock QoS SchedulerRevisiting CephFS MDS and mClock QoS Scheduler
Revisiting CephFS MDS and mClock QoS SchedulerYongseok Oh
 
High-Performance Networking Using eBPF, XDP, and io_uring
High-Performance Networking Using eBPF, XDP, and io_uringHigh-Performance Networking Using eBPF, XDP, and io_uring
High-Performance Networking Using eBPF, XDP, and io_uringScyllaDB
 
NVMe over Fabrics Demystified
NVMe over Fabrics Demystified NVMe over Fabrics Demystified
NVMe over Fabrics Demystified Brad Eckert
 
Storage tiering and erasure coding in Ceph (SCaLE13x)
Storage tiering and erasure coding in Ceph (SCaLE13x)Storage tiering and erasure coding in Ceph (SCaLE13x)
Storage tiering and erasure coding in Ceph (SCaLE13x)Sage Weil
 
Performance optimization for all flash based on aarch64 v2.0
Performance optimization for all flash based on aarch64 v2.0Performance optimization for all flash based on aarch64 v2.0
Performance optimization for all flash based on aarch64 v2.0Ceph Community
 
2021.02 new in Ceph Pacific Dashboard
2021.02 new in Ceph Pacific Dashboard2021.02 new in Ceph Pacific Dashboard
2021.02 new in Ceph Pacific DashboardCeph Community
 
Evolution of MongoDB Replicaset and Its Best Practices
Evolution of MongoDB Replicaset and Its Best PracticesEvolution of MongoDB Replicaset and Its Best Practices
Evolution of MongoDB Replicaset and Its Best PracticesMydbops
 
[발표자료] 오픈소스 Pacemaker 활용한 zabbix 이중화 방안(w/ Zabbix Korea Community)
[발표자료] 오픈소스 Pacemaker 활용한 zabbix 이중화 방안(w/ Zabbix Korea Community) [발표자료] 오픈소스 Pacemaker 활용한 zabbix 이중화 방안(w/ Zabbix Korea Community)
[발표자료] 오픈소스 Pacemaker 활용한 zabbix 이중화 방안(w/ Zabbix Korea Community) 동현 김
 
Meet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike Steenbergen
Meet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike SteenbergenMeet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike Steenbergen
Meet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike Steenbergendistributed matters
 
2019.06.27 Intro to Ceph
2019.06.27 Intro to Ceph2019.06.27 Intro to Ceph
2019.06.27 Intro to CephCeph Community
 
IT Automation with Ansible
IT Automation with AnsibleIT Automation with Ansible
IT Automation with AnsibleRayed Alrashed
 
Introduction of Redis as NoSQL Database
Introduction of Redis as NoSQL DatabaseIntroduction of Redis as NoSQL Database
Introduction of Redis as NoSQL DatabaseAbhijeet Shekhar
 

La actualidad más candente (20)

ceph optimization on ssd ilsoo byun-short
ceph optimization on ssd ilsoo byun-shortceph optimization on ssd ilsoo byun-short
ceph optimization on ssd ilsoo byun-short
 
Ceph issue 해결 사례
Ceph issue 해결 사례Ceph issue 해결 사례
Ceph issue 해결 사례
 
Ceph Object Storage Reference Architecture Performance and Sizing Guide
Ceph Object Storage Reference Architecture Performance and Sizing GuideCeph Object Storage Reference Architecture Performance and Sizing Guide
Ceph Object Storage Reference Architecture Performance and Sizing Guide
 
Ceph Day Beijing - Ceph All-Flash Array Design Based on NUMA Architecture
Ceph Day Beijing - Ceph All-Flash Array Design Based on NUMA ArchitectureCeph Day Beijing - Ceph All-Flash Array Design Based on NUMA Architecture
Ceph Day Beijing - Ceph All-Flash Array Design Based on NUMA Architecture
 
Ceph as software define storage
Ceph as software define storageCeph as software define storage
Ceph as software define storage
 
Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...
Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...
Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...
 
What you need to know about ceph
What you need to know about cephWhat you need to know about ceph
What you need to know about ceph
 
Ceph Object Storage Performance Secrets and Ceph Data Lake Solution
Ceph Object Storage Performance Secrets and Ceph Data Lake SolutionCeph Object Storage Performance Secrets and Ceph Data Lake Solution
Ceph Object Storage Performance Secrets and Ceph Data Lake Solution
 
Revisiting CephFS MDS and mClock QoS Scheduler
Revisiting CephFS MDS and mClock QoS SchedulerRevisiting CephFS MDS and mClock QoS Scheduler
Revisiting CephFS MDS and mClock QoS Scheduler
 
High-Performance Networking Using eBPF, XDP, and io_uring
High-Performance Networking Using eBPF, XDP, and io_uringHigh-Performance Networking Using eBPF, XDP, and io_uring
High-Performance Networking Using eBPF, XDP, and io_uring
 
NVMe over Fabrics Demystified
NVMe over Fabrics Demystified NVMe over Fabrics Demystified
NVMe over Fabrics Demystified
 
Storage tiering and erasure coding in Ceph (SCaLE13x)
Storage tiering and erasure coding in Ceph (SCaLE13x)Storage tiering and erasure coding in Ceph (SCaLE13x)
Storage tiering and erasure coding in Ceph (SCaLE13x)
 
Performance optimization for all flash based on aarch64 v2.0
Performance optimization for all flash based on aarch64 v2.0Performance optimization for all flash based on aarch64 v2.0
Performance optimization for all flash based on aarch64 v2.0
 
2021.02 new in Ceph Pacific Dashboard
2021.02 new in Ceph Pacific Dashboard2021.02 new in Ceph Pacific Dashboard
2021.02 new in Ceph Pacific Dashboard
 
Evolution of MongoDB Replicaset and Its Best Practices
Evolution of MongoDB Replicaset and Its Best PracticesEvolution of MongoDB Replicaset and Its Best Practices
Evolution of MongoDB Replicaset and Its Best Practices
 
[발표자료] 오픈소스 Pacemaker 활용한 zabbix 이중화 방안(w/ Zabbix Korea Community)
[발표자료] 오픈소스 Pacemaker 활용한 zabbix 이중화 방안(w/ Zabbix Korea Community) [발표자료] 오픈소스 Pacemaker 활용한 zabbix 이중화 방안(w/ Zabbix Korea Community)
[발표자료] 오픈소스 Pacemaker 활용한 zabbix 이중화 방안(w/ Zabbix Korea Community)
 
Meet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike Steenbergen
Meet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike SteenbergenMeet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike Steenbergen
Meet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike Steenbergen
 
2019.06.27 Intro to Ceph
2019.06.27 Intro to Ceph2019.06.27 Intro to Ceph
2019.06.27 Intro to Ceph
 
IT Automation with Ansible
IT Automation with AnsibleIT Automation with Ansible
IT Automation with Ansible
 
Introduction of Redis as NoSQL Database
Introduction of Redis as NoSQL DatabaseIntroduction of Redis as NoSQL Database
Introduction of Redis as NoSQL Database
 

Destacado

Rabbit mq, amqp and php
Rabbit mq, amqp and phpRabbit mq, amqp and php
Rabbit mq, amqp and phprodeob
 
Ceph Day Bring Ceph To Enterprise
Ceph Day Bring Ceph To EnterpriseCeph Day Bring Ceph To Enterprise
Ceph Day Bring Ceph To EnterpriseAlex Lau
 
B 8スポンサー講演資料 osnexus steven umbehocker (アファーム・ビジネスパートナーズ株)
B 8スポンサー講演資料 osnexus steven umbehocker (アファーム・ビジネスパートナーズ株)B 8スポンサー講演資料 osnexus steven umbehocker (アファーム・ビジネスパートナーズ株)
B 8スポンサー講演資料 osnexus steven umbehocker (アファーム・ビジネスパートナーズ株)softlayerjp
 
TUT18972: Unleash the power of Ceph across the Data Center
TUT18972: Unleash the power of Ceph across the Data CenterTUT18972: Unleash the power of Ceph across the Data Center
TUT18972: Unleash the power of Ceph across the Data CenterEttore Simone
 
Ceph アーキテクチャ概説
Ceph アーキテクチャ概説Ceph アーキテクチャ概説
Ceph アーキテクチャ概説Emma Haruka Iwao
 
Ceph, Now and Later: Our Plan for Open Unified Cloud Storage
Ceph, Now and Later: Our Plan for Open Unified Cloud StorageCeph, Now and Later: Our Plan for Open Unified Cloud Storage
Ceph, Now and Later: Our Plan for Open Unified Cloud StorageSage Weil
 
BlueStore: a new, faster storage backend for Ceph
BlueStore: a new, faster storage backend for CephBlueStore: a new, faster storage backend for Ceph
BlueStore: a new, faster storage backend for CephSage Weil
 

Destacado (8)

Rabbit mq, amqp and php
Rabbit mq, amqp and phpRabbit mq, amqp and php
Rabbit mq, amqp and php
 
Ceph Day Bring Ceph To Enterprise
Ceph Day Bring Ceph To EnterpriseCeph Day Bring Ceph To Enterprise
Ceph Day Bring Ceph To Enterprise
 
Ceph ベンチマーク
Ceph ベンチマークCeph ベンチマーク
Ceph ベンチマーク
 
B 8スポンサー講演資料 osnexus steven umbehocker (アファーム・ビジネスパートナーズ株)
B 8スポンサー講演資料 osnexus steven umbehocker (アファーム・ビジネスパートナーズ株)B 8スポンサー講演資料 osnexus steven umbehocker (アファーム・ビジネスパートナーズ株)
B 8スポンサー講演資料 osnexus steven umbehocker (アファーム・ビジネスパートナーズ株)
 
TUT18972: Unleash the power of Ceph across the Data Center
TUT18972: Unleash the power of Ceph across the Data CenterTUT18972: Unleash the power of Ceph across the Data Center
TUT18972: Unleash the power of Ceph across the Data Center
 
Ceph アーキテクチャ概説
Ceph アーキテクチャ概説Ceph アーキテクチャ概説
Ceph アーキテクチャ概説
 
Ceph, Now and Later: Our Plan for Open Unified Cloud Storage
Ceph, Now and Later: Our Plan for Open Unified Cloud StorageCeph, Now and Later: Our Plan for Open Unified Cloud Storage
Ceph, Now and Later: Our Plan for Open Unified Cloud Storage
 
BlueStore: a new, faster storage backend for Ceph
BlueStore: a new, faster storage backend for CephBlueStore: a new, faster storage backend for Ceph
BlueStore: a new, faster storage backend for Ceph
 

Similar a Ceph Day London 2014 - Best Practices for Ceph-powered Implementations of Storage as-a-Service

Track B-3 解構大數據架構 - 大數據系統的伺服器與網路資源規劃
Track B-3 解構大數據架構 - 大數據系統的伺服器與網路資源規劃Track B-3 解構大數據架構 - 大數據系統的伺服器與網路資源規劃
Track B-3 解構大數據架構 - 大數據系統的伺服器與網路資源規劃Etu Solution
 
Mike Pittaro - High Performance Hardware for Data Analysis
Mike Pittaro - High Performance Hardware for Data Analysis Mike Pittaro - High Performance Hardware for Data Analysis
Mike Pittaro - High Performance Hardware for Data Analysis PyData
 
High Performance Hardware for Data Analysis
High Performance Hardware for Data AnalysisHigh Performance Hardware for Data Analysis
High Performance Hardware for Data AnalysisMike Pittaro
 
Whd master deck_final
Whd master deck_final Whd master deck_final
Whd master deck_final Juergen Domnik
 
Ceph Community Talk on High-Performance Solid Sate Ceph
Ceph Community Talk on High-Performance Solid Sate Ceph Ceph Community Talk on High-Performance Solid Sate Ceph
Ceph Community Talk on High-Performance Solid Sate Ceph Ceph Community
 
Ceph Deployment with Dell Crowbar - Ceph Day Frankfurt
Ceph Deployment with Dell Crowbar - Ceph Day Frankfurt Ceph Deployment with Dell Crowbar - Ceph Day Frankfurt
Ceph Deployment with Dell Crowbar - Ceph Day Frankfurt Ceph Community
 
Ceph Day New York 2014: Best Practices for Ceph-Powered Implementations of St...
Ceph Day New York 2014: Best Practices for Ceph-Powered Implementations of St...Ceph Day New York 2014: Best Practices for Ceph-Powered Implementations of St...
Ceph Day New York 2014: Best Practices for Ceph-Powered Implementations of St...Ceph Community
 
Backup management with Ceph Storage - Camilo Echevarne, Félix Barbeira
Backup management with Ceph Storage - Camilo Echevarne, Félix BarbeiraBackup management with Ceph Storage - Camilo Echevarne, Félix Barbeira
Backup management with Ceph Storage - Camilo Echevarne, Félix BarbeiraCeph Community
 
Výhody a benefity nasazení Oracle Database Appliance
Výhody a benefity nasazení Oracle Database ApplianceVýhody a benefity nasazení Oracle Database Appliance
Výhody a benefity nasazení Oracle Database ApplianceMarketingArrowECS_CZ
 
High Performance Hardware for Data Analysis
High Performance Hardware for Data AnalysisHigh Performance Hardware for Data Analysis
High Performance Hardware for Data AnalysisMike Pittaro
 
High Performance Hardware for Data Analysis
High Performance Hardware for Data AnalysisHigh Performance Hardware for Data Analysis
High Performance Hardware for Data Analysisodsc
 
Presentation architecting a cloud infrastructure
Presentation   architecting a cloud infrastructurePresentation   architecting a cloud infrastructure
Presentation architecting a cloud infrastructurexKinAnx
 
Presentation architecting a cloud infrastructure
Presentation   architecting a cloud infrastructurePresentation   architecting a cloud infrastructure
Presentation architecting a cloud infrastructuresolarisyourep
 
Optimizing Dell PowerEdge Configurations for Hadoop
Optimizing Dell PowerEdge Configurations for HadoopOptimizing Dell PowerEdge Configurations for Hadoop
Optimizing Dell PowerEdge Configurations for HadoopMike Pittaro
 
Citrix Synergy 2014 - Syn233 Building and operating a Dev Ops cloud: best pra...
Citrix Synergy 2014 - Syn233 Building and operating a Dev Ops cloud: best pra...Citrix Synergy 2014 - Syn233 Building and operating a Dev Ops cloud: best pra...
Citrix Synergy 2014 - Syn233 Building and operating a Dev Ops cloud: best pra...Citrix
 
New Ceph capabilities and Reference Architectures
New Ceph capabilities and Reference ArchitecturesNew Ceph capabilities and Reference Architectures
New Ceph capabilities and Reference ArchitecturesKamesh Pemmaraju
 
Software Defined Storage, Big Data and Ceph - What Is all the Fuss About?
Software Defined Storage, Big Data and Ceph - What Is all the Fuss About?Software Defined Storage, Big Data and Ceph - What Is all the Fuss About?
Software Defined Storage, Big Data and Ceph - What Is all the Fuss About?Red_Hat_Storage
 
SQream DB - Bigger Data On GPUs: Approaches, Challenges, Successes
SQream DB - Bigger Data On GPUs: Approaches, Challenges, SuccessesSQream DB - Bigger Data On GPUs: Approaches, Challenges, Successes
SQream DB - Bigger Data On GPUs: Approaches, Challenges, SuccessesArnon Shimoni
 
Citrix Synergy 2014: Going the CloudPlatform Way
Citrix Synergy 2014: Going the CloudPlatform WayCitrix Synergy 2014: Going the CloudPlatform Way
Citrix Synergy 2014: Going the CloudPlatform WayIliyas Shirol
 
How to Build a Compute Cluster
How to Build a Compute ClusterHow to Build a Compute Cluster
How to Build a Compute ClusterRamsay Key
 

Similar a Ceph Day London 2014 - Best Practices for Ceph-powered Implementations of Storage as-a-Service (20)

Track B-3 解構大數據架構 - 大數據系統的伺服器與網路資源規劃
Track B-3 解構大數據架構 - 大數據系統的伺服器與網路資源規劃Track B-3 解構大數據架構 - 大數據系統的伺服器與網路資源規劃
Track B-3 解構大數據架構 - 大數據系統的伺服器與網路資源規劃
 
Mike Pittaro - High Performance Hardware for Data Analysis
Mike Pittaro - High Performance Hardware for Data Analysis Mike Pittaro - High Performance Hardware for Data Analysis
Mike Pittaro - High Performance Hardware for Data Analysis
 
High Performance Hardware for Data Analysis
High Performance Hardware for Data AnalysisHigh Performance Hardware for Data Analysis
High Performance Hardware for Data Analysis
 
Whd master deck_final
Whd master deck_final Whd master deck_final
Whd master deck_final
 
Ceph Community Talk on High-Performance Solid Sate Ceph
Ceph Community Talk on High-Performance Solid Sate Ceph Ceph Community Talk on High-Performance Solid Sate Ceph
Ceph Community Talk on High-Performance Solid Sate Ceph
 
Ceph Deployment with Dell Crowbar - Ceph Day Frankfurt
Ceph Deployment with Dell Crowbar - Ceph Day Frankfurt Ceph Deployment with Dell Crowbar - Ceph Day Frankfurt
Ceph Deployment with Dell Crowbar - Ceph Day Frankfurt
 
Ceph Day New York 2014: Best Practices for Ceph-Powered Implementations of St...
Ceph Day New York 2014: Best Practices for Ceph-Powered Implementations of St...Ceph Day New York 2014: Best Practices for Ceph-Powered Implementations of St...
Ceph Day New York 2014: Best Practices for Ceph-Powered Implementations of St...
 
Backup management with Ceph Storage - Camilo Echevarne, Félix Barbeira
Backup management with Ceph Storage - Camilo Echevarne, Félix BarbeiraBackup management with Ceph Storage - Camilo Echevarne, Félix Barbeira
Backup management with Ceph Storage - Camilo Echevarne, Félix Barbeira
 
Výhody a benefity nasazení Oracle Database Appliance
Výhody a benefity nasazení Oracle Database ApplianceVýhody a benefity nasazení Oracle Database Appliance
Výhody a benefity nasazení Oracle Database Appliance
 
High Performance Hardware for Data Analysis
High Performance Hardware for Data AnalysisHigh Performance Hardware for Data Analysis
High Performance Hardware for Data Analysis
 
High Performance Hardware for Data Analysis
High Performance Hardware for Data AnalysisHigh Performance Hardware for Data Analysis
High Performance Hardware for Data Analysis
 
Presentation architecting a cloud infrastructure
Presentation   architecting a cloud infrastructurePresentation   architecting a cloud infrastructure
Presentation architecting a cloud infrastructure
 
Presentation architecting a cloud infrastructure
Presentation   architecting a cloud infrastructurePresentation   architecting a cloud infrastructure
Presentation architecting a cloud infrastructure
 
Optimizing Dell PowerEdge Configurations for Hadoop
Optimizing Dell PowerEdge Configurations for HadoopOptimizing Dell PowerEdge Configurations for Hadoop
Optimizing Dell PowerEdge Configurations for Hadoop
 
Citrix Synergy 2014 - Syn233 Building and operating a Dev Ops cloud: best pra...
Citrix Synergy 2014 - Syn233 Building and operating a Dev Ops cloud: best pra...Citrix Synergy 2014 - Syn233 Building and operating a Dev Ops cloud: best pra...
Citrix Synergy 2014 - Syn233 Building and operating a Dev Ops cloud: best pra...
 
New Ceph capabilities and Reference Architectures
New Ceph capabilities and Reference ArchitecturesNew Ceph capabilities and Reference Architectures
New Ceph capabilities and Reference Architectures
 
Software Defined Storage, Big Data and Ceph - What Is all the Fuss About?
Software Defined Storage, Big Data and Ceph - What Is all the Fuss About?Software Defined Storage, Big Data and Ceph - What Is all the Fuss About?
Software Defined Storage, Big Data and Ceph - What Is all the Fuss About?
 
SQream DB - Bigger Data On GPUs: Approaches, Challenges, Successes
SQream DB - Bigger Data On GPUs: Approaches, Challenges, SuccessesSQream DB - Bigger Data On GPUs: Approaches, Challenges, Successes
SQream DB - Bigger Data On GPUs: Approaches, Challenges, Successes
 
Citrix Synergy 2014: Going the CloudPlatform Way
Citrix Synergy 2014: Going the CloudPlatform WayCitrix Synergy 2014: Going the CloudPlatform Way
Citrix Synergy 2014: Going the CloudPlatform Way
 
How to Build a Compute Cluster
How to Build a Compute ClusterHow to Build a Compute Cluster
How to Build a Compute Cluster
 

Último

Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfAnna Loughnan Colquhoun
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceMartin Humpolec
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncObject Automation
 
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdfJamie (Taka) Wang
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?SANGHEE SHIN
 

Último (20)

Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdf
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your Salesforce
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation Inc
 
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?
 

Ceph Day London 2014 - Best Practices for Ceph-powered Implementations of Storage as-a-Service

  • 1. Dell and CEPH Steve Smith: Steve_l_smith@dell.com @SteveSAtDell Paul Brook Paul_brook@dell.com Twitter @PaulBrookAtDell Ceph Day London October 22nd 2014
  • 2. agenda • Why we are here. – we sell CEPH support • You need hardware to sit this on. Here are some ideas • Some best practice shared with CEPH colleagues this year • A concept – (Research Data – would like your input) Dell Corporation
  • 3. Dell is a certified reseller of Red Hat-Inktank Services, Support and Training. • Need to Access and buy Red Hat Services & Support? 15+ Years of Red Hat and Dell • Red Hat 1-year /3-year subscription packages – Inktank Pre-Production subscription – Gold (24*7) Subscription • Red Hat Professional Services – Ceph Pro services Starter Pack – Additional days services options – Ceph Training from Red Hat Or…you can download CEPH for Free Dell Corporation 3Confidential
  • 5. Dell OpenStack Cloud Solution You Get Stuff Stuff Dell Corporation
  • 6. Best Practices (well…….some) With acknowledgement and thanks to Kyle and Mark at InkTank Dell Corporation
  • 7. Planning your Ceph Implementation • Business Requirements – Budget considerations, organisational commitment – Replacing Enterprise SAN/NAS for cost saving – xaaS use cases for massive-scale, cost-effective storage – Avoid lock-in – use open source and industry standards – Steady-state vs. Spike data usage • Sizing requirements – What is the initial storage capacity? – What is the expected growth rate? • Workload requirements – Does the workload need high performance or it is more capacity focused? – What are IOPS/Throughput requirements? – What applications will be running on Ceph cluster? – What type of data will be stored? Dell Corporation
  • 8. Architectural considerations – Redundancy and replication considerations • Tradeoff between Cost vs. Reliability (use-case dependent) • How many node failures can be tolerated? • In a multi-rack scenario, should a whole rack failure be tolerated? • Is there a need for multi-site data replication? • Erasure coding (more capacity with the same raw disk. More CPU load) • Plan for redundancy of the monitor nodes – distribute across fault zones • 3 copies = 8 nines availability, less than 1 second downtime per year • Many many things affect performance - in Ceph, above Ceph and below Ceph. Dell Corporation
  • 9. Understanding Your Workload Dell Corporation
  • 10. CEPH Architecture Refresh Dell Corporation
  • 11. Understanding Ceph (1) Dell Corporation
  • 12. Understanding Ceph (2) Dell Corporation
  • 13. Understanding The Storage Server Dell Corporation
  • 14. Multi-Site Issues • Within a CEPH cluster RADOS enforces Strong Consistency • The Writer process will wait for the ACK, which happens after the primary copy, the replicated copies and the journals have all been written. • On a WAN this might extend latencies unacceptably. • Alternatives • For S3/Swift systems, federated gateways between CEPH clusters, RADOS uses Eventual Consistency. • For remote backup use RBD with sync agents and incremental snapshots. Dell Corporation
  • 15. Recommended Storage Server Configurations CEPH and InkTank recommendations are a bit out of date. • CPU – 1 core GHz per OSD – so a 2 x 8-core Intel Haswell 2.0GHz server could support 32 OSDs – less for AMD • Memory – 2GB per OSD – Must be ECC • Disk Controller – SAS or SATA without extender for data and journal, RAID 1 for operating system disks • Data Disks – Size doesn’t matter! Rebuilds happen across hundreds of placement groups. – 12 disks seems a good number • Journal Disks – SSDs – write optimised Dell Corporation
  • 16. Intel Processors Dell Corporation
  • 17. Memory Considerations C0 C1 C2 C3 C0 C1 C2 C3 C4 C5 C6 C7 C4 C5 C6 C7 • Always populate all channels – groups of 8 • Anything less loses significant memory bandwidth • Speed drops with 3DPC (sometimes 2DPC) • Use Dual Rank RDIMMs for maximum performance and expandability • Important to PIN process and data to same NUMA node • But let OS processes float • Or try Hyperthreading • Sensible memory is now 64GB (8 x 8GB RDIMMs) Dell Corporation
  • 18. STORAGE NODE LOAD BALANCER x2 Dell PowerEdge R515 6 core AMD CPU, 32GB RAM 2x 300GB SAS drives (OS) 12x 3TB SATA drives 2x 10GbE, 1x 1GbE, IPMI M RADOS GATEWAY STORAGE NODE DreamObjects Hardware Specs STORAGE NODE STORAGE NODE STORAGE NODE STORAGE NODE STORAGE NODE x4 x90 MANAGEMENT NODE x3 MANAGEMENT NODE Dell PowerEdge R415 2x 1TB SATA 1x 10GbE Dell Corporation
  • 19. Ceph Gateway Server • Gateway does CRC32 and MD5 checksumming – Now included in Intel AVX2 on Haswell • 64GB memory (minimum sensible) • 2 separate 10GbE NICs, 1 for client comms, 1 for store/retrieve • Make sure you have enough file handles, default is 100 - you should start at 4096! • Load balancing with multiple gateways Dell Corporation
  • 20. Ceph Cluster Monitors • Best practice to deploy monitor role on dedicated hardware – Not resource intensive but critical – Stewards of the cluster – Using separate hardware ensures no contention for resources • Make sure monitor processes are never starved for resources – If running monitor process on shared hardware, fence off resources • Deploy an odd number of monitors (3 or 5) – Need to have an odd number of monitors for quorum voting – Clusters < 200 nodes work well with 3 monitors – Larger clusters may benefit from 5 – Main reason to go to 7 is to have redundancy in fault zones • Add redundancy to monitor nodes as appropriate – Make sure the monitor nodes are distributed across fault zones – Consider refactoring fault zones if needing more than 7 monitors – Build in redundant power, cooling, disk 2 0 Dell Corporation
  • 21. Networking Overview • Plan for low latency and high bandwidth • Use 10GbE switches within the rack • Use 40GbE uplinks between racks in the datacentre • Use more bandwidth at the backend compared to the front end • Enable Jumbo frames • Replication is done by the storage not the client • Client writes to primary and journal • Primary writes to replicas through back end network • Backend also does recovery and rebalancing 2 1 Dell Corporation
  • 22. Potential Dell Server Hardware Choices • Rackable Storage Node – Dell PowerEdge R720XD OR new 13g R730/R730xd • Bladed Storage Node – Dell PowerEdge C8000XD Disk and PowerEdge C8220 CPU – 2x Xeon E5-2687 CPU, 128GB RAM – 2x 400GB SSD drives (OS and optionally Journals) – 12x 3TB NL SAS drive – 2x 10GbE, 1x 1GbE, IPMI • Monitor Node – Dell PowerEdge R415 – 2x 1TB SATA – 1x 10GbE Dell Corporation 2Confidential 2
  • 23. Mixed Use Deployments • For simplicity, dedicate hardware to specific role – That may not always be practical (e.g., small clusters) – If needed, can combine multiple functions on same hardware • Multiple Ceph Roles (e.g., OSD+RGW, OSD+MDS, Mon+RGW) – Balance IO-intensive with CPU/memory intensive roles – If both roles are relatively light (e.g., Mon and RGW) can combine • Multiple Applications (e.g., OSD+Compute, Mon+Horizon) – In OpenStack environment, may need to mix components – Follow same logic of balancing IO-intensive with CPU intensive 2 3 Dell Corporation
  • 24. Super-size CEPH • Lots of Disk space • CEPH Rules apply • Great for cold dark storage • Surprisingly popular with Customers • 3PB raw in a rack! R730/R730XD or R720/R720XD PowerVault JBOD Dell Corporation
  • 25. Other Design Guidelines • Use simple components, don't buy more than you need. –Save money on RAID, redundant NICs, PS and buy more disks • Keep networks as flat as possible (East-West) –VLANs don't scale – Use Software Defined Networking for multi-tenancy in cloud • Design the fault zones carefully for NoSPoF –Rack –Row –Datacentre 2 5 Dell Corporation
  • 26. Research Data: Beta Slides Dell Corporation
  • 27. Concept: Get started? Keep, Search, Collaborate- Publish Research Data & Publications Digital - Pre-Publication (Any Format?) Digital -Other (Any Format?) Dell Corporation
  • 28. Concept: Get started? Keep, Search, Collaborate- Publish Research Data & Publications Digital - Pre-Publication (Any Format?) Digital -Other (Any Format?) How tag metadata? How Search? Data Security? File types to store? How long to store? How Collaborate? Dell Corporation
  • 29. Holding a tin cup below a Niagara Falls of data!" Data keeps on coming &……. ..coming……& coming……….. Has anyone else had this problem and already solved it. ? Open Source is best protection/longevity. “Web 2.0/Social has already solved scale-storage problem” Dell Corporation
  • 30. Solve problems one at a time OpenStack Layer (Access) CEPH Storage Identity Management Governance Policy & Control PUBLISH: Existing Publishing routes Dell Corporation
  • 31. Solve problems one at a time OpenStack Layer (Access) CEPH Storage Identity Management Governance Policy & Control Start Here PUBLISH: Existing Publishing routes Dell Corporation

Notas del editor

  1. Welcome to a short overview of Ceph storage in Dell OpenStack-Powered Cloud Solutions Ceph is a transformational storage technology available as free open source software. It’s a universal storage solution that provides block, file, and object storage from a scalable cluster built on top of standard utility server hardware. Dell has partnered with Inktank, the Ceph experts, to bring a validated Ceph storage solution to Dell cloud customers
  2. Suggested notes: Paul_ We sell Red Hat /Inktank support and training and stuff. If you want it/need it – we can help you get it
  3. Not even the least bit complicated. – But if we are positioning this OUTSIDE CEPH community – what is best way ? Cloud scale-low cost-flexible stoRage -
  4. “Executive Pitch”