To The Cloud and Back: A Look At Hybrid Analytics
•Descargar como PPTX, PDF•
0 recomendaciones•601 vistas
To The Cloud and Back: A Look At Hybrid Analytics
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Destacado
Destacado (20)
Similar a To The Cloud and Back: A Look At Hybrid Analytics
Similar a To The Cloud and Back: A Look At Hybrid Analytics (20)
Más de DataWorks Summit/Hadoop Summit
Más de DataWorks Summit/Hadoop Summit (20)
Último
Último (20)
To The Cloud and Back: A Look At Hybrid Analytics
- 1. To The Cloud and Back: A Look at Hybrid Analytics Keith Manthey
- 2. © Dell EMC 2016 - All rights reserved2 ITIL-based IT processes Client-server scale-up apps Infrastructure resiliency Traditional CIO challenge: next-generation infrastructures are needed
- 3. © Dell EMC 2016 - All rights reserved3 ITIL-based IT processes Client-server scale-up apps Infrastructure resiliency Cloud-native Coexisting IT paradigms Traditional DevOps based IT processes Distributed scale-out apps Application resiliency CIO challenge: next-generation infrastructures are needed
- 4. © Dell EMC 2016 - All rights reserved4 Traditional Cloud-native Modern IT Off-Premises On-Premises Available both on and off premises
- 6. CLOUD ENABLED STORAGE DATALAKE DATACENTER Through 2020, IDC predicts 4.5x faster spending on cloud based big data analytics then on-premise solutions By 2020, IDC predicts that usage of big data analytics for actionable intelligence will double over today On premise and Cloud use of Hybrid Analytics will become a dominant use of the technology More specifically – Moving data from On-Premise to the Cloud and Back
- 7. The journey to here
- 8. © Dell EMC 2016 - All rights reserved8 Data Management DATA LAKE SOLUTION FOR EDW MODERNISATION Clickstream Web & Social Geolocation Sensor & Machine Server Logs EXISTINGSOURCES ERP CRM Commodity Compute DATA SERVICES OPERATIONAL SERVICES HORTONWORKS DATA PLATFORM HADOOP CORE Business Analytics Visualization & Dashboards IT Applications NEWSOURCES 2 3 1 ETL/ELT OFFLOAD ACTIVE ARCHIVE ENRICH WITH NEW DATA TYPES MULTI-PROTOCOL ACCESS ENTERPRISE-GRADE DATA MANAGEMENT 5 NFS, SMB, HTTP, Swift 1 2 3 4 5 4 New Data Flow Current Data Flow Legend OFFLOAD Isilon
- 9. © Dell EMC 2016 - All rights reserved9 1. Active Archive – Optimise Enterprise Data Warehouse storage by archiving cold data and still analyse it as needed 2. ETL Offload – Improve EDW performance by offloading ETL processing to Hadoop 3. Semi/Unstructured Data Analytics – Increase confidence in business decisions with new data sources 4. Multi-protocol Access – Enable applications to access/update Hadoop data using NFS, SMB, HTTP, Swift and other file/object based access methods 5. Data Management – Enterprise-grade data management at Hadoop economics DATA LAKE BENEFITS Unique to Isilon
- 10. © Dell EMC 2016 - All rights reserved10 ISILON MOMENTUM 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 • >12,000 Clusters in the Field • Approaching 1PB Average Cluster Capacity • 1.3EB of Hadoop Usage • 7,000+ Customers World Wide • 1,300 New Customers in 2015 • #1 in Data Lakes >1,200 HDFS Customers Scale-Out NAS Leader
- 11. A look forward
- 12. Ability to: • Enact Policy Based Data Movement • Migrate data from On-Prem to Cloud and vice versa • Leverage Hybrid Analytics to greatest Effect To The Cloud and Back
- 13. Limits: 100 Gb/Sec 1 Petabyte (1 million gigabytes) ~1 Day to transfer one way To The Cloud and Back
- 14. Data Silos OnPremOnlyfiles CloudOnlyfiles Strong Bandwidth Typically Hand Scripted Expensive Extraction Limit outbound data movement
- 15. Our Research
- 16. Problem Domain 1. File movement automation in place today is labor intensive and fraught with peril 2. Non-prescriptive file movement out of the cloud will be extremely expensive with limited value in return. 3. Most file movements might not return to exactly the same target location. For example: Location 1, File 1 Location 2, File 1 – begets File 2 Location 3, File 2
- 17. What if…. 1. There was a way to move files by policy (i.e., rules based) to various locations, including the cloud and back 1. For Example – only target net new files created in the cloud in certain directories for movement. 2. The rules based file movement could allow for multiple targets and / or destinations for files. 1. For Example – move net new files in one directory to a single argetand net new files in a second directory to three targets.
- 18. OneFS in the Cloud
- 19. Our Journey to Here 1. OneFS already is a software defined asset. 1. SDEdge is a software defined storage offering 2. CloudPools is a software feature in SDEdge that uses the Public Cloud (Azure, AWS, Google) as a back up target 2. OneFS has been loaded and run in the public cloud as noted in the previous slide. 3. Dell EMC has a long history of data mobility across our product suite (replication, backups, etc) 4. Dell EMC has a long history of policy based file movement features.
- 20. Futures 1. We are exploring what OneFS with all its features and abilities to move data up to the cloud and back would look like. 2. What about a cloud environment that contained a OneFS daemon that allowed policy based file movement.? 3. The cloud environment file movement (up and back) could be controlled by a Isilon cluster
- 21. On-Prem Files Movement policies Future State? Cloud Files OneFS Daemon
Notas del editor
- The challenge is: These applications require two distinct architectural paradigms. Traditional applications tend to be built in a vertically scaling manner. As more resources are required the systems on which they run must grow to support the scale-up of the individual app. Because of this design, application ecosystems tend to be contained within the boundary of a single datacenter. This has resulted in a model where resiliency and availability was delivered by the infrastructure itself. Additionally, in order to ensure system wide stability a rich set of checks and balances and change control methodologies emerged and were codified into a set of IT processes known as ITIL. CLICK
- Cloud native Applications, on the other hand, are built on a set of micro-services that scale horizontally rather than vertically. As more resources are required, systems don’t get bigger, but rather 10’s, 100’s or even 1000’s of additional application instances are deployed and these instances can run almost anywhere. Because these applications can run across a variety of datacenters, both on and off prem, organizations are not able to guarantee stability of the infrastructure and therefor the burden of reliability falls to the application itself. As a result new Dev/Ops processes designed to support rapid software iteration and delivery have been developed and are being adopted. CLICK
- So, each of these models are critical to the modern IT ecosystem. And thanks to the ubiquity of highly reliable and cost effective networks, these infrastructure services can be delivered almost anywhere. CLICK
- We’ve extended the data lake to the enterprise edge to address the storage needs of edge and remote locations We’ve also tightly integrated the data lake to public, private and hosted clouds to provide virtually limitless storage capacity to future proof your storage. the data lake now extends beyond the data center to your choice of cloud options
- We’ve extended the data lake to the enterprise edge to address the storage needs of edge and remote locations We’ve also tightly integrated the data lake to public, private and hosted clouds to provide virtually limitless storage capacity to future proof your storage. the data lake now extends beyond the data center to your choice of cloud options