Enviar búsqueda
Cargar
Apache Hadoop 3.0 What's new in YARN and MapReduce
•
Descargar como PPTX, PDF
•
13 recomendaciones
•
7,853 vistas
DataWorks Summit/Hadoop Summit
Seguir
Apache Hadoop 3.0 What's new in YARN and MapReduce
Leer menos
Leer más
Tecnología
Denunciar
Compartir
Denunciar
Compartir
1 de 22
Descargar ahora
Recomendados
Moving towards enterprise ready Hadoop clusters on the cloud
Moving towards enterprise ready Hadoop clusters on the cloud
DataWorks Summit/Hadoop Summit
To The Cloud and Back: A Look At Hybrid Analytics
To The Cloud and Back: A Look At Hybrid Analytics
DataWorks Summit/Hadoop Summit
Apache Hadoop YARN: Present and Future
Apache Hadoop YARN: Present and Future
DataWorks Summit
A New "Sparkitecture" for modernizing your data warehouse
A New "Sparkitecture" for modernizing your data warehouse
DataWorks Summit/Hadoop Summit
Scale-Out Resource Management at Microsoft using Apache YARN
Scale-Out Resource Management at Microsoft using Apache YARN
DataWorks Summit/Hadoop Summit
Streamline Hadoop DevOps with Apache Ambari
Streamline Hadoop DevOps with Apache Ambari
DataWorks Summit/Hadoop Summit
Evolving HDFS to Generalized Storage Subsystem
Evolving HDFS to Generalized Storage Subsystem
DataWorks Summit/Hadoop Summit
Scheduling Policies in YARN
Scheduling Policies in YARN
DataWorks Summit/Hadoop Summit
Recomendados
Moving towards enterprise ready Hadoop clusters on the cloud
Moving towards enterprise ready Hadoop clusters on the cloud
DataWorks Summit/Hadoop Summit
To The Cloud and Back: A Look At Hybrid Analytics
To The Cloud and Back: A Look At Hybrid Analytics
DataWorks Summit/Hadoop Summit
Apache Hadoop YARN: Present and Future
Apache Hadoop YARN: Present and Future
DataWorks Summit
A New "Sparkitecture" for modernizing your data warehouse
A New "Sparkitecture" for modernizing your data warehouse
DataWorks Summit/Hadoop Summit
Scale-Out Resource Management at Microsoft using Apache YARN
Scale-Out Resource Management at Microsoft using Apache YARN
DataWorks Summit/Hadoop Summit
Streamline Hadoop DevOps with Apache Ambari
Streamline Hadoop DevOps with Apache Ambari
DataWorks Summit/Hadoop Summit
Evolving HDFS to Generalized Storage Subsystem
Evolving HDFS to Generalized Storage Subsystem
DataWorks Summit/Hadoop Summit
Scheduling Policies in YARN
Scheduling Policies in YARN
DataWorks Summit/Hadoop Summit
Evolving HDFS to a Generalized Storage Subsystem
Evolving HDFS to a Generalized Storage Subsystem
DataWorks Summit/Hadoop Summit
Scaling HDFS to Manage Billions of Files with Distributed Storage Schemes
Scaling HDFS to Manage Billions of Files with Distributed Storage Schemes
DataWorks Summit
Near Real-Time Network Anomaly Detection and Traffic Analysis using Spark bas...
Near Real-Time Network Anomaly Detection and Traffic Analysis using Spark bas...
DataWorks Summit/Hadoop Summit
Ingest and Stream Processing - What will you choose?
Ingest and Stream Processing - What will you choose?
DataWorks Summit/Hadoop Summit
Real-time Hadoop: The Ideal Messaging System for Hadoop
Real-time Hadoop: The Ideal Messaging System for Hadoop
DataWorks Summit/Hadoop Summit
Backup and Disaster Recovery in Hadoop
Backup and Disaster Recovery in Hadoop
DataWorks Summit/Hadoop Summit
Lessons learned from running Spark on Docker
Lessons learned from running Spark on Docker
DataWorks Summit
YARN Federation
YARN Federation
DataWorks Summit/Hadoop Summit
Hadoop in the Cloud - The what, why and how from the experts
Hadoop in the Cloud - The what, why and how from the experts
DataWorks Summit/Hadoop Summit
IoT:what about data storage?
IoT:what about data storage?
DataWorks Summit/Hadoop Summit
Apache Hadoop YARN: state of the union
Apache Hadoop YARN: state of the union
DataWorks Summit
Practice of large Hadoop cluster in China Mobile
Practice of large Hadoop cluster in China Mobile
DataWorks Summit
How the Internet of Things are Turning the Internet Upside Down
How the Internet of Things are Turning the Internet Upside Down
DataWorks Summit
Cloudy with a Chance of Hadoop - Real World Considerations
Cloudy with a Chance of Hadoop - Real World Considerations
DataWorks Summit/Hadoop Summit
Operationalizing YARN based Hadoop Clusters in the Cloud
Operationalizing YARN based Hadoop Clusters in the Cloud
DataWorks Summit/Hadoop Summit
Spark Uber Development Kit
Spark Uber Development Kit
DataWorks Summit/Hadoop Summit
Hive2.0 sql speed-scale--hadoop-summit-dublin-apr-2016
Hive2.0 sql speed-scale--hadoop-summit-dublin-apr-2016
alanfgates
Migrating your clusters and workloads from Hadoop 2 to Hadoop 3
Migrating your clusters and workloads from Hadoop 2 to Hadoop 3
DataWorks Summit
The state of SQL-on-Hadoop in the Cloud
The state of SQL-on-Hadoop in the Cloud
DataWorks Summit/Hadoop Summit
Deep Learning using Spark and DL4J for fun and profit
Deep Learning using Spark and DL4J for fun and profit
DataWorks Summit/Hadoop Summit
Apache Hadoop YARN: Past, Present and Future
Apache Hadoop YARN: Past, Present and Future
DataWorks Summit/Hadoop Summit
Apache Hadoop YARN: Past, Present and Future
Apache Hadoop YARN: Past, Present and Future
DataWorks Summit/Hadoop Summit
Más contenido relacionado
La actualidad más candente
Evolving HDFS to a Generalized Storage Subsystem
Evolving HDFS to a Generalized Storage Subsystem
DataWorks Summit/Hadoop Summit
Scaling HDFS to Manage Billions of Files with Distributed Storage Schemes
Scaling HDFS to Manage Billions of Files with Distributed Storage Schemes
DataWorks Summit
Near Real-Time Network Anomaly Detection and Traffic Analysis using Spark bas...
Near Real-Time Network Anomaly Detection and Traffic Analysis using Spark bas...
DataWorks Summit/Hadoop Summit
Ingest and Stream Processing - What will you choose?
Ingest and Stream Processing - What will you choose?
DataWorks Summit/Hadoop Summit
Real-time Hadoop: The Ideal Messaging System for Hadoop
Real-time Hadoop: The Ideal Messaging System for Hadoop
DataWorks Summit/Hadoop Summit
Backup and Disaster Recovery in Hadoop
Backup and Disaster Recovery in Hadoop
DataWorks Summit/Hadoop Summit
Lessons learned from running Spark on Docker
Lessons learned from running Spark on Docker
DataWorks Summit
YARN Federation
YARN Federation
DataWorks Summit/Hadoop Summit
Hadoop in the Cloud - The what, why and how from the experts
Hadoop in the Cloud - The what, why and how from the experts
DataWorks Summit/Hadoop Summit
IoT:what about data storage?
IoT:what about data storage?
DataWorks Summit/Hadoop Summit
Apache Hadoop YARN: state of the union
Apache Hadoop YARN: state of the union
DataWorks Summit
Practice of large Hadoop cluster in China Mobile
Practice of large Hadoop cluster in China Mobile
DataWorks Summit
How the Internet of Things are Turning the Internet Upside Down
How the Internet of Things are Turning the Internet Upside Down
DataWorks Summit
Cloudy with a Chance of Hadoop - Real World Considerations
Cloudy with a Chance of Hadoop - Real World Considerations
DataWorks Summit/Hadoop Summit
Operationalizing YARN based Hadoop Clusters in the Cloud
Operationalizing YARN based Hadoop Clusters in the Cloud
DataWorks Summit/Hadoop Summit
Spark Uber Development Kit
Spark Uber Development Kit
DataWorks Summit/Hadoop Summit
Hive2.0 sql speed-scale--hadoop-summit-dublin-apr-2016
Hive2.0 sql speed-scale--hadoop-summit-dublin-apr-2016
alanfgates
Migrating your clusters and workloads from Hadoop 2 to Hadoop 3
Migrating your clusters and workloads from Hadoop 2 to Hadoop 3
DataWorks Summit
The state of SQL-on-Hadoop in the Cloud
The state of SQL-on-Hadoop in the Cloud
DataWorks Summit/Hadoop Summit
Deep Learning using Spark and DL4J for fun and profit
Deep Learning using Spark and DL4J for fun and profit
DataWorks Summit/Hadoop Summit
La actualidad más candente
(20)
Evolving HDFS to a Generalized Storage Subsystem
Evolving HDFS to a Generalized Storage Subsystem
Scaling HDFS to Manage Billions of Files with Distributed Storage Schemes
Scaling HDFS to Manage Billions of Files with Distributed Storage Schemes
Near Real-Time Network Anomaly Detection and Traffic Analysis using Spark bas...
Near Real-Time Network Anomaly Detection and Traffic Analysis using Spark bas...
Ingest and Stream Processing - What will you choose?
Ingest and Stream Processing - What will you choose?
Real-time Hadoop: The Ideal Messaging System for Hadoop
Real-time Hadoop: The Ideal Messaging System for Hadoop
Backup and Disaster Recovery in Hadoop
Backup and Disaster Recovery in Hadoop
Lessons learned from running Spark on Docker
Lessons learned from running Spark on Docker
YARN Federation
YARN Federation
Hadoop in the Cloud - The what, why and how from the experts
Hadoop in the Cloud - The what, why and how from the experts
IoT:what about data storage?
IoT:what about data storage?
Apache Hadoop YARN: state of the union
Apache Hadoop YARN: state of the union
Practice of large Hadoop cluster in China Mobile
Practice of large Hadoop cluster in China Mobile
How the Internet of Things are Turning the Internet Upside Down
How the Internet of Things are Turning the Internet Upside Down
Cloudy with a Chance of Hadoop - Real World Considerations
Cloudy with a Chance of Hadoop - Real World Considerations
Operationalizing YARN based Hadoop Clusters in the Cloud
Operationalizing YARN based Hadoop Clusters in the Cloud
Spark Uber Development Kit
Spark Uber Development Kit
Hive2.0 sql speed-scale--hadoop-summit-dublin-apr-2016
Hive2.0 sql speed-scale--hadoop-summit-dublin-apr-2016
Migrating your clusters and workloads from Hadoop 2 to Hadoop 3
Migrating your clusters and workloads from Hadoop 2 to Hadoop 3
The state of SQL-on-Hadoop in the Cloud
The state of SQL-on-Hadoop in the Cloud
Deep Learning using Spark and DL4J for fun and profit
Deep Learning using Spark and DL4J for fun and profit
Similar a Apache Hadoop 3.0 What's new in YARN and MapReduce
Apache Hadoop YARN: Past, Present and Future
Apache Hadoop YARN: Past, Present and Future
DataWorks Summit/Hadoop Summit
Apache Hadoop YARN: Past, Present and Future
Apache Hadoop YARN: Past, Present and Future
DataWorks Summit/Hadoop Summit
Apache Hadoop YARN: state of the union
Apache Hadoop YARN: state of the union
DataWorks Summit
Dataworks Berlin Summit 18' - Apache hadoop YARN State Of The Union
Dataworks Berlin Summit 18' - Apache hadoop YARN State Of The Union
Wangda Tan
Running Services on YARN
Running Services on YARN
DataWorks Summit/Hadoop Summit
Apache Hadoop 3 updates with migration story
Apache Hadoop 3 updates with migration story
Sunil Govindan
Apache Hadoop YARN: State of the Union
Apache Hadoop YARN: State of the Union
DataWorks Summit
YARN - Past, Present, & Future
YARN - Past, Present, & Future
DataWorks Summit
Apache Hadoop YARN: Past, Present and Future
Apache Hadoop YARN: Past, Present and Future
DataWorks Summit
Apache Hadoop YARN: state of the union - Tokyo
Apache Hadoop YARN: state of the union - Tokyo
DataWorks Summit
Apache Hadoop YARN: state of the union
Apache Hadoop YARN: state of the union
DataWorks Summit
The Enterprise and Connected Data, Trends in the Apache Hadoop Ecosystem by A...
The Enterprise and Connected Data, Trends in the Apache Hadoop Ecosystem by A...
Big Data Spain
Big data spain keynote nov 2016
Big data spain keynote nov 2016
alanfgates
Get Started Building YARN Applications
Get Started Building YARN Applications
Hortonworks
Hadoop Summit San Jose 2015: YARN - Past, Present and Future
Hadoop Summit San Jose 2015: YARN - Past, Present and Future
Vinod Kumar Vavilapalli
Hadoop & cloud storage object store integration in production (final)
Hadoop & cloud storage object store integration in production (final)
Chris Nauroth
Bikas saha:the next generation of hadoop– hadoop 2 and yarn
Bikas saha:the next generation of hadoop– hadoop 2 and yarn
hdhappy001
YARN - Hadoop Next Generation Compute Platform
YARN - Hadoop Next Generation Compute Platform
Bikas Saha
Apache Hadoop YARN: Understanding the Data Operating System of Hadoop
Apache Hadoop YARN: Understanding the Data Operating System of Hadoop
Hortonworks
Hadoop & Cloud Storage: Object Store Integration in Production
Hadoop & Cloud Storage: Object Store Integration in Production
DataWorks Summit/Hadoop Summit
Similar a Apache Hadoop 3.0 What's new in YARN and MapReduce
(20)
Apache Hadoop YARN: Past, Present and Future
Apache Hadoop YARN: Past, Present and Future
Apache Hadoop YARN: Past, Present and Future
Apache Hadoop YARN: Past, Present and Future
Apache Hadoop YARN: state of the union
Apache Hadoop YARN: state of the union
Dataworks Berlin Summit 18' - Apache hadoop YARN State Of The Union
Dataworks Berlin Summit 18' - Apache hadoop YARN State Of The Union
Running Services on YARN
Running Services on YARN
Apache Hadoop 3 updates with migration story
Apache Hadoop 3 updates with migration story
Apache Hadoop YARN: State of the Union
Apache Hadoop YARN: State of the Union
YARN - Past, Present, & Future
YARN - Past, Present, & Future
Apache Hadoop YARN: Past, Present and Future
Apache Hadoop YARN: Past, Present and Future
Apache Hadoop YARN: state of the union - Tokyo
Apache Hadoop YARN: state of the union - Tokyo
Apache Hadoop YARN: state of the union
Apache Hadoop YARN: state of the union
The Enterprise and Connected Data, Trends in the Apache Hadoop Ecosystem by A...
The Enterprise and Connected Data, Trends in the Apache Hadoop Ecosystem by A...
Big data spain keynote nov 2016
Big data spain keynote nov 2016
Get Started Building YARN Applications
Get Started Building YARN Applications
Hadoop Summit San Jose 2015: YARN - Past, Present and Future
Hadoop Summit San Jose 2015: YARN - Past, Present and Future
Hadoop & cloud storage object store integration in production (final)
Hadoop & cloud storage object store integration in production (final)
Bikas saha:the next generation of hadoop– hadoop 2 and yarn
Bikas saha:the next generation of hadoop– hadoop 2 and yarn
YARN - Hadoop Next Generation Compute Platform
YARN - Hadoop Next Generation Compute Platform
Apache Hadoop YARN: Understanding the Data Operating System of Hadoop
Apache Hadoop YARN: Understanding the Data Operating System of Hadoop
Hadoop & Cloud Storage: Object Store Integration in Production
Hadoop & Cloud Storage: Object Store Integration in Production
Más de DataWorks Summit/Hadoop Summit
Running Apache Spark & Apache Zeppelin in Production
Running Apache Spark & Apache Zeppelin in Production
DataWorks Summit/Hadoop Summit
State of Security: Apache Spark & Apache Zeppelin
State of Security: Apache Spark & Apache Zeppelin
DataWorks Summit/Hadoop Summit
Unleashing the Power of Apache Atlas with Apache Ranger
Unleashing the Power of Apache Atlas with Apache Ranger
DataWorks Summit/Hadoop Summit
Enabling Digital Diagnostics with a Data Science Platform
Enabling Digital Diagnostics with a Data Science Platform
DataWorks Summit/Hadoop Summit
Revolutionize Text Mining with Spark and Zeppelin
Revolutionize Text Mining with Spark and Zeppelin
DataWorks Summit/Hadoop Summit
Double Your Hadoop Performance with Hortonworks SmartSense
Double Your Hadoop Performance with Hortonworks SmartSense
DataWorks Summit/Hadoop Summit
Hadoop Crash Course
Hadoop Crash Course
DataWorks Summit/Hadoop Summit
Data Science Crash Course
Data Science Crash Course
DataWorks Summit/Hadoop Summit
Apache Spark Crash Course
Apache Spark Crash Course
DataWorks Summit/Hadoop Summit
Dataflow with Apache NiFi
Dataflow with Apache NiFi
DataWorks Summit/Hadoop Summit
Schema Registry - Set you Data Free
Schema Registry - Set you Data Free
DataWorks Summit/Hadoop Summit
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
DataWorks Summit/Hadoop Summit
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
DataWorks Summit/Hadoop Summit
Mool - Automated Log Analysis using Data Science and ML
Mool - Automated Log Analysis using Data Science and ML
DataWorks Summit/Hadoop Summit
How Hadoop Makes the Natixis Pack More Efficient
How Hadoop Makes the Natixis Pack More Efficient
DataWorks Summit/Hadoop Summit
HBase in Practice
HBase in Practice
DataWorks Summit/Hadoop Summit
The Challenge of Driving Business Value from the Analytics of Things (AOT)
The Challenge of Driving Business Value from the Analytics of Things (AOT)
DataWorks Summit/Hadoop Summit
Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
DataWorks Summit/Hadoop Summit
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
DataWorks Summit/Hadoop Summit
Scaling HDFS to Manage Billions of Files with Distributed Storage Schemes
Scaling HDFS to Manage Billions of Files with Distributed Storage Schemes
DataWorks Summit/Hadoop Summit
Más de DataWorks Summit/Hadoop Summit
(20)
Running Apache Spark & Apache Zeppelin in Production
Running Apache Spark & Apache Zeppelin in Production
State of Security: Apache Spark & Apache Zeppelin
State of Security: Apache Spark & Apache Zeppelin
Unleashing the Power of Apache Atlas with Apache Ranger
Unleashing the Power of Apache Atlas with Apache Ranger
Enabling Digital Diagnostics with a Data Science Platform
Enabling Digital Diagnostics with a Data Science Platform
Revolutionize Text Mining with Spark and Zeppelin
Revolutionize Text Mining with Spark and Zeppelin
Double Your Hadoop Performance with Hortonworks SmartSense
Double Your Hadoop Performance with Hortonworks SmartSense
Hadoop Crash Course
Hadoop Crash Course
Data Science Crash Course
Data Science Crash Course
Apache Spark Crash Course
Apache Spark Crash Course
Dataflow with Apache NiFi
Dataflow with Apache NiFi
Schema Registry - Set you Data Free
Schema Registry - Set you Data Free
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
Mool - Automated Log Analysis using Data Science and ML
Mool - Automated Log Analysis using Data Science and ML
How Hadoop Makes the Natixis Pack More Efficient
How Hadoop Makes the Natixis Pack More Efficient
HBase in Practice
HBase in Practice
The Challenge of Driving Business Value from the Analytics of Things (AOT)
The Challenge of Driving Business Value from the Analytics of Things (AOT)
Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
Scaling HDFS to Manage Billions of Files with Distributed Storage Schemes
Scaling HDFS to Manage Billions of Files with Distributed Storage Schemes
Último
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
LoriGlavin3
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
LoriGlavin3
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
Mydbops
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
LoriGlavin3
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
LoriGlavin3
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
AliaaTarek5
How to write a Business Continuity Plan
How to write a Business Continuity Plan
Databarracks
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demo
HarshalMandlekar2
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
BookNet Canada
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
Wes McKinney
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
ThousandEyes
Manual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance Audit
Skynet Technologies
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
panagenda
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
Scott Andery
A Framework for Development in the AI Age
A Framework for Development in the AI Age
Cprime
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Pim van der Noll
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
LoriGlavin3
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
BookNet Canada
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
Farhan Tariq
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Mark Goldstein
Último
(20)
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
How to write a Business Continuity Plan
How to write a Business Continuity Plan
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demo
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Manual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance Audit
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
A Framework for Development in the AI Age
A Framework for Development in the AI Age
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Apache Hadoop 3.0 What's new in YARN and MapReduce
1.
1 © Hortonworks
Inc. 2011 – 2016. All Rights Reserved Apache Hadoop 3.0: What’s new in YARN & MapReduce Tokyo, Oct.26 2016 Junping Du junping_du@apache.org
2.
2 © Hortonworks
Inc. 2011 – 2016. All Rights Reserved About Speakers ⬢ Junping Du – Apache Hadoop Committer & PMC member – Lead Software Engineer @ Hortonworks YARN Core Team – 10+ years for developing enterprise software (5+ years for being “Hadooper”)
3.
3 © Hortonworks
Inc. 2011 – 2016. All Rights Reserved Agenda ⬢ Evolutions in YARN & MR (Done and In Progress) ⬢ Timeline Estimation for Apache Hadoop 3.0 Release
4.
4 © Hortonworks
Inc. 2011 – 2016. All Rights Reserved First, A bit of Vision… ⬢ Evolution of Hadoop start with YARN ⬢ YARN Evolution will continue to drive Hadoop forward Hadoop 3
5.
5 © Hortonworks
Inc. 2011 – 2016. All Rights Reserved Several important trends in age of Hadoop 3.0 + YARN and Other Platform Services Storage Resource Management Security Service Discovery Management Monitoring Alerts IOT Assembly Kafka Storm HBase Solr Governance MR Tez Spark … Innovating frameworks: Flink, DL(TensorFlow), etc. Various Environments On Premise Private Cloud Public Cloud
6.
6 © Hortonworks
Inc. 2011 – 2016. All Rights Reserved Evolutions in YARN & MR ⬢ Re-architecture for YARN Timeline Service - ATS v2 ⬢ Service Native Support in YARN ⬢ YARN Scheduling Enhancements ⬢ More Cloud Friendly ⬢ Better User Experiences ⬢ Other Enhancements
7.
7 © Hortonworks
Inc. 2011 – 2016. All Rights Reserved Timeline Service Revolution – ATS v2 ⬢ Why ATS v2? – Scalability & Performance To get rid of v1 limitation: •Single global instance of writer/reader •Local disk based LevelDB storage – Usability •Handle flows as first-class concepts and model aggregation •Add configuration and metrics as first-class members •Better support for queries – Reliability v1 limitation: •Data is stored in a local disk •Single point of failure (SPOF) for timeline server – Flexibility •Data model is more describable •Extended to more specific info to app
8.
8 © Hortonworks
Inc. 2011 – 2016. All Rights Reserved Core Design for ATS v2 ⬢ Distributed write path – Logical per app collector + physical per node writer – Collector/Writer launched as an auxiliary service in NM. – Standalone writers will be added later. ⬢ Pluggable backend storage – Built in with a scalable and reliable implementation (HBase) ⬢ Enhanced data model – Entity (bi-directional relation) with flow, queue, etc. – Configuration, Metric, Event, etc. ⬢ Separate reader instances ⬢ Aggregation & Accumulation – Aggregation: rolling up the metric values to the parent •Online aggregation for apps and flow runs •Offline aggregation for users, flows and queues – Accumulation: rolling up the metric values across time interval •Accumulated resource consumption for app, flow, etc.
9.
9 © Hortonworks
Inc. 2011 – 2016. All Rights Reserved ATS v2 Architecture Resource Manager RMApp NodeManager Info of Collectors { app_1, app_2, …. } app_1 AM Syncapp_1 Collector app_n Collector Aux Service AM timeline info Timeline Writer RM app Events NM Collector Service Timeline Writer NM_n … NM_1 app_1 container NM Collector Service Sync Container Monitor 1 1Timeline Reader User Queries Container metric info HBase container info (to be added)
10.
1 0 © Hortonworks Inc.
2011 – 2016. All Rights Reserved Data Model in ATS v2 Entity ID + Type Configurations Metadata(Info) Parent-Child Relationships Metrics Events Metric ID Metadata Single Value or Time Series(with timestamps) Cluster Type Cluster Attributes Flow Type User Flow Runs Flow Attributes Flow Run Type User Running apps Flow Run Attributes Application Type User Flow + Run Queue Attempts Attempt Type Application Queue Containers Container Type Attempt Attributes Entities of first class citizens User Username(ID) Aggregated metrics Queue Queue(ID) Sub queues Aggregated metrics Aggregation Event ID Metadata Timestamp
11.
1 1 © Hortonworks Inc.
2011 – 2016. All Rights Reserved Status for ATS v2 ⬢ For other details, like: – Aggregations (app/flow/user/queue level, offline or online) – HBase table schema for EntityTable, ApplicationTable, FlowRunTable, etc. – Reader APIs (RESTful) Please refer to previous talks in Hadoop Summit 2016 San Jose: https://www.youtube.com/watch?v=adV-DFa-8us&index=6&list=PLKnYDs_-dq16K1NH83Bke2dGGUO3YKZ5b ⬢ Status –Phase I (YARN-2928): already released as an alpha feature in 3.0.0-alpha1 –Phase II (YARN-5355): In progress
12.
1 2 © Hortonworks Inc.
2011 – 2016. All Rights Reserved Native Service Support in YARN A native YARN framework. YARN-4692 – Abstract common Framework (Similar to Slider) to support long running service – More simplified API Better support for long running service – Recognition of long running service • Affect the policy of preemption, container reservation, etc. – Auto-restart of containers • Containers in long running service are more stateful – Service/application upgrade support • More services are expected to run long enough to across versions – Dynamic container configuration • Only reserve resource for necessary moment
13.
1 3 © Hortonworks Inc.
2011 – 2016. All Rights Reserved API Simplification - REST Existing APIs are too low level and not easy to work with. Simple REST API layer fronting YARN – YARN-4793. Simplified API layer for services and beyond Create and manage lifecycle of YARN services. Example: ZooKeeper App
14.
1 4 © Hortonworks Inc.
2011 – 2016. All Rights Reserved Discovery services in YARN YARN Service Discovery via DNS: YARN-4757 – Expose existing service information in YARN registry via DNS • Current YARN service registry’s records will be converted into DNS entries – Enabling Container to IP mappings - enables discovery of the IPs of containers via standard DNS lookups. • Application – zkapp1.user1.yarncluster.com -> 192.168.10.11:8080 • Container – container-1454001598828-0001-01-00004.yarncluster.com -> 192.168.10.18
15.
1 5 © Hortonworks Inc.
2011 – 2016. All Rights Reserved More Cloud Friendly ⬢ Elastic –Dynamic Resource Configuration •YARN-291 •Allow tune down/up on NM’s resource in runtime –Graceful decommissioning of NodeManagers •YARN-914 •Drains a node that’s being decommissioned to allow running containers to finish ⬢ Efficient –Support for container resizing •YARN-1197 •Allows applications to change the size of an existing container –Task level native optimization •MAPREDUCE-2841
16.
1 6 © Hortonworks Inc.
2011 – 2016. All Rights Reserved More Cloud Friendly (Contd.) ⬢ Isolation –Embrace container technology to achieve better isolation –Resource isolation support for disk and network •YARN-2619 (disk), YARN-2140 (network) •Containers get a fair share of disk and network resources using Cgroups –Docker support in LinuxContainerExecutor •YARN-3611 •Support to launch Docker containers alongside process •Packaging and resource isolation ⬢ Operation –Container upgrades (YARN-4726) •”Do an upgrade of my Spark / HBase apps with minimal impact to end-users” –AM Restart With Work Preserving •MAPREDUCE-6608
17.
1 7 © Hortonworks Inc.
2011 – 2016. All Rights Reserved Scheduling Enhancements Application priorities: YARN-1963 – Inner-queue priority support Affinity / anti-affinity: YARN-1042 – More restraints on locations Global Scheduling: YARN-5139 – Get rid of per node scheduling model – Enhance container scheduling throughput
18.
1 8 © Hortonworks Inc.
2011 – 2016. All Rights Reserved Operational and User Experience Enhancements (YARN-3368)
19.
1 9 © Hortonworks Inc.
2011 – 2016. All Rights Reserved Other YARN work could get released in Hadoop 3.X ⬢ Resource profiles –YARN-3926 –Users can specify resource profile name instead of individual resources –Resource types read via a config file ⬢ YARN federation –YARN-2915 –Allows YARN to scale out to tens of thousands of nodes –Cluster of clusters which appear as a single cluster to an end user ⬢ Gang Scheduling –YARN-624 More Details in tomorrow noon session “Apache Hadoop YARN: Past, Present and Future” by Junping Du and Jian He
20.
2 0 © Hortonworks Inc.
2011 – 2016. All Rights Reserved Release Timeline for Apache Hadoop 3.0 ⬢ 3.0.0-alpha1 is released on Sep/3/2016 ⬢ alpha2 in Q4. 2016 (Estimated) ⬢ beta1 in early Q1. 2017 (Estimated) ⬢ GA in Q1/Q2 2017 (Estimated)
21.
2 1 © Hortonworks Inc.
2011 – 2016. All Rights Reserved HDP Evolution with Apache Hadoop and YARN
22.
2 2 © Hortonworks Inc.
2011 – 2016. All Rights Reserved2 2 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Thank you!
Descargar ahora