Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Destacado
Destacado (18)
Similar a hadoop_module
Similar a hadoop_module (20)
Más de Army Public School and College -Faisal
Más de Army Public School and College -Faisal (20)
Último
Último (20)
hadoop_module
- 3. What is Hadoop? Apache Hadoop is an open source software framework for storage and large scale processing of data-sets on clusters of commodity hardware
- 4. Hadoop was created by Doug Cutting and Mike Cafarella in 2005 Named the project after son's toy elephant
- 5. Moving Computation to Data Computation Data
- 6. Scalability at Hadoop’s core!
- 10. Once a year 365 Computers Once a day
- 11. Hourly
- 12. New Approach to Data Keep all data
- 13. New Kinds of Analysis Schema-on read style
- 14. New Kinds of Analysis Schema-on read styleNew analysis
- 15. Small Data & Complex Algorithm Large Data & Simple Algorithm Vs.
- 16. Lecture #2 Apache Framework Hadoop Modules
- 17. Apache Framework Basic Modules Hadoop Common Hadoop Distributed File System (HDFS) Hadoop YARN Hadoop MapReduce
- 18. Apache Framework Basic Modules Hadoop Common Hadoop Distributed File System (HDFS) Hadoop YARN Hadoop MapReduce
- 19. Apache Framework Basic Modules Hadoop Common Hadoop Distributed File System (HDFS) Hadoop YARN Hadoop MapReduce
- 20. Apache Framework Basic Modules Hadoop Common Hadoop Distributed File System (HDFS) Hadoop YARN Hadoop MapReduce
- 26. Lecture #3 Hadoop Distributed File System (HDFS)
- 27. HDFS Hadoop Distributed File System Distributed, scalable, and portable file- system written in Java for the Hadoop framework
- 28. HDFS
- 31. MapReduce Engine Task Tracker Job Tracker
- 32. Apache Hadoop NextGen MapReduce (YARN)
- 34. What is Yarn? • YARN enhances the power of a Hadoop compute cluster Scalability
- 35. What is Yarn? • YARN enhances the power of a Hadoop compute cluster Scalability Improved cluster utilization
- 36. What is Yarn? • YARN enhances the power of a Hadoop compute cluster Scalability MapReduce Compatibility Improved cluster utilization
- 37. What is Yarn? • YARN enhances the power of a Hadoop compute cluster Scalability MapReduce Compatibility Improved cluster utilization Supports Other Workloads
- 38. Lecture #4 The Hadoop “Zoo”
- 40. How to figure out the Zoo??
- 44. Original Google Stack Data Integration
- 48. Facebook’s Version of the Stack Data Integration Coordination Languages Compilers Data Store
- 49. Yahoo’s Version of the Stack Data Integration Coordination Languages Compilers Data Store
- 50. LinkedIn’s Version of the Stack Data Integration Coordination Languages Compilers Data Store
- 51. Cloudera’s Version of the Stack Coordination Languages Compilers Data Integration Data Store
- 54. Apache Sqoop • Tool designed for efficiently transferring bulk data between Apache Hadoop and structured datastores such as relational databases
- 56. HBASE • Column-oriented database management system • Key-value store • Based on Google Big Table • Can hold extremely large data • Dynamic data model • Not a Relational DBMS
- 58. PIG High level programming on top of Hadoop MapReduce
- 59. PIG The language: Pig Latin
- 60. PIG Data analysis problems as data flows
- 61. PIG Originally developed at Yahoo 2006
- 62. Pig for ETL
- 63. Pig for ETL
- 64. Pig for ETL
- 66. Apache Hive • Data warehouse software facilitates querying and managing large datasets residing in distributed storage
- 68. Apache Hive Facilitates querying and managing large datasets in HDFS
- 69. Apache Hive Mechanism to project structure onto this data and query the data using a SQL- like language called HiveQL
- 71. Oozie Workflow scheduler system to manage Apache Hadoop jobs
- 73. Oozie Supports MapReduce, Pig, Apache Hive, and Sqoop, etc.
- 75. Zookeeper Provides operational services for a Hadoop cluster group services
- 76. Zookeeper Centralized service for: maintaining configuration information naming services providing distributed synchronization and providing group services
- 77. Zookeeper Centralized service for: maintaining configuration information
- 78. Zookeeper Centralized service for: maintaining configuration information naming services
- 79. Zookeeper Centralized service for: maintaining configuration information naming services providing distributed synchronization and providing group services
- 80. Flume Distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data
- 83. Impala • Cloudera's open source massively parallel processing (MPP) SQL query engine Apache Hadoop
- 84. Additional Cloudera Hadoop Components Spark The New Paradigm
- 86. Spark Apache Spark™ is a fast and general engine for large-scale data processing
- 87. Spark Benefits Multi-stage in-memory primitives provides performance up to 100 times faster for certain applications
- 88. Spark Benefits Allows user programs to load data into a cluster's memory and query it repeatedly Well-suited to machine learning!!!
- 89. Up Next Tour of the Cloudera’s Quick Start VM