Big Data Course Materials delivered by BigDatta HUB team to students in the Egyptians Universities

1. Big Data
INTRODUCTION TO BIG DATA AND HADOOP

2. Hadoop distribution
Hadoop HDFS
Introduction to BIG DATA
Big Data different tools
Introduction to Hadoop
Hadoop MapRedue
Agenda

3. What is BIG DATA?

4. What if you could handle bigdata?

5. Structured, Unstructured?!

6. Big Data problem?

7. BIG DATA Architecture

8. When to say: I have Big Data?!
Big
Data
Volume
Velocity
Variety

9. Internet in just 1 min !!!

10. Big Data Workflow
Big
Data
Volume
Velocity
Variety
Acquire
Store
Analyze
VisualizeManage
Share
Integrate

11. Big Data use case: Unravelling Epidemics with Big Data

12. Big Data use case: Smart Cities

13. Big Data different tools

14. Big Data Analysis Approaches

15. Big Data Analysis Approaches
 batch processing is used for
very large files or where a
fast response time is not
critical. The files to be
transmitted are gathered
over a period and then send
together as a batch.

16. So, Big Data !!!
 The basic idea behind the phrase “Big Data” is that everything we do is
increasingly leaving a digital trace, which we can use and analyse.

17. Traditional Vs. Big Data Analysis
Traditional Approach Big Data Approach
Analyze small
subsets of data
Analyze all data
Analyzed
information
All available information
All available
information
analyzed

18. Introducing Apache
 Hadoop is designed for batch and high throughput
processing jobs.

19. Hadoop Distributed File System HDFS

20. HDFS Architecture Guide

21. Hadoop Map-Reduce

22. Map-Reduce Architecture Guide

23. Word count MapReduce example

24. Hadoop Versions, What’s different?!

25. Hadoop distribution

26. Who uses Hadoop?

27. Practical session
SETTING UP YOUR ENVIRONMENT !

28. Wrap up your Knowledge…
 1- Virtualization

29. How is your Laptop looks like?!
Operating System(Linux-Windows-UNIX)
Hypervisor(VMware-Virtual Box)
VM(Linux) VM(Win10)
App
(Firefox-Word)

30. Prerequisites Software

31. Installing JAVA
sudo apt-get update
sudo apt-get install sun-java6-jdk
Java –version

32. Install & Configure ssh
 sudo apt-get install openssh-server
 ssh-keygen -t rsa -P "“
 #cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

33. Hadoop Download & Install

34. Hadoop Download & Install

35. Hadoop Download & Install

36. Hadoop Download & Install

37. Hadoop Download & Install

38. Hadoop Download & Install

39. Hadoop Download & Install

40. Hadoop Download & Install

41. Hadoop Download & Install
 In the Linux Terminal, Writ:
wget
http://supergsego.com/apache/hadoop/common/hado
op-1.2.1/hadoop-1.2.1-bin.tar.gz
and then hit ENTER.

42. Editing .bashrc file
#gedite ~/.bashrc
Add the following lines at the end of the file:
export HADOOP_PREFIX=/opt/hadoop/
export PATH=$PATH:$HADOOP_PREFIX/bin

43. Hadoop main Installation
 #tar –zxvf hadoop-1.2.1-bin.tar.gz
 #mv /home/hadoop-1.2.1 /opt/hadoop
 #gedite /opt/hadoop/conf/hadoop-env.sh

44. Editing conf/*-site.xml files
 1- “Core-site.xml” File:
 #gedit /opt/hadoop/conf/core-site.xml

45. Editing conf/*-site.xml files
 2-”Mapred-site.xml” File
 #gedit /opt/hadoop/conf/mapred-site.xml

46. Editing conf/*-site.xml files
 3-”hdfs-site.xml” File.
 #gedit /opt/hadoop/conf/hdfs-site.xml

47. Formatting Namenode File System
 #hadoop namenode –format

48. Firing Hadoop Deamons
 #start-all.sh

49. Testing Installation
 Open your browser and type : “ Localhost:50070 ” in url area :

50. Testing Installation
 Localhost:50070

52. Thanks
BIGDATA HUB TEAM