This document discusses using Hadoop to analyze weather data. It analyzes weather data from the National Climatic Data Center to find the maximum temperature for each year. The Hadoop architecture includes HDFS for distributed storage and MapReduce for parallel processing. Weather records are input to mappers and key-value pairs of years and temperatures are output. Reducers take these pairs to find the maximum temperature for each year, such as 280 for 2010. The document concludes Hadoop is well-suited for processing large sensor data sets like weather data stored at NCDC.

1. WEATHER DATA ANALYTICS
USING HADOOP
By,
M.S. Najima Begum,
II B.Sc(Computer Science)
Department of Computer Science,
ANJA College, Sivakasi.

2. About Hadoop
• The Apache Hadoop software library is a framework that
allows for the distributed processing of large data sets across
clusters of computers using simple programming models.
• It is designed to scale up from single servers to thousands of
machines, each offering local computation and storage.

3. Abstract
In this paper, we have used Hadoop platform to analyse
the weather data of various years using MapReduce programs
written in JAVA and we can find the maximum or minimum
temperature every year that can be used for other purposes.

4. The Paper includes these modules:
• Hadoop Common: The common utilities that support the
other Hadoop modules.
• Hadoop Distributed File System (HDFS™): A distributed
file system that provides high-throughput access to application
data.
• Hadoop MapReduce: A YARN-based system for parallel
processing of large data sets.

5. Hadoop Architecture
Hadoop Architecture = HDFS Architecture + MAPREDUCE

6. MapReduce

7. WEATHER DATA ANALYTICS
Weather prediction is the application of technology to predict
the action of the atmosphere for a given location.

8. DATA
• The data we will use is from the National Climatic Data Center
(ncdc, http://www.nc.noah.gov/).
• The data is stored using a line-oriented ASCII format, in which
each line is a record.
• The format supports a rich set of meteorological elements,
many of which are optional or with variable data lengths.

9. FINDING MAXIMUM TEMPERATURE IN
VARIOUS YEARS
• INPUT
0043011990999991950051512004...9999999N9+00221+99999999999...
0043011990999991950051518004...9999999N9-00111+99999999999...
0043012650999991949032412004...0500001N9+01111+99999999999...
0043012650999991949032418004...0500001N9+00781+99999999999...
• INPUT TO MAPPER:
(106,0043011990999992010051512004...9999999N9+00221+99999999999...)
(212,0043011990999991950051518004...9999999N9-00111+99999999999...)
(318,00430126509999920119032412004...0500001N9+01111+99999999999..)
(424,0043012650999991949032418004...0500001N9+00781+99999999999...)

10. INPUT TO REDUCER
The input to the reducer is the output of the mapper
• OUTPUT FROM MAPPER
(2010, 22)
(1950, −11)
(2011, 111)
(1949, 78)

11. WEATHER
DATA
PROCESSING

18. OUTPUT
2010 280
The maximum temperature in the year 2010 is
280.

19. CONCLUSION
we have analysed the weather data using big data
environment. The method used in our paper is Hadoop with
mapreducer which processes the sensor data which is stored in
the National Climatic Data Cementum (NCDC)

21. Any Queries…