39. ✓Map:cat / Reduce:wc
✓HDFS
$ hadoop dfs -copyFromLocal /usr/local/hadoop/*.txt /dfs/test/input/
$ hadoop jar /usr/local/hadoop/contrib/streaming/hadoop-0.20.1-streaming.jar -input /dfs/test/input/ -output /dfs/test/
output -mapper cat -reducer "wc -l"
09/09/26 17:00:29 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
09/09/26 17:00:30 WARN mapred.JobClient: No job jar file set. User classes may not be found. See
JobConf(Class) or JobConf#setJar(String).
09/09/26 17:00:30 INFO mapred.FileInputFormat: Total input paths to process : 4
09/09/26 17:00:31 INFO mapred.FileInputFormat: Total input paths to process : 4
09/09/26 17:00:33 INFO streaming.StreamJob: map 0% reduce 0%
09/09/26 17:00:42 INFO streaming.StreamJob: map 100% reduce 0%
09/09/26 17:00:44 INFO streaming.StreamJob: map 100% reduce 100%
09/09/26 17:00:44 INFO streaming.StreamJob: Output: /dfs/test/output
$ hadoop dfs -cat /dfs/test/output/*
8842
40. ✓python
http://www.michael-noll.com/wiki/Writing_An_Hadoop_MapReduce_Program_In_Python
$ hadoop jar /usr/local/hadoop/contrib/streaming/hadoop-0.20.1-streaming.jar -input /dfs/test/input/ -output /dfs/test/
output -mapper "python map.py" -reducer "python reduce.py"
09/09/26 17:29:25 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
09/09/26 17:29:26 WARN mapred.JobClient: No job jar file set. User classes may not be found. See
JobConf(Class) or JobConf#setJar(String).
09/09/26 17:29:26 INFO mapred.FileInputFormat: Total input paths to process : 4
09/09/26 17:29:26 INFO mapred.FileInputFormat: Total input paths to process : 4
09/09/26 17:29:30 INFO streaming.StreamJob: map 0% reduce 0%
09/09/26 17:29:33 INFO streaming.StreamJob: map 100% reduce 0%
09/09/26 17:29:35 INFO streaming.StreamJob: map 100% reduce 100%
09/09/26 17:29:35 INFO streaming.StreamJob: Output: /dfs/test/output
$ hadoop dfs -cat /dfs/test/output/*
via 1942
the 1476
to 1394
in 819
a 816
cutting) 740