Tal y como se indica en los manuales de Cloudera Manager, para testear la instalación se puede lanzar un ejemplo de un programa para estimar el valor de pi en Mapreduce. Esta tarea lleva a cabo una simulación para estimar el valor de Pi basándose en un muestreo de puntos. El programa primero mapea una serie de puntos a unos ficheros, uno por tarea map. Luego calcula una estimación de Pi basándose en esos puntos, en la propia tarea MapReduce:
sudo -u hdfs hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 10 100
Number of Maps = 10
Samples per Map = 100
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Starting Job
14/03/10 18:01:09 INFO client.RMProxy: Connecting to ResourceManager at astwin-H87-HD3/10.42.0.12:8032
14/03/10 18:01:10 INFO input.FileInputFormat: Total input paths to process : 10
14/03/10 18:01:10 INFO mapreduce.JobSubmitter: number of splits:10
14/03/10 18:01:10 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1394469409802_0007
14/03/10 18:01:10 INFO impl.YarnClientImpl: Submitted application application_1394469409802_0007
14/03/10 18:01:10 INFO mapreduce.Job: The url to track the job: http://astwin-H87-HD3:8088/proxy/application_1394469409802_0007/
14/03/10 18:01:10 INFO mapreduce.Job: Running job: job_1394469409802_0007
14/03/10 18:01:18 INFO mapreduce.Job: Job job_1394469409802_0007 running in uber mode : false
14/03/10 18:01:18 INFO mapreduce.Job: map 0% reduce 0%
14/03/10 18:01:22 INFO mapreduce.Job: map 20% reduce 0%
14/03/10 18:01:24 INFO mapreduce.Job: map 30% reduce 0%
14/03/10 18:01:25 INFO mapreduce.Job: map 40% reduce 0%
14/03/10 18:01:26 INFO mapreduce.Job: map 50% reduce 0%
14/03/10 18:01:27 INFO mapreduce.Job: map 60% reduce 0%
14/03/10 18:01:28 INFO mapreduce.Job: map 70% reduce 0%
14/03/10 18:01:29 INFO mapreduce.Job: map 80% reduce 0%
.....
Samples per Map = 100
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Starting Job
14/03/10 18:01:09 INFO client.RMProxy: Connecting to ResourceManager at astwin-H87-HD3/10.42.0.12:8032
14/03/10 18:01:10 INFO input.FileInputFormat: Total input paths to process : 10
14/03/10 18:01:10 INFO mapreduce.JobSubmitter: number of splits:10
14/03/10 18:01:10 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1394469409802_0007
14/03/10 18:01:10 INFO impl.YarnClientImpl: Submitted application application_1394469409802_0007
14/03/10 18:01:10 INFO mapreduce.Job: The url to track the job: http://astwin-H87-HD3:8088/proxy/application_1394469409802_0007/
14/03/10 18:01:10 INFO mapreduce.Job: Running job: job_1394469409802_0007
14/03/10 18:01:18 INFO mapreduce.Job: Job job_1394469409802_0007 running in uber mode : false
14/03/10 18:01:18 INFO mapreduce.Job: map 0% reduce 0%
14/03/10 18:01:22 INFO mapreduce.Job: map 20% reduce 0%
14/03/10 18:01:24 INFO mapreduce.Job: map 30% reduce 0%
14/03/10 18:01:25 INFO mapreduce.Job: map 40% reduce 0%
14/03/10 18:01:26 INFO mapreduce.Job: map 50% reduce 0%
14/03/10 18:01:27 INFO mapreduce.Job: map 60% reduce 0%
14/03/10 18:01:28 INFO mapreduce.Job: map 70% reduce 0%
14/03/10 18:01:29 INFO mapreduce.Job: map 80% reduce 0%
.....
.....
Job Finished in 24.515 seconds
Estimated value of Pi is 3.14800000000000000000
Job Finished in 24.515 seconds
Estimated value of Pi is 3.14800000000000000000
No hay comentarios:
Publicar un comentario