2

我一直在使用这个 jobclient.monitorandprintjob() 方法将 map reduce 作业的输出打印到控制台。我的用法是这样的：

job_client.monitorAndPrintJob(job_conf, job_client.getJob(j.getAssignedJobID()))

其输出如下（打印在控制台上）：

13/03/04 07:20:00 INFO mapred.JobClient: Running job: job_201302211725_10139<br>
13/03/04 07:20:01 INFO mapred.JobClient:  map 0% reduce 0%<br>
13/03/04 07:20:08 INFO mapred.JobClient:  map 100% reduce 0%<br>
13/03/04 07:20:13 INFO mapred.JobClient:  map 100% reduce 100%<br>
13/03/04 07:20:13 INFO mapred.JobClient: Job complete: job_201302211725_10139<br>
13/03/04 07:20:13 INFO mapred.JobClient: Counters: 26<br>
13/03/04 07:20:13 INFO mapred.JobClient:   Job Counters<br>
13/03/04 07:20:13 INFO mapred.JobClient:     Launched reduce tasks=1<br>
13/03/04 07:20:13 INFO mapred.JobClient:     Aggregate execution time of mappers(ms)=5539<br>
13/03/04 07:20:13 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0<br>
13/03/04 07:20:13 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0<br>
13/03/04 07:20:13 INFO mapred.JobClient:     Launched map tasks=2<br>
13/03/04 07:20:13 INFO mapred.JobClient:     Data-local map tasks=2<br>
13/03/04 07:20:13 INFO mapred.JobClient:     Aggregate execution time of reducers(ms)=4337<br>
13/03/04 07:20:13 INFO mapred.JobClient:   FileSystemCounters<br>
13/03/04 07:20:13 INFO mapred.JobClient:     MAPRFS_BYTES_READ=583<br>
13/03/04 07:20:13 INFO mapred.JobClient:     MAPRFS_BYTES_WRITTEN=394<br>
13/03/04 07:20:13 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=140219<br>
13/03/04 07:20:13 INFO mapred.JobClient:   Map-Reduce Framework<br>
13/03/04 07:20:13 INFO mapred.JobClient:     Map input records=6<br>
13/03/04 07:20:13 INFO mapred.JobClient:     Reduce shuffle bytes=136<br>
13/03/04 07:20:13 INFO mapred.JobClient:     Spilled Records=22<br>
13/03/04 07:20:13 INFO mapred.JobClient:     Map output bytes=116<br>
13/03/04 07:20:13 INFO mapred.JobClient:     CPU_MILLISECONDS=1320<br>
13/03/04 07:20:13 INFO mapred.JobClient:     Map input bytes=64<br>
13/03/04 07:20:13 INFO mapred.JobClient:     Combine input records=13<br>
13/03/04 07:20:13 INFO mapred.JobClient:     SPLIT_RAW_BYTES=180<br>
13/03/04 07:20:13 INFO mapred.JobClient:     Reduce input records=11<br>
13/03/04 07:20:13 INFO mapred.JobClient:     Reduce input groups=11<br>
13/03/04 07:20:13 INFO mapred.JobClient:     Combine output records=11<br>
13/03/04 07:20:13 INFO mapred.JobClient:     PHYSICAL_MEMORY_BYTES=734961664<br>
13/03/04 07:20:13 INFO mapred.JobClient:     Reduce output records=11<br>
13/03/04 07:20:13 INFO mapred.JobClient:     VIRTUAL_MEMORY_BYTES=9751805952<br>
13/03/04 07:20:13 INFO mapred.JobClient:     Map output records=13<br>
13/03/04 07:20:13 INFO mapred.JobClient:     GC time elapsed (ms)=0<br>

我希望将上述输出/日志打印在文本文件中，而不是控制台中。有什么建议么？

4

1 回答 1

2

在您的 HADOOP_HOME/conf 中，您可能会找到一个名为 :的文件log4j.properties。我相信您可以在那里配置登录的位置和方式。

准确地说，您将使用滚动文件 appender，因此您应从文件中取消注释（只需删除#）以下行log4j.properties：

# Rolling File Appender
#

#log4j.appender.RFA=org.apache.log4j.RollingFileAppender
#log4j.appender.RFA.File=${hadoop.log.dir}/${hadoop.log.file}

# Logfile size and and 30-day backups
#log4j.appender.RFA.MaxFileSize=1MB
#log4j.appender.RFA.MaxBackupIndex=30

#log4j.appender.RFA.layout=org.apache.log4j.PatternLayout
#log4j.appender.RFA.layout.ConversionPattern=%d{ISO8601} %-5p %c{2} - %m%n
#log4j.appender.RFA.layout.ConversionPattern=%d{ISO8601} %-5p %c{2} (%F:%M(%L)) - %m%n

并根据自己的喜好自定义其他参数。

有关 log4j 配置的更多信息，请阅读此处。

于 2013-03-04T15:20:45.630 回答