yarn [hadoop 2.2] where is the location mapper or reducer log outputed?

Question

I want to check the log mapper or reducer output ？I can not find it in syslog under container foler?So where is the log outputing?

        public class SkipStat {
private static Log log = LogFactory.getLog(SkipStat.class);

private static BlockWorkerRepository blockWorkerRepository;
static {
    blockWorkerRepository = new BlockWorkerRepositoryImpl();
}

private static class SkipInfoMapper extends Mapper<Object, BSONObject, Text, AssignmentWritable> {

    private final String invalidResult = "^";

    private static final Calendar currentCalendar = Calendar.getInstance();
    static {
        currentCalendar.add(Calendar.HOUR, -24);
    }

    protected void map(Object key, BSONObject value, Context context) throws IOException, InterruptedException {
        String result = (String) value.get("result");
        log.info("lol... get one result " + result); // LOG ...
        if (invalidResult.equals(result)) {

Answer 1

The historyserver is collecting those logs and keeping them. You can view them using the historyserver WebUI or using yarn logs CLI. See Simplifying user-logs management and access in YARN. Before they're uploaded the logs are:

Logs for all the containers belonging to a single Application and that ran on a given NM are aggregated and written out to a single (possibly compressed) log file at a configured location in the FS

The ApplicationMasterUI will show current executing application logs.

Answer 2

To enable log aggregation, which then allows you to run the "yarn logs" command to see the logs for an application id, set these in your yarn-site.xml:

yarn.log-aggregation-enable
yarn.log-aggregation.retain-seconds
yarn.log-aggregation.retain-check-interval-seconds

It seemed necessary to restart yarn after making this change.

Note the command that was referenced in a comment will work after aggregation has been enabled:

yarn logs -applicationId application_1397033985470_0006

See: http://hadoop.apache.org/docs/r2.2.0/hadoop-yarn/hadoop-yarn-common/yarn-default.xml

Descriptions for these parameters are available in the 2.3.0 documentation: http://hadoop.apache.org/docs/r2.3.0/hadoop-yarn/hadoop-yarn-common/yarn-default.xml