Spring Boot jetty/tomcat embedded access log configuration

Question

I config logback.xml it work perfectly but logback-access.xml not work.

in maven pom.xml

   <dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-access</artifactId>
  </dependency>

in src/main/resource

logback.xml
logback-access.xml

Is there any way to config access log?

Answer 1

You would have to include the relevant feature in your server container. E.g. for Tomcat add a LogbackValve in an EmbeddedServletContainerCustomizer bean. The TomcatEmbeddedServletContainerFactory has a addContextValves method for this purpose.

Answer 2

For embedded Jetty you can also write this as part of your Spring Boot configuration:

@Bean
public EmbeddedServletContainerFactory jettyConfigBean() {
    JettyEmbeddedServletContainerFactory jef = new JettyEmbeddedServletContainerFactory();
    jef.addServerCustomizers(new JettyServerCustomizer() {
        public void customize(Server server) {
            HandlerCollection handlers = new HandlerCollection();
            for (Handler handler : server.getHandlers()) {
                handlers.addHandler(handler);
            }
            RequestLogHandler reqLogs = new RequestLogHandler();
            NCSARequestLog reqLogImpl = new NCSARequestLog("./logs/access-yyyy_mm_dd.log");
            reqLogImpl.setRetainDays(30);
            reqLogImpl.setAppend(true);
            reqLogImpl.setExtended(false);
            reqLogImpl.setLogTimeZone("GMT");
            reqLogs.setRequestLog(reqLogImpl);
            handlers.addHandler(reqLogs);
            server.setHandler(handlers);

            // For Jetty 9.3+, use the following
            //RequestLogHandler reqLogs = new RequestLogHandler();
            //reqLogs.setServer(server);
            //RequestLogImpl rli = new RequestLogImpl();
            //rli.setResource("/logback-access.xml");
            //rli.setQuiet(false);
            //rli.start();
            //reqLogs.setRequestLog(rli);
            //handlers.addHandler(reqLogs);
            //server.setHandler(handlers);
        }
    });
    return jef;
}

Answer 3

After many hours of trying to get a solution to work with SpringBoot 1.4 + Jetty + Logback-access I have finally found an answer to my woes.

Jetty's API interface changed in v9.3 and Logback-access no longer works.

http://shibboleth.1660669.n2.nabble.com/Jetty-9-3-access-logging-recommended-configuration-td7620755.html

There has been a pull request in the Logback project to get it to work again.

https://github.com/qos-ch/logback/pull/269

There a couple of solutions which are mentioned in the above pull request.

Option 1.

Use the org.eclipse.jetty.server.Slf4jRequestLog implementation to route the logging configuration back to classic Logback.

JettyConfiguration @Bean

RequestLogHandler requestLogsHandler = new RequestLogHandler();
requestLogsHandler.setServer(server);
Slf4jRequestLog log = new Slf4jRequestLog();
log.setLoggerName("com.example.accesslog");
requestLogsHandler.setRequestLog(log);
handlers.addHandler(requestLogsHandler);
server.setHandler(handlers);

logback.xml

<appender name="FILE-ACCESS" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${LOG_PATH}/main.log</file>
    <encoder>
        <!-- You'll have to work this out -->
    </encoder>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
        <fileNamePattern>${LOG_PATH}/main.%d{yyyy-MM-dd}-%i.log
        </fileNamePattern>
        <timeBasedFileNamingAndTriggeringPolicy
                class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
            <maxFileSize>20MB</maxFileSize>
        </timeBasedFileNamingAndTriggeringPolicy>
        <maxHistory>14</maxHistory>
    </rollingPolicy>
</appender>

<logger name="com.example.accesslog">
    <appender-ref ref="FILE-ACCESS" />
</logger>

This works but you lose all the access log specific parameters in the custom PatternLayout available with Logback-access. You probably need to roll your own pattern classes.

Thought this might work but it didn't (or I didn't do it properly).

<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
        <layout class="ch.qos.logback.access.PatternLayout">
            <pattern>%h %l %u [%t] "%r" %s %b "%i{Referer}" "%i{User-Agent}"</pattern>
        </layout>
</encoder>

Option 2

Also alluded to in the pull request above is a fix of the issue which can be used until it gets sorted.

Create a class that adds the missing interface and use that instead of RequestLogImpl.

New class

package com.example.ch.qos.logback.access.jetty;

import ch.qos.logback.access.jetty.RequestLogImpl;
import org.eclipse.jetty.util.component.LifeCycle;

public class LogbackAccessRequestLogImplFix1052 extends RequestLogImpl implements LifeCycle {

}

Jetty Configuration @Bean

RequestLogHandler requestLogs = new RequestLogHandler();
requestLogs.setServer(server);
LogbackAccessRequestLogImplFix1052 rli = new LogbackAccessRequestLogImplFix1052();
rli.setResource("/logback-access.xml");
rli.setQuiet(false);
requestLogs.setRequestLog(rli);
handlers.addHandler(requestLogs);
server.setHandler(handlers);

I tried both and ended up going with option 2 for now as I have spent tooooooooo long on this already. I would prefer option 1 though so I can keep all my logging config in the same file.

Good luck.

Answer 4

This version of programmatically adding a Tomcat valve for logback-access improves on the author's original solution a bit.

Thanks to wacai for that. Here is my version that

• removes the trailing : from ${logback.access.config.path:}
• assumes src/main/resources/logback-access.xml
• removes configuration option to change the name of logback-access.xml
• works in Spring Boot 1.3.3

NOTE: you need logback-access 1.1.6 to load config from resources - automatically searches resources for logback-access.xml.

import ch.qos.logback.access.tomcat.LogbackValve;
import org.apache.catalina.Context;
import org.springframework.boot.context.embedded.ConfigurableEmbeddedServletContainer;
import org.springframework.boot.context.embedded.EmbeddedServletContainerCustomizer;
import org.springframework.boot.context.embedded.tomcat.TomcatContextCustomizer;
import org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainerFactory;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class LogbackAccessEventConfiguration {

    @Bean
    public EmbeddedServletContainerCustomizer containerCustomizer() {

        return new EmbeddedServletContainerCustomizer() {

            @Override
            public void customize(ConfigurableEmbeddedServletContainer container) {
                if (container instanceof TomcatEmbeddedServletContainerFactory) {
                    ((TomcatEmbeddedServletContainerFactory) container)
                            .addContextCustomizers(new TomcatContextCustomizer() {

                                @Override
                                public void customize(Context context) {
                                    LogbackValve logbackValve = new LogbackValve();
                                    logbackValve.setFilename("logback-access.xml");
                                    context.getPipeline().addValve(logbackValve);
                                }
                            });
                }
            }
        };
    }

}

Answer 5

Alternative way is to register Servlet Filter and write to regular log.

To prevent mixing access events with other events disable additivity:

<appender name="ACCESS-LOG"
          class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>./log/evilAccess.log</file>
    <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
        <Pattern>
            %d{yyyy-MM-dd HH:mm:ss} %msg ip=%mdc{ip} session=%mdc{session} user=%mdc{user}%n
        </Pattern>
    </encoder>
    <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
        <fileNamePattern>./log/evilAccess-%d{yyyy-MM-dd}.%i.log</fileNamePattern>
        <maxFileSize>5MB</maxFileSize>
        <maxHistory>30</maxHistory>
    </rollingPolicy>
</appender>

<logger name="com.evil.web.log.MyAccessLogFilter" level="debug" additivity="false">
    <appender-ref ref="ACCESS-LOG" />
</logger>

<logger name="com.evil.web.log.MyAccessLogFilter" level="debug" additivity="false">
    <appender-ref ref="ACCESS-LOG" />
</logger>

LoggingFilter is regular Servlet Filter, which can be easily registered in Spring Boot app via @ServletComponentScan on @Configuration class + @WebFilter on implements javax.servlet.Filter or via bean config:

@Bean
myAccessLogFilter myAccessLogFilter() {
    SaAccessLogFilter filter = new MyAccessLogFilter();
    // filter.setMaxPayloadLength(100);
    return filter;
}

@Bean
FilterRegistrationBean registration() {
    FilterRegistrationBean registration = new FilterRegistrationBean(myAccessLogFilter());
    registration.setOrder(1);
    registration.setEnabled(true);
    return registration;
}

I recommend to use at least GenericFilterBean or better OncePerRequestFilter. Some logging filters already provided by Spring Web inside org.springframework.web.filter package:

• AbstractRequestLoggingFilter
• CommonsRequestLoggingFilter
• ServletContextRequestLoggingFilter

I define own implementation based on OncePerRequestFilter to fill Slf4j MDC context with IP address and other information...

Answer 6

Implementation of the currently accepted answer, cortesy of Ego Slayer, posted as community wiki:

---

I get start from http://spring.io/guides/gs/rest-service/

just create file here src/main/java/hello/MyConfig.java

package hello;

import org.apache.catalina.valves.AccessLogValve;
import org.springframework.boot.context.embedded.ConfigurableEmbeddedServletContainerFactory;
import org.springframework.boot.context.embedded.EmbeddedServletContainerCustomizer;
import org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainerFactory;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import ch.qos.logback.access.tomcat.LogbackValve;

@Configuration
public class MyConfig {

    @Bean
    public EmbeddedServletContainerCustomizer containerCustomizer(){
        return new EmbeddedServletContainerCustomizer() {
            @Override
            public void customize(ConfigurableEmbeddedServletContainerFactory factory) {

                if(factory instanceof TomcatEmbeddedServletContainerFactory){
                    TomcatEmbeddedServletContainerFactory containerFactory = (TomcatEmbeddedServletContainerFactory) factory;

                    LogbackValve  logbackValve = new LogbackValve();
                    logbackValve.setFilename("src/main/resources/logback-access.xml");
                    containerFactory.addContextValves(logbackValve);


                }

            }
        };
    }
}

and add logback-access in maven pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.springframework</groupId>
    <artifactId>gs-rest-service</artifactId>
    <version>0.1.0</version>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>0.5.0.M6</version>
    </parent>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
        </dependency>
        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-access</artifactId>
            <version>1.0.13</version>
        </dependency>
    </dependencies>

    <properties>
        <start-class>hello.Application</start-class>
    </properties>

    <build>
        <plugins>
            <plugin> 
                <artifactId>maven-compiler-plugin</artifactId> 
                <version>2.3.2</version> 
            </plugin>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

    <repositories>
        <repository>
            <id>spring-snapshots</id>
            <url>http://repo.spring.io/libs-snapshot</url>
            <snapshots><enabled>true</enabled></snapshots>
        </repository>
    </repositories>
    <pluginRepositories>
        <pluginRepository>
            <id>spring-snapshots</id>
            <url>http://repo.spring.io/libs-snapshot</url>
            <snapshots><enabled>true</enabled></snapshots>
        </pluginRepository>
    </pluginRepositories>
</project>

Answer 7

In our case we updated a project from SpringBoot 1.3.0 to 1.5.16.RELEASE that comes with Jetty v9.x+ which broke the generation of logback access log as reported earlier in this thread. To fix the issue we did following steps -

1. Added logback-access-spring-boot-starter in dependencies, along with logback-classic and logback-core.
2. Added following code in my JettyConfig

   @Bean
RequestLog makeRequestLog() {
    RequestLog requestLog = new Jetty93RequestLogImpl()
    requestLog.resource = '/logback-access.xml'
    requestLog
}
// Jetty 9.x
private static class Jetty93RequestLogImpl extends RequestLogImpl implements LifeCycle {
}         

Answer 8

An example using Spring boot 2(2.1.4.RELEASE). Works well for me.

@Component
public class JettyCustomizationConfig implements WebServerFactoryCustomizer<ConfigurableJettyWebServerFactory> {

    @Override
    public void customize(ConfigurableJettyWebServerFactory server) {
        server.addServerCustomizers(customJettyServer());
    }

    private JettyServerCustomizer customJettyServer() {
        return server -> {
            HandlerCollection handlers = new HandlerCollection();
            RequestLogHandler requestLogHandler = new RequestLogHandler();
            requestLogHandler.setServer(server);
            RequestLogImpl requestLog = new RequestLogImpl();
            requestLog.setResource("/logback-access.xml");
            requestLog.setQuiet(false);
            requestLog.start();
            requestLogHandler.setRequestLog(requestLog);
            handlers.addHandler(server.getHandler());
            handlers.addHandler(requestLogHandler);
            server.setHandler(handlers);
        };
    }
}

logback-access.xml

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <statusListener class="ch.qos.logback.core.status.OnConsoleStatusListener" />
    <property name="log.path" value="logs" />

    <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>${log.path}/requests/seastar_request_%d{yyyy-MM-dd}.log</fileNamePattern>
        </rollingPolicy>
    </appender>

    <appender-ref ref="FILE"/>
</configuration>

Answer 9

In case it helps anyone:

In addition to configuring the LogbackValve like so

@Bean
public TomcatServletWebServerFactory servletContainer() {
    TomcatServletWebServerFactory tomcat = new TomcatServletWebServerFactory();
    tomcat.addContextValves(new LogbackValve());

    return tomcat;
}

I also had to place logback-access.xml under src/main/resources/conf. If I placed it in src/main/resources it was NOT loaded automatically.

This is using Spring Boot v2.5.2 and logback-access v1.2.5.