Log exceptions in separate files for each tenant in multi-tenancy architecture

Question

I having an application with multi-tenancy support, that is one server and multiple DB, there will be separate DB for each tenant. All the exceptions thrown in the application will be logged in one single log. The tenantID will be printed along with the exception.

I would like to handle it in separate file, i.e for each tenant a separate log file. This will be helpful in identifying that this exception is caused because of the activity done by a user belonging to a particular tenant. Is there any possibilities for achieving this using custom ObjectRenderer or any other techniques. Thanks in advance.

Answer 1

My suggestion is to create your own Appenders. In custom Appenders you can do any thing you want like separate log file etc..,

Reference : How to create a own Appender in log4j? http://logging.apache.org/log4j/2.x/manual/extending.html

Answer 2

I would suggest you to use logback with MDC.

See documentation at

• https://logback.qos.ch/
• https://logback.qos.ch/manual/
• https://logback.qos.ch/manual/mdc.html

I was using request response filter, where in the request filter I would add the tenantId to MDC context reading the value from request object and remove it from MDC in the response filter.

And in the logback configuration, configured filename as,

<file>${FILE_PATH}/${tenantId}.log</file>

specifying discriminator as,

<discriminator>
    <key>tenantId</key>
    <defaultValue>defaultFileName</defaultValue>
</discriminator>

Here the logback treats tenantId as discriminator and gets it from MDC Context, if it is unavailable it is set to default value.

Below is the complete configuration we are using,

<property name="USER_HOME" value="/home/logs" />

<appender name="FILE-THREAD" class="ch.qos.logback.classic.sift.SiftingAppender">

    <!-- This is MDC value -->
    <!-- This is being set via request response filter via Java code -->
    <discriminator>
        <key>tenantId</key>
        <defaultValue>applyService</defaultValue>
    </discriminator>

    <sift>

        <!-- A standard RollingFileAppender, the log file is based on 'logFileName' at runtime  -->
        <appender name="FILE-${tenantId}"
                  class="ch.qos.logback.core.rolling.RollingFileAppender">
            <file>${USER_HOME}/${tenant}.log</file>

            <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
                <Pattern>
                    %d{yyyy-MM-dd HH:mm:ss} [%level] [%logger:%line] [%mdc{tenant}] [%mdc{username}] - %msg%n
                </Pattern>
            </encoder>

            <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
                <!-- rollover daily -->
                <fileNamePattern>${USER_HOME}/${tenant}-%d{yyyy-MM-dd}.%i.log.zip</fileNamePattern>
                <!-- each file should be at most 10MB, keep 60 days worth of history-->
                <maxFileSize>10MB</maxFileSize>
                <maxHistory>60</maxHistory>
            </rollingPolicy>

        </appender>

    </sift>
</appender>

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <layout class="ch.qos.logback.classic.PatternLayout">
        <Pattern>
            %d{yyyy-MM-dd HH:mm:ss} [%level] [%logger:%line] [%mdc{tenant}] [%mdc{username}] - %msg%n
        </Pattern>
    </layout>
</appender>

<logger name="com.test.client" level="debug" additivity="false">
    <appender-ref ref="FILE-THREAD" />
</logger>

<root level="INFO">
    <appender-ref ref="STDOUT" />
</root>

Java Code:

   @Provider
public class RequestandResponseFilter implements ContainerRequestFilter, ContainerResponseFilter {
    
    @Override
    public void filter(ContainerRequestContext requestContext, ContainerResponseContext responseContext)
            throws IOException {
        
        MDC.remove( "tenantId" );

    }

    @Override
    public void filter(ContainerRequestContext requestContext) throws IOException {
         String reqJsonStr = EMPTY_STRING;
         
         try (BufferedReader buffer = new BufferedReader(new InputStreamReader( requestContext.getEntityStream() ))) {
             reqJsonStr = buffer.lines().collect(Collectors.joining("\n"));
            }
         if( reqJsonStr != null && !reqJsonStr.isEmpty() ) {
             JsonObject reqJson = //convertStringtoJson
             MDC.put("tenantId", reqJson.get("tenantId"));
         }
         
    }
    
}