How to log messages in Java web app and view logs by session, request, in a tree?

Question

Currently, in my web app logs, I have big text files full of messages. They are difficult to use in debugging and analysis. I'm imagining the following, and I'm wondering, does it already exist, or do I need to build it?

I want to have log messages associated with the current request and session so that I can view a log in a UI as a collapsible tree. At the summary level I might view by user/session and under those see requests with the total time spent in the request. From there I can drill down into a request and see the log messages at deeper levels. Requests that contain warnings or errors would be highlighted or selectable with a filter.

I also want the system to be extensible and not a black box. I want to connect asynchronous tasks to sessions/requests even if those things were happening in other threads. And I want to integrate this with client side Javascript errors. That is another issue, but the point is I need something extensible.

Answer 1

I'm using following pattern:

• My Log4j pattern is using MDC - %d{ISO8601} %p [%X{REQUEST_ID}][%X{SESSION_ID}][%X{USER}] %class %m%n

• In application for every request I'm using MDC.put() method to set REQUEST_ID, SESSION_ID, USER

  MDC.put("SESSIONION_ID", sessionId);
  MDC.put("REQUEST_ID", requestId);
  MDC.put("USER", user);
  

• I have defined this log pattern in OtrosLogViewer. Your log pattern definition should looks like:

  type=log4j
  pattern=TIMESTAMP LEVEL [PROP(REQUEST_ID)][PROP(SESSION_ID)][PROP(USER)] CLASS MESSAGE
  dateFormat=yyyy-MM-dd HH:mm:ss,SSS
  name="MyAppLog"
  charset=UTF-8
  

• I open logs with OtrosLogViewer which allows me to easy filter by property REQUEST_ID, SESSIONS_ID or USER_ID. It can also filter logs by class, date, thread, etc. Logs can be open from remote servers using SFTP, FTP, SMB

• Additionally you can integrate OtrosLogViewer with your Intellij. See example on Youtube: https://www.youtube.com/watch?v=SbOFF56_7-A?hd=1

Disclaimer: I am the author of OtrosLogViewer