Reading a log file which gets rolled over

Question

I am trying to use a simple program to read from a log file. The code used is as follows:

RandomAccessFile in = new RandomAccessFile("/home/hduser/Documents/Sample.txt", "r");
String line;
while(true) {
if((line = in.readLine()) != null) {
System.out.println(line);
} else {
Thread.sleep(2000); 

The code works well for new lines being added to the log file but it does not replicate the rollover process. i.e. when the content of the log file is cleared I expect the java console to continue reading text from the first line newly written to the log. Could that be possible? What changes need to be made to the existing code to achieve that?

Answer 1

I am sorry... My Bad.. I don't want it to go blank.. I just want the next new line written to the log to be read.

Since what you need is able to read from beginning when you file is cleared, you will need to monitor the length of file and reset the cursor pointer when length of file reduces. You can reset the cursor using seek(..) method.

See code below -

RandomAccessFile in = new RandomAccessFile("/home/hduser/Documents/Sample.txt", "r");
String line;
long length = 0;//used to check the file length
while (true) {
    if(in.length()<length){//new condition to reset position if file length is reduced 
        in.seek(0);
    }
    if ((line = in.readLine()) != null) {
        System.out.println(line);
        length = in.length();
    } else {
        Thread.sleep(2000);
    }
}

Answer 2

At my work I had to deal with the processing of logs that can be rolled over without missing any data. What I do is store a tiny memo file that contains:

• A hash of the first 1024 bytes (or less) of the log (I used SHA-1 or something because it's easy)
• The number of bytes used to generate the hash
• The current file position

I close the log file after processing all lines, or some maximum number of lines, and update the memo file. I sleep for a tiny bit and then open the log file again. This allows me to check whether a rollover has occurred. A rollover is detected when:

1. The current file is smaller than the last file position
2. The hash is not the same

In my case, I can use the hash to find the correct log file, and work backwards to get up to date. Once I know I've picked up where I left off in the correct file, I can continue reading and memoizing my position. I don't know if this is relevant to what you want to do, but maybe that gives you ideas.

If you don't have any persistence requirements, you probably don't need to store any memo files. If your 'rollover' just clears the log and doesn't move it away, you probably don't need to remember any file hashes.

Answer 3

it does not replicate the rollover process. i.e. when the content of the log file is cleared I expect the java console to continue reading text from the first line newly written to the log. Could that be possible?

Struggling with this as well. +1 to @paddy for the hash idea.

Another solution (depending on your operating system) is to use the use the inode of the file although this may only work under unix:

Long inode = (Long)Files.getAttribute(logFile.toPath(), "unix:ino");

This returns the inode of the underlying file-system associated with log-file. If the inode changes then the file is a brand new file. This assumes when the log is rolled over that it is moved aside and the same file is not written over.

To make this work you would record the inode of the file you are reading then check to see if the inode has changed if you haven't gotten any new data in some period of time.