Rolling file implementation

Question

I am always curious how a rolling file is implemented in logs.

How would one even start creating a file writing class in any language in order to ensure that the file size is not exceeded.

The only possible solution I can think of is this:

write method:
    size = file size + size of string to write
    if(size > limit)
        close the file writer
        open file reader
        read the file
        close file reader
        open file writer (clears the whole file)
        remove the size from the beginning to accommodate for new string to write
        write the new truncated string
    write the string we received

This seems like a terrible implementation, but I can not think up of anything better.

Specifically I would love to see a solution in java.

EDIT: By remove size from the beginning is, let's say I have 20 byte string (which is the limit), I want to write another 3 byte string, therefore I remove 3 bytes from the beginning, and am left with end 17 bytes, and by appending the new string I have 20 bytes.

Answer 1

Because your question made me look into it, here's an example from the logback logging framework. The RollingfileAppender#rollover() method looks like this:

public void rollover() {
    synchronized (lock) {
        // Note: This method needs to be synchronized because it needs exclusive
        // access while it closes and then re-opens the target file.
        //
        // make sure to close the hereto active log file! Renaming under windows
        // does not work for open files
        this.closeOutputStream();

        try {
            rollingPolicy.rollover(); // this actually does the renaming of files
        } catch (RolloverFailure rf) {
            addWarn("RolloverFailure occurred. Deferring roll-over.");
            // we failed to roll-over, let us not truncate and risk data loss
            this.append = true;
        }

        try {
            // update the currentlyActiveFile           
            currentlyActiveFile = new File(rollingPolicy.getActiveFileName());

            // This will also close the file. This is OK since multiple
            // close operations are safe.
            // COMMENT MINE this also sets the new OutputStream for the new file
            this.openFile(rollingPolicy.getActiveFileName()); 
        } catch (IOException e) {
            addError("setFile(" + fileName + ", false) call failed.", e);
        }
    }
}

As you can see, the logic is pretty similar to what you posted. They close the current OutputStream, perform the rollover, then open a new one (openFile()). Obviously, this is all done in a synchronized block since many threads are using the logger, but only one rollover should occur at a time.

A RollingPolicy is a policy on how to perform a rollover and a TriggeringPolicy is when to perform a rollover. With logback, you usually base these policies on file size or time.