A rotating log file in perl

Question

I have implemented a log file that will be storing the cpu and memory state of a process after every minute.I have limited the maximum size of the file to 3MB (thats enough for my purpose).

The script will be called by a cron job after every minute and the script will log the details for that minute and will rename the file as "Log_.log".

When the size reaches "3MB - 100 bytes" I reset the file pointer to point to the begining and will overwrite the first entry in the log file and will now rename the file as "Log_<0+some offset>.log".

As I am renaming the file after every minute to update the file pointer position, is it a good/efficient way ?

I do not want to maintain more than one log file for this purpose.

Another option for me is to maintain the file pointer position in a file ,but ....another file !! not interested in maintaining one if this option is good :)

Thanks in Advance.

Answer 1

Are you an engineer? This is a nice example of some simple task, solved by a perfectly working but overly complex solution.

Unless the content you put in takes exactly as many bytes as the content you take out, writing "in" a file will actually cause the whole following part after your writing position to be rewritten to disk. Append is much cheaper.

Renaming the file to store the pointer works - but it's not very elegant, and makes stuff more complex (for one, your process needs write rights to the directory in which the file resides - else just write access to two files is sufficient)

Unless disk space is an issue (and really, it rarely is), your approach is less efficient than say, append everything to a file, and rotate the file when it reaches its maximum size. This way you always have the last 3MB of logs available, and maximum 3MB more in your current file. It will make parsing the file a lot easier too, instead of recalculating the entire pointer position thing.

Update to answer your comment:

Renaming a file every minute (or even every second) shouldn't slow down your system significantly, don't worry about that.

Our concerns are mainly with "why you think you need to rename the file". It's not better technically, it's not better from a logical point of view, it makes a lot of other (future) tasks harder. You could store the file pointer in a seperate file, or at the end of your file, and there are better^H^H^H^H^H^H simpler solutions that don't require the file pointer at all.

Answer 2

I'm confused why you would rename your file. What does this accomplish?

Are the log entries fixed size? Or variable size?

If the entries are fixed size, then there is no trouble in re-writing the existing file from the start: you won't ever have incomplete entries in your file, and if you are writing a counter or timestamps to the file, it should be clear where the 'cursor' is located.

If the entries are variable size, then you should probably not begin re-writing the file from the beginning without somehow making it clear where the 'cursor' is located in the file, and write code that is resilient to reading truncated log entries.

Can you re-use existing tools such as RRDtool?