rsyslog dynamic filename log rotation

Question

All hosts are sending all logs to RSyslog server (ver 5.8.10).

RSyslog uses the following template to save log files -

$template RemoteHost,"/var/log/x/host/%$YEAR%-%$MONTH%-%$DAY%/%HOSTNAME%/%APP-NAME%.log"

So windows event log from host11 will be logged to - /var/log/x/host/2013-09-24/host11/EvntSLog.log

I would now like to setup logrotate such that and entire day's worth of log files is zipped-up and sent to '/nfs/archive/'. So the above log file when archived should look like this - /nfs/archive/2013-09-24.tgz. Note here that i am not zipping up individual log files, i am zipping up an entire directory.

How can i achieve this using logrotate/cron ?

Answer 1

For now, i have decided to do this with a python script that zips everything up and copies it over to a nfs partition. If someone has managed to implement this using logrotate, please respond to this thread.

Answer 2

I suppose you know how logrotate/cron work.

You can use olddir to set the directory on the same physical disk, and use postrotate to move all contents of olddir to a directory on different partition.

olddir /var/log/x/host/host11/

postrotate
    mv /var/log/x/host/host11/* /nfs/archive/
endscript

Or (if you do not want to use any postrotate workarounds) you can use a symlink:

sudo ln -s /var/log/x/host/host11 /nfs/archive

NOTE:

Please use wildcards with caution. If you specify *, logrotate will rotate all files, including previously rotated ones. A way around this is to use the olddir directive or a more exact wildcard (such as *.log).

I recommend, that you do not use the date as your file/directory name (referencing to your template). You can set it in logrotate. That way you don't have to use any (in your case multiple) wildcards.