Reading files from multiple directories in Logstash?

Question

I read my log files (cron_log, auth_log, mail_log, etc) using this config:

file{
path => '/path/to/log/file/*_log'
}

So I read my log files and check:

if(path) ~= "cron"  -----match--------
if(path) ~= "auth"  -----match--------

Now I have a directories like: Server1 Server2 Server3......In Server 1 there are subdirectories: authlog cronlog.....Inside authlog there are subdirectories date wise (like 2014.05.26, 2014.05.27) which finally contain log file for the day, which I have to parse.

So presently I was having one config file which use to read files using *_log and I use to run that config file and all log files present in /path/to/log/file/*_log were parsed.

Now I have to read from many directories (as explained above).

Will I have to write separate config file for each directory??

What's the best way to achieve this using logstash??

Answer 1

Ruby globs interpret ** as including all subdirectories.

So, for example, you could give the file input a path such as:

/path/to/date/folders/**/*_log