Match Range of Lines in Log

Question

I am trying to figure out how to take a log that that has millions of lines in a day and easily dump a range (based on begin and end timestamp) of lines to another file. Here is an excerpt from the log to show how it is constructed:

00:04:59.703: <abcxyz,v1 from YYY::Process at 14 Nov 2012 07:04:59>
00:04:59.703: 20121114070459 - XXX - 7028429950500220900257201211131000000003536
00:04:59.703: </abcxyz,v1>
00:04:59.711: <abcxyz,v1 from YYY::Process at 14 Nov 2012 07:04:59>
00:04:59.711: 20121114070459 - XXX - 7028690080500220900257201211131000000003538
00:04:59.711: </abcxyz,v1>
00:04:59.723: <abcxyz,v1 from YYY::Process at 14 Nov 2012 07:04:59>
00:04:59.723: 20121114070459 - XXX - 7028395150500220900257201211131000000003540
00:04:59.723: </abcxyz,v1>
00:04:59.744: <abcxyz,v1 from YYY::Process at 14 Nov 2012 07:04:59>

As you can see there are multiple lines per millisecond. What I would like to do is be able to give as an input a begin and end timestamp such as begin=11:00: and end=11:45: and have it dump all the lines in that range.

I have been racking my brain trying to figure this one out, but so far haven't come up with a satisfactory result.

UPDATE: Of course just the first thing I try after I post the question seems to work. Here is what I have:

sed -n '/^06:25/,/^08:25:/p' logFile > newLogFile

More than happy to take suggestions if there is a better way.

Answer 1

I think your sed oneliner is ok for the task.

Besides, you can optimize that for speed (considering the file has millions of lines), exiting the sed script when the desired block was printed (assuming there are no repeated blocks of time in a file).

sed -n '/^06:25/,/^08:25/{p;/^08:25/q}' logFile > newLogFile

This tells sed to quit when the last line of the block was found.

Answer 2

You can use following oneliner:

awk -v start='00:04:59.000' -v end='00:04:59.900' \
  '{if(start <= $1 && end >= $1) print $0}' < your.log > reduced.log

Notice the full format of start and end ranges - it's to keep it simple and doesn't make much problem IMO