Query IIS logs with extra fields using LogParser

Question

I have IIS log with extra field 'foo'.

#Fields: foo date s-sitename ...
foo1 2009-02-15 W3SVC1 ...
foo2 2009-02-15 W3SVC1 ...

As result all LogParser queries are broken:

logparser -i:IISW3C  "SELECT c-ip, s-ip FROM my.log"

Statistics:
-----------
Elements processed: 0
Elements output:    0
Execution time:     0.00 seconds

Is it possible to inform LogParser about such extra fields, so it can parse IIS files?

Answer 1

Try W3C format (-i:W3C).

If that doesn't work and this is a one-time analysis, you could create a script to strip out that column. If this is an ongoing activity, you might want to consider using a standard format, or at least moving the extra field to the end.

By the way LogParser does support custom input formats.

Answer 2

Don't know about LogParser but if you're not successful with that you could try splunk which seems to handle different log formats easily enough.

Answer 3

use option -iHeaderFile to define your own fields. logparser -h will prove additional inforamtion

Answer 4

It's very easy to do !

Simply use the TRIM function around your string. This way, you can type any string you want as a custom extra field in a Log Parser Query.

Reference : http://logparserplus.com/Functions#function_TRIM

For example, I do it in this query (used to retrieve the Average and Max time) :

logparser -i:IISW3C -rtp:-1 -o:NAT -headers:OFF -iw:ON "SELECT TRIM('my-website-custom-extra-column-name.com') AS siteName, TRIM('foo-bar-custom-extra-column-name') AS fooBar, AVG(time-taken) As AverageTimeTaken, MAX(time-taken) As MaxTimeTaken, COUNT(*) As Hits, TO_LOWERCASE(cs-uri-stem) As Uri FROM C:\inetpub\yourwebsite.com\ex*.log TO c:\myOutputParsedLog.txt WHERE (Extract_Extension(To_Lowercase(cs-uri-stem)) IN ('aspx')) GROUP BY TO_LOWERCASE(cs-uri-stem) ORDER BY AverageTimeTaken DESC"