Creating a dashboard to analyze IIS web traffic

Question

I am using Windows Server 2003 with IIS6 and Windows Server 2008R2 with IIS7. I need to create a dashboard similar to google analytics that we be view via webpage by non-technical users. I figure this could be done analyzing IIS log files but I am not sure what the best route is for accomplishing this.

What is the best way to create a dashboard that can be access be a to analyze IIS web traffic?

Is analyzing IIS log files the best way to go?
If so, which IIS logging format would be best for this?
Is ODBC logging that that big of a performance hit?

Are there any free or cheap apps out there to accomplish this?

Answer 1

One option is to create an application that uses Microsoft's LogParser tool. It supports a variety of file formats and allows you to query the IIS log files with SQL-like syntax. It also provides a programmable COM API that can be utilized from .NET. There are numerous examples of using LogParser available. Below are a few links, as well a the official forum.

http://www.codinghorror.com/blog/2005/08/microsoft-logparser.html

http://linuxlore.blogspot.com/2006/11/howto-use-microsofts-logparser-to.html

http://support.microsoft.com/kb/910447

http://weblogs.asp.net/rrobbins/archive/2007/12/03/asp-net-code-to-query-ftp-logs-using-logparser.aspx

http://www.codeproject.com/KB/recipes/SimpleLogParse.aspx

http://gluegood.blogspot.com/2008/09/freeware-logparserzip.html

You can also find a few example projects that use LogParser on CodePlex.

In terms of free apps, a couple I know of include:

• IIS Advanced Logging Extension - Has some sort of analytics integration, but I'm not sure of the details.
• Web Log Analyzer - Fairly new, but source code is provided.
• SmartStats - It is a commercial product that's not too expensive, but also provides a free edition that can be used from a single server

Answer 2

LogParser works well -> thanks @Garett

Here is an example query from a IIS6 log (IISW3C format):

C:\Program Files\Log Parser 2.2>logparser -i:IISw3c -o:CSV "SELECT date, s-compu tername, cs-uri-stem, cs-uri-query INTO test2.csv FROM ex131109.log WHERE cs-uri -stem NOT LIKE '%gif%' AND cs-uri-stem NOT LIKE '%.js' AND cs-uri-stem NOT LIKE '%.css' AND cs-uri-stem NOT LIKE '%.bmp' AND cs-uri-stem NOT LIKE '%.jpg'"

The log takes in the IIS format and outputs a CSV file that filters out image, javascript, and css files.