How can I Monitor HTTP Traffic and Get a List of URLs in C#?

Question

Basically, what we at the company are doing right now is plainly using HttpFox in Firefox to manually get webtrends tags and values that are appended as query string in an image/gif type url that is listed whenever we type a specific url on the browser. after starting the addon, and going to a particular URL, a list of URLs appear on the HttpFox url list. one of the URLs contains the query string we needed, and we copy that list onto the excel file afterwards. So if there are 2000 pages to get, we open those 2000 pages and do the copying and pasting one by one.

That leads to my task of automating the 'tag collection' by making a program via C# that does the sniffing/monitoring once a particular url is entered. I'll provide a list of urls to be read, and the program will do the monitoring, and fetching of data, as well as recording the data to the database.

My problem is how to fetch that list of URLs using C# once i type a specific URL - something like fiddler and httpfox do, but shouldn't be browser dependent.

Btw, Both Fiddler and HttpFox do the job that i need, but i don't think we can feed fiddler with a list of url's to sniff at. HttpFox is good but it's only run via firefox, adn importing is so manual.

HttpFox doesn't have a name for that 'list' of URLs, but under Fiddler, it's named 'Web Sessions'. Hope this one helps you understand more of my problem.

I've already looked at HttpListener and HttpWebRequest but can't seem to get the result that I am looking for.

Any help would be appreciated. :)

Answer 1

You could embed Fiddler as a .NET library, and run it as a part of your application: http://fiddler.wikidot.com/fiddlercore