How to reliably categorize HTTP sessions in proxy to corresponding browser' windows/tabs user is viewing?

Question

I was using the Fiddler core .NET library as a local proxy to record the user activity in the web. However, I ended up with a problem which seems dirty to solve. I have a web browser say Google Chrome, and the user opened like 10 different tabs each with different web URLs. The problem is that the proxy records all the HTTP session initiated by each page separately, causing me to figure out using my intelligence the tab which the corresponding HTTP session belonged to. I understand that this is because of the stateless nature of the HTTP protocol. However, I am just wondering if there an easy way to do this? I ended up with below c# code for that in Fiddler. Still, it's not a reliable solution.

This is a modification of the sample project bundled with Fiddler core for .NET 4. Basically what it does is filtering HTTP sessions initiated in last few seconds to find the first request or switching to another page made by the same tab in the browser. It almost works, but not seems to be a universal solution.

 Fiddler.FiddlerApplication.AfterSessionComplete += delegate(Fiddler.Session oS)
        {
            //exclude other HTTP methods
            if (oS.oRequest.headers.HTTPMethod == "GET" || oS.oRequest.headers.HTTPMethod == "POST")
                //exclude other HTTP Status codes
                if (oS.oResponse.headers.HTTPResponseStatus == "200 OK" || oS.oResponse.headers.HTTPResponseStatus == "304 Not Modified")
                {
                    //exclude other MIME responses (allow only text/html)
                    var accept = oS.oRequest.headers.FindAll("Accept");

                    if (accept != null)
                    {
                        if(accept.Count>0)  
                        if (accept[0].Value.Contains("text/html"))
                        {

                            //exclude AJAX
                            if (!oS.oRequest.headers.Exists("X-Requested-With")) 
                            {
                                //find the referer for this request
                                  var referer = oS.oRequest.headers.FindAll("Referer");
                                //if no referer then assume this as a new request and display the same 
                                if(referer!=null)
                                {
                                    //if no referer then assume this as a new request and display the same 
                                    if (referer.Count > 0)
                                    {
                                        //lock the sessions
                                        Monitor.Enter(oAllSessions);

                                       //filter further using the response
                                        if (oS.oResponse.MIMEType == string.Empty || oS.oResponse.MIMEType == "text/html")

                                            //get all previous sessions with the same process ID this session request
                                        if(oAllSessions.FindAll(a=>a.LocalProcessID == oS.LocalProcessID)
                                            //get all previous sessions within last second (assuming the new tab opened initiated multiple sessions other than parent)
                                            .FindAll(z => (z.Timers.ClientBeginRequest > oS.Timers.ClientBeginRequest.AddSeconds(-1)))
                                            //get all previous sessions that belongs to the same port of the current session
                                            .FindAll(b=>b.port == oS.port ).FindAll(c=>c.clientIP ==oS.clientIP)
                                            //get all previus sessions with the same referrer URL of the current session
                                            .FindAll(y => referer[0].Value.Equals(y.fullUrl))
                                            //get all previous sessions with the same host name of the current session
                                            .FindAll(m=>m.hostname==oS.hostname).Count==0 ) //if count ==0 that means this is the parent request
                                                 Console.WriteLine(oS.fullUrl);

                                        //unlock sessions
                                        Monitor.Exit(oAllSessions);
                                    }
                                    else
                                        Console.WriteLine(oS.fullUrl);

                                }
                                else
                                    Console.WriteLine(oS.fullUrl);

                                Console.WriteLine(); 

                            }
                        }
                    }
                }

        };

Answer 1

There is a workaround to do this by script injection. Basically, we have to send a java-script with each HTTP response sent to the browser which in turn sends window.URL value back to the proxy by an HTTP request.

Answer 2

There is no way to do this with perfect accuracy. All the proxy sees is the HTTP data which doesn't contain any information about which tab or process was the source of the request (although the HTTP Referer header may help. In IE, you can enable the X-Download-Initiator header which will help even more.)

But generally, as you've found, you can make informed guesses if you know something about the types of pages you are dealing with, but that's the best you can do.