Changing the user agent of the WebBrowser control

Question

I am trying to change the UserAgent of the WebBrowser control in a Winforms application.

I have successfully achieved this by using the following code:

[DllImport("urlmon.dll", CharSet = CharSet.Ansi)]
private static extern int UrlMkSetSessionOption(
    int dwOption, string pBuffer, int dwBufferLength, int dwReserved);

const int URLMON_OPTION_USERAGENT = 0x10000001;

public void ChangeUserAgent()
{
    List<string> userAgent = new List<string>();
    string ua = "Googlebot/2.1 (+http://www.google.com/bot.html)";

    UrlMkSetSessionOption(URLMON_OPTION_USERAGENT, ua, ua.Length, 0);
}

The only problem is that this only works once. When I try to run the ChangeUserAgent() method for the second time it doesn't work. It stays set to the first changed value. This is quite annoying and I've tried everything but it just won't change more than once.

Does anyone know of a different, more flexible approach?

Thanks

Answer 1

I'm not sure whether I should just copy/paste from a website, but I'd rather leave the answer here, instead of a link. If anyone can clarify in comments, I'll be much obliged.

Basically, you have to extend the WebBrowser class.

public class ExtendedWebBrowser : WebBrowser
{
    bool renavigating = false;

    public string UserAgent { get; set; }

    public ExtendedWebBrowser()
    {
        DocumentCompleted += SetupBrowser;

        //this will cause SetupBrowser to run (we need a document object)
        Navigate("about:blank");
    }

    void SetupBrowser(object sender, WebBrowserDocumentCompletedEventArgs e)
    {
        DocumentCompleted -= SetupBrowser;
        SHDocVw.WebBrowser xBrowser = (SHDocVw.WebBrowser)ActiveXInstance;
        xBrowser.BeforeNavigate2 += BeforeNavigate;
        DocumentCompleted += PageLoaded;
    }

    void PageLoaded(object sender, WebBrowserDocumentCompletedEventArgs e)
    {

    }

    void BeforeNavigate(object pDisp, ref object url, ref object flags, ref object targetFrameName,
        ref object postData, ref object headers, ref bool cancel)
    {
        if (!string.IsNullOrEmpty(UserAgent))
        {
            if (!renavigating)
            {
                headers += string.Format("User-Agent: {0}\r\n", UserAgent);
                renavigating = true;
                cancel = true;
                Navigate((string)url, (string)targetFrameName, (byte[])postData, (string)headers);
            }
            else
            {
                renavigating = false;
            }
        }
    }
}

Note: To use the method above you’ll need to add a COM reference to “Microsoft Internet Controls”.

He mentions your approach too, and states that the WebBrowser control seems to cache this user agent string, so it will not change the user agent without restarting the process.

Answer 2

The easiest way:

webBrowser.Navigate("http://localhost/run.php", null, null,
                    "User-Agent: Here Put The User Agent");

Answer 3

Also, there is a refresh option in the function (according to MSDN). It worked well for me (you should set it before any user agent change). Then the question code could be changed like this:

[DllImport("urlmon.dll", CharSet = CharSet.Ansi)]
private static extern int UrlMkSetSessionOption(
    int dwOption, string pBuffer, int dwBufferLength, int dwReserved);

const int URLMON_OPTION_USERAGENT = 0x10000001;
const int URLMON_OPTION_USERAGENT_REFRESH = 0x10000002;

public void ChangeUserAgent()
{
    string ua = "Googlebot/2.1 (+http://www.google.com/bot.html)";

    UrlMkSetSessionOption(URLMON_OPTION_USERAGENT_REFRESH, null, 0, 0);
    UrlMkSetSessionOption(URLMON_OPTION_USERAGENT, ua, ua.Length, 0);
}

Answer 4

I'd like to add to @Jean Azzopardi's answer.

void BeforeNavigate(object pDisp, ref object url, ref object flags, ref object targetFrameName,
        ref object postData, ref object headers, ref bool cancel)
{
    // This alone is sufficient, because headers is a "Ref" parameters, and the browser seems to pick this back up.
    headers += string.Format("User-Agent: {0}\r\n", UserAgent);
}

This solution worked best for me. Using the renavigating caused other weird issues for me, like the browser content suddenly vanishing, and sometimes still getting Unsupported Browser. With this technique, all the requests in Fiddler had the correct User Agent.