C# WebBrowser: cannot get latest web page HTML after HTTP POST?

Question

I have the following:

   protected static void GetWebPageWorker()
   {
       using (WebBrowser browser = new WebBrowser())
       {
           //  browser.ClientSize = new Size(_width, _height);
           browser.ScrollBarsEnabled = false;
           browser.ScriptErrorsSuppressed = true;
           browser.Navigate(_url);

           // Wait for control to load page
           while (browser.ReadyState != WebBrowserReadyState.Complete)
               Application.DoEvents();

           //Insert the search term in to the input textfield
           browser.Document.GetElementById(search_div_id).OuterText = search_term;

           //Click on the search button (POST request)
           browser.Document.GetElementById(search_button_id).InvokeMember("Click");

           //This didn't work
           //browser.Update();

           while (browser.ReadyState != WebBrowserReadyState.Complete)
               Application.DoEvents();

           //PROBLEM this is returning the original HTML before POST
           //and not the HTML from the web page after the POST.
           html = browser.DocumentText;
       }
   }

it inserts a value to search for in a HTML element and then clicks the search button and the POST is done. However the WebBrowser object doesn't seem to refresh its DocumentText property so that I can return the HTML of the page after the search button has been clicked.

How can I fix this?

Answer 1

WebBrowser is most likely the wrong tool for this task. You would use WebBrowser when you want to actually be able to see the web page on the screen. It looks like you are instead just trying to POST something and get the result.

Try WebClient instead, it is much simplier and works much better for POST:

using (var client = new WebClient())
{
    client.Headers["User-Agent"] = "Mozilla/4.0 (Compatible; Windows NT 5.1; 
           MSIE 6.0) (compatible; MSIE 6.0; Windows NT 5.1; 
          .NET CLR 1.1.4322; .NET CLR 2.0.50727)";

    const string url = "http://www.example.com";    

     var inputs = new NameValueCollection {
         {"search_div_id", search_term},
     }

    var response = client.UploadValues(url, inputs);
    string html = Encoding.UTF8.GetString(response);
}

Answer 2

Please use the following Statements after Invoking Click Member Function. It will surely work.

`HtmlElement hleGetData = (HtmlElement)hdoc.GetElementById("getButton");
hleGetData.InvokeMember("click");
while (webBrowser1.ReadyState != WebBrowserReadyState.Complete)
{
    System.Windows.Forms.Application.DoEvents();
};
//Some DoEvents....
System.Windows.Forms.Application.DoEvents();
System.Windows.Forms.Application.DoEvents();
System.Windows.Forms.Application.DoEvents();
System.Windows.Forms.Application.DoEvents();
System.Windows.Forms.Application.DoEvents();`

Answer 3

webBrowser1.Refresh(); and webBrowser1.Update(); don't work after POST request.

while (webBrowser1.ReadyState != WebBrowserReadyState.Complete)
        {
            System.Windows.Forms.Application.DoEvents();
           //System.Threading.Thread.Sleep(500);
        }

don`t work also.

Only way at the moment is to use webBrowser1.Navigate("URL"); once again after every POST request.

Microsoft Visual Studio Community 2019 Version 16.7.5 VisualStudio.16.Release/16.7.5+30523.141 Microsoft .NET Framework Version 4.8.03752