Not all cookies being saved using cookiecontainer and webclient/httpwebrequest

Question

I am trying to log in to a site using CookieAwareClient (code here: C# WebClient Log onto Website). When I log in to the site using my web browser, I get about 10 cookies. If I disable JavaScript and try to log in, I get 5 (activeTab, id, mcim, PHPSESSID, username). But when I try to log in using the CookieAwareClient the only cookie being saved is PHPSESSID.

What could be the reason the that the other cookies are not being saved? I know the WebClient does not execute JavaScript but the 3 other cookies that are generated when JavaScript is disabled are not being preserved either.

Answer 1

First, an obvious question: before you turned off script in your browser and tried to log in, did you clear your cookies first? If not, clear cookies and try again.

Assuming you did clear cookies first and still got 5 new cookies after a no-script login, then by looking at the missing cookie names I'd guess that your CookieAwareClient is not actually successfully logging in the user. Lack of a "username" cookie is what I'd expect if the login didn't succeed. When you look at the HTML returned by your login request, does it look like a successful login? Or is it returning some sort of error?

If the HTML returned is a successful login, then the next possible culprit I'd look at would be additional requests being executed by the browser but not executed by your code. For example, the login page could have a META REFRESH which redirected to another page (which set a cookie). Or the login page could contain an IFRAME, IMG, SCRIPT, etc. which in turn set a cookie. To diagnose this, use Fiddler, Firebug, or other similar tool to see what specific HTTP requests are being executed by your browser as part of the login process. Make sure that the POST to the login page is doing all the cookie-setting, and if not, then you'll need to add additional requests to your code after the initial login.

If it turns out there's only one HTTP request setting cookies, then the problem is likley to be the HTTP headers or POST data that your code is sending (or not sending) to the server. The browser is likely sending different headers or POST data. Diagnosing this also requires Firebug, Fiddler, or similar tool to compare the HTTP headers and POST data that a browser is sending vs. what your code is sending.

If none of these suggestions work, post a comment and we can iterate.