WinHTTP VBA subsequent request cannot use the previous login credentials?

Question

I'm using WinHTTP in Access 2007 VBA to fetch some list of items requiring a cookie login credential account.

First I login through https://www.example.com/login.php with this:

  Dim strCookie As String, strResponse As String, _
    strUrl As String
'
  Dim xobj As Object
'
  Set xobj = New WinHttp.WinHttpRequest
'
  strUrl = "https://www.example.com/login.php"
  xobj.Open "POST", strUrl, False
  xobj.SetRequestHeader "User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
  xobj.SetRequestHeader "Content-Type", "application/x-www-form-urlencoded"
  xobj.Send "username=johndoe2&password=mypassword"
'
  strCookie = xobj.GetResponseHeader("Set-Cookie")
  strResponse = xobj.ResponseText

The content of strResponse indicates that my login is OK, as johndoe2 is welcomed in this string. strCookie saves the Set-Cookie returned by the HTTP server after the successful login.

Next I need to get a confidential page only accessible for a logged user: https://www.example.com/secret-contents.php. I do this, with previous Set-Cookie header strCookie, resent to the server:

'
' now try to get confidential contents:
'
  strUrl = "https://www.example.com/secret-contents.php"
  xobj.Open "GET", strUrl, False
  xobj.SetRequestHeader "Cookie", strCookie
  xobj.Send
'
  strCookie = xobj.GetResponseHeader("Set-Cookie")
  strResponse = xobj.ResponseText

Unfortunately, it's failed, as the new strResponse indicates that the fetched content is not the required one, but rather again the login page. And also strCookie has changed.

This has been tested and produces no effect, as it's only for Windows/OS linked authentication, such as the famous basic, NTLM, digest and Kerberos authentications, not for that based on cookie:

xobj.SetCredentials "johndoe2", "mypassword", 0

What else to send as headers to the remote server other than Set-Cookie, in order to use the previously certified credential ?

The server uses typo3 CMS framework.

Answer 1

In this half day, I finally figured out how to use the previous login credential for subsequent requests, thanks to the help of Alex K., fiddler2 has opened the door to enter the HTTP headers's world. I would like to share what has worked for me today.

The work consists of 2 steps, login via a URL1 and then fetch the HTML content of the credential required URL2.

1. Login via URL1, kept the same as in the Question:

  Dim strCookie As String, strResponse As String, _
    strUrl As String
'
  Dim xobj As Object
'
  Set xobj = New WinHttp.WinHttpRequest
'
  strUrl = "https://www.example.com/login.php"
  xobj.Open "POST", strUrl, False
  xobj.SetRequestHeader "User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
  xobj.SetRequestHeader "Content-Type", "application/x-www-form-urlencoded"
  xobj.Send "username=johndoe2&password=mypassword"
'
  strCookie = xobj.GetResponseHeader("Set-Cookie")
  strResponse = xobj.ResponseText

2. Get the content of the username/password protected URL2:

'
' now try to get confidential contents:
'
  strUrl = "https://www.example.com/secret-contents.php"
  xobj.Open "GET", strUrl, False
'
' these 2 instructions are determining:
'
  xobj.SetRequestHeader "Connection", "keep-alive"
  xobj.SetRequestHeader "User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
'
  xobj.SetRequestHeader "Cookie", strCookie
  xobj.Send
'
  strCookie = xobj.GetResponseHeader("Set-Cookie")
  strResponse = xobj.ResponseText

One notices the usage of two additional headers sent to the HTTTP server in the second step:

"Connection", "keep-alive"

"User-Agent", "My pretending navigator blabla..."

without them the URL2 could not be successfully fetched, instead, a good configured website will redirect you to URL1 for authentication again.

In one word, the session must have a keep-alive Connection in order to re-use the gained login credential.

This works indifferently for the http and https protocols.

The HTML login input field names depend on the target site, here username and password. The conceiver of the website may use such words as user, pass; loginuser, loginpass; ... you can easily figure this out by looking at the source code of the login form.