VBA to download csv file from a website with login and password

Question

Ole Henrik Skogstrøm kindly posted a reply in the thread: "How do i download a file using VBA (Without internet explorer)".

Have used his code as I wish to download a csv file from www.ft.com and save it to the temp file on my c drive. It is not that often I need to do this, so I decided to use simple excel VBA. I have set up a temporary test subscription account at www.FT.com to illustrate what I would like to download, username is "ft.testing.acct@gmail.com" password is "fttestpassword".

After logging in, the "export data" link can be seen in the top right hand side of the page:

http://portfolio.ft.com/holdings/overview/3415c458-40bf-4e63-903a-37302a88bd83?popout=true&..wsod..=off

The url passed when you click this link is:

http://portfolio.ft.com/PortfolioAPI/ExportToCSV?containerID=3415c458-40bf-4e63-903a-37302a88bd83&type=Holdings&customName=suggested__0YourPortfolio&duration=15&startDate=undefined&endDate=undefined

The following code returns a file, but this file just has "{"json":{"triggerLogin":true}}" in it.

Sub downloadingpositions()
Dim myURL As String
myURL = "http://portfolio.ft.com/PortfolioAPI/ExportToCSV?containerID=3415c458-40bf-4e63-903a-37302a88bd83&type=Holdings&customName=suggested__0YourPortfolio&duration=15&startDate=undefined&endDate=undefined"

Dim WinHttpReq As Object
Set WinHttpReq = CreateObject("Microsoft.XMLHTTP")
WinHttpReq.Open "GET", myURL, False, "ft.testing.acct@gmail.com", "fttestpasword"
WinHttpReq.Send

myURL = WinHttpReq.responseBody
If WinHttpReq.Status = 200 Then
Set oStream = CreateObject("ADODB.Stream")
oStream.Open
oStream.Type = 1
oStream.Write WinHttpReq.responseBody
oStream.SaveToFile "c:\temp\testing.csv ", 2 ' 1 = no overwrite, 2 = overwrite
oStream.Close
End If
End Sub

Can anyone point me in the right direction as to why I'm not getting logged in/ not getting the full csv file??

Answer 1

First thing I noticed about your code is that the password was missing an "s". Then when you have a strange long number in the url is because there is some kind of authentication needed to generate the url "3415c458-40bf-4e63-903a-37302a88bd83"

When this happens you need to locate the signin url and then use a "POST" method to validate the data. Once you have done that then you can send the regular "GET" request and the data will show.

Sub downloadingpositions()

    Const strLOGING_URL As String = "https://registration.ft.com/registration/barrier/login?username=ft.testing.acct@gmail.com&password=fttestpassword"
    Const strPORTFOLIO_URL As String = "http://portfolio.ft.com/PortfolioAPI/ExportToCSV?containerID=3415c458-40bf-4e63-903a-37302a88bd83&type=Holdings&customName=suggested__0YourPortfolio&duration=15&startDate=undefined&endDate=undefined"
    Const strINCORRECT_CREDENTIALS As String = "Your email address and password do not match our records"
    Const strFILE_NAME As String = "c:\temp\testing.csv"

    Dim WinHttpReq As Object
    Dim oStream As Object

    Set WinHttpReq = CreateObject("WinHttp.WinHttpRequest.5.1")

    ' Post to the url to set the credentials
    WinHttpReq.Open "POST", strLOGING_URL, 0
    WinHttpReq.Send


    ' Make sure authetication went through
    If Not UCase$(WinHttpReq.ResponseText) Like "*" & UCase$(strINCORRECT_CREDENTIALS) & "*" Then

        ' Get the information.
        WinHttpReq.Open "GET", strPORTFOLIO_URL, 0
        WinHttpReq.Send

        ' If we have succedeed then write the response to the file.
        If WinHttpReq.Status = 200 Then
            Set oStream = CreateObject("ADODB.Stream")
            oStream.Open
            oStream.Type = 1
            oStream.Write WinHttpReq.ResponseBody
            oStream.SaveToFile strFILE_NAME, 2    ' 1 = no overwrite, 2 = overwrite
            oStream.Close
        End If

    Else
        MsgBox strINCORRECT_CREDENTIALS, vbOKOnly + vbCritical, "Error"
    End If

End Sub

I hope this helps :)