Extracting geo-coordinates for List of Addresses (VBA EXCEL)

Question

I have a large list of locations that I need to find the latitude and longitude for to use in another application. I've created some code to go to a website and retrieve this information. The first coordinate is placed fine most of the time. However, not all the addresses are recorded in excel. Sometimes I'll get empty cells. I can't seem to figure out what's wrong. Here's is the code:

Sub GetCoordinates()
    Dim IE As New InternetExplorer
    Dim Doc As HTMLDocument

    Dim rowNum As Integer
    Dim lastRow As Integer

    Dim tURL As String
    Dim coordinates As String

    IE.Visible = True

    'This will be the number of rows with an address (5 is for testing purposes)
    lastRow = 5

    For rowNum = 1 To lastRow
        'The URL is determined by the address in a cell (ex. in C2)
        tURL = "http://dbsgeo.com/latlon/?" & Cells(rowNum, "C").Value
        IE.navigate tURL

        Do
        'Wait until the page loads
        Loop Until IE.readyState = READYSTATE_COMPLETE

        Set Doc = IE.document

        coordinates = Trim(Doc.getElementById("latlon").innerText)

        Cells(rowNum, "B").Value = coordinates
    Next rowNum

    IE.Quit
End Sub

It's not that the code doesn't run; The problem is I'll get coordinates for say 3 out of 8 addresses, in random order. There are even times when the wrong coordinates are recorded. I don't know why it's like that. Can someone tell me what's wrong with my code?

Answer 1

The problem seems to be that those fields "latlon" for example, are being populated by some javascript functions which aren't executed until after the page has loaded, so relying on the IE.document.readyState is not reliable in this instance.

Ordinarily, I say use the WinAPI sleep function (put this at the top of your module):

Public Declare Sub Sleep Lib "kernel32" (ByVal dwMilliseconds As Long)

Then, do your "waiting" loop like:

Do
   Sleep 250
Loop Until IE.ReadyState = READYSTATE_COMPLETE And Not IE.Busy 

This was still not working for me for this particular website, though, although if I did Sleep 1000 that seemed to work, the timing will vary based on traffic, network speed, connectivity, available system resources, etc., so even that is not 100% reliable.

I also tried to find a way to emulate the XMLHTTPRequest (which is faster than using the browser directly) but this was also not working.

I looked at the other site. It is possible to fill forms and submit/click controls on a website using VBA, for the most part, but I never found those to be very reliable and didn't try that site.

I spent waaaaay too much time trying to force that solution to work, but it is a bad solution because it's a bad website that's hacked together by some 3rd party using Google's Geocode API.

You should just use Google's API directly

Documentation:

https://developers.google.com/maps/articles/geocodingupgrade

You can construct a URL like:

http://maps.googleapis.com/maps/api/geocode/xml?address=Paris,+France

You can load that to an XML/DOM parser (use a vba reference to Microsoft XML, v6.0 (aka MSXML2) which is what I would recommend. This does not use a browser, it is a stream that is capable of loading an XML from a url (like above) directly in memory.

With a DOM parser, use the DOMDocument's .GetElementsByTagName("location")(0) and then pull the lat/lon from the child nodes.

Let me know if you have trouble implementing that.