How to extract Geographic data from open street map or Google maps

Question

i need to retrieve all the city names from a specific country using openstreet map or google maps. is there any API available?

or is there any other way of getting this world geographic data?

Answer 1

You should definitely checkout GeoNames. They have the entire world in a standardized database. You can download it or use their API.

I download the US database and use a connector I created in C# to insert States, Cities, Towns, and Zip Codes in my database.

    public static class GeoNamesConnector
{
    #region GeoName Constants
    private static readonly string GeoNamesPath = HttpContext.Current.Server.MapPath("~/App_Data/GeoNames/US.txt");
    const int GeoNameIdColumn = 0;
    const int NameColumn = 1;
    const int LatitudeColumn = 4;
    const int LongitudeColumn = 5;
    const int FeatureCodeColumn = 7;
    const int CountryCodeColumn = 8;
    const int Admin1CodeColumn = 10;
    const int Admin2CodeColumn = 11;
    #endregion

    #region AlternateName Constants
    private static readonly string AlternateNamesPath = HttpContext.Current.Server.MapPath("~/App_Data/GeoNames/alternateNames.txt");
    const int AlternateNameIdColumn = 0;
    const int AltNameGeoNameIdColumn = 1;
    const int IsoLanguageColumn = 2;
    const int AlternateNameColumn = 3;
    #endregion

    public static void AddAllEntities(GeoNamesEntities entities)
    {
        //Remember to turn off Intellitrace
        Stopwatch stopwatch = new Stopwatch();
        stopwatch.Start();
        var geoNamesSortedList = AddGeoNames(entities);
        Trace.WriteLine(String.Format("Added GeoNames: {0}", stopwatch.Elapsed));
        stopwatch.Restart();

        SetupGeoNameChildRelationships(geoNamesSortedList, entities);
        Trace.WriteLine(String.Format("Setup GeoName parent/child relationships: {0}", stopwatch.Elapsed));
        stopwatch.Restart();

        AddPostalCodeAlternateNames(geoNamesSortedList, entities);
        Trace.WriteLine(String.Format("Added postal codes and relationships with parent GeoNames: {0}", stopwatch.Elapsed));
    }

    private static SortedList<int, GeoName> AddGeoNames(GeoNamesEntities entities)
    {
        var lineReader = File.ReadLines(GeoNamesPath);
        var geoNames = from line in lineReader.AsParallel()
                       let fields = line.Split(new char[] { '\t' })
                       let fieldCount = fields.Length
                       where fieldCount >= 9
                       let featureCode = fields[FeatureCodeColumn]
                       where featureCode == "ADM1" || featureCode == "ADM2" || featureCode == "PPL"
                       let name = fields[NameColumn]
                       let id = string.IsNullOrEmpty(fields[GeoNameIdColumn]) ? 0 : int.Parse(fields[GeoNameIdColumn])
                       orderby id
                       select new GeoName
                       {
                           Id = Guid.NewGuid(),
                           GeoNameId = id,
                           Name = fields[NameColumn],
                           Latitude = string.IsNullOrEmpty(fields[LatitudeColumn]) ? 0 : Convert.ToDecimal(fields[LatitudeColumn]),
                           Longitude = string.IsNullOrEmpty(fields[LongitudeColumn]) ? 0 : Convert.ToDecimal(fields[LongitudeColumn]),
                           FeatureCode = featureCode,
                           CountryCode = fields[CountryCodeColumn],
                           Admin1Code = fieldCount < 11 ? "" : fields[Admin1CodeColumn],
                           Admin2Code = fieldCount < 12 ? "" : fields[Admin2CodeColumn]
                       };
        var sortedList = new SortedList<int, GeoName>();
        int i = 1;
        foreach (var geoname in geoNames)
        {
            sortedList.Add(geoname.GeoNameId, geoname);
            entities.GeographicAreas.AddObject(geoname);
            if (i++ % 20000 == 0)
                entities.SaveChanges();
        }
        entities.SaveChanges();
        return sortedList;
    }

    private static void SetupGeoNameChildRelationships(SortedList<int, GeoName> geoNamesSortedList, GeoNamesEntities entities)
    {
        foreach (var geoName in geoNamesSortedList.Where(g => g.Value.FeatureCode == "ADM2" || g.Value.FeatureCode == "ADM1"))
        {
            //Setup parent child relationship
            IEnumerable<KeyValuePair<int, GeoName>> children = null;
            switch (geoName.Value.FeatureCode)
            {
                case "ADM1":
                    children =
                        geoNamesSortedList.Where(
                            g =>
                            g.Value.FeatureCode == "ADM2" &&
                            g.Value.Admin1Code == geoName.Value.Admin1Code);
                    break;
                case "ADM2":
                    children =
                        geoNamesSortedList.Where(
                            g =>
                            g.Value.FeatureCode == "PPL" &&
                            g.Value.Admin1Code == geoName.Value.Admin1Code &&
                            g.Value.Admin2Code == geoName.Value.Admin2Code);
                    break;
            }
            if (children != null)
            {
                foreach (var child in children)
                    geoName.Value.Children.Add(child.Value);
            }
            entities.SaveChanges();
        }
    }

    private static void AddPostalCodeAlternateNames(SortedList<int, GeoName> geoNamesSortedList, GeoNamesEntities entities)
    {
        var lineReader = File.ReadLines(AlternateNamesPath);
        var alternativeNames = from line in lineReader.AsParallel()
                               let fields = line.Split(new char[] { '\t' })
                               let fieldCount = fields.Length
                               where fieldCount >= 4 && fields[IsoLanguageColumn] == "post"
                               let geoNameId = int.Parse(fields[AltNameGeoNameIdColumn])
                               orderby geoNameId
                               select new AlternateName
                               {
                                   Id = Guid.NewGuid(),
                                   AlternateNameId = int.Parse(fields[AlternateNameIdColumn]),
                                   ParentGeoNameId = geoNameId,
                                   Name = fields[AlternateNameColumn],
                                   IsoLanguage = fields[IsoLanguageColumn]
                               };
        //Iterate through to convert from lazy (AsParallel) so it is ready for use
        foreach (var alternateName in alternativeNames)
        {
            int key = alternateName.ParentGeoNameId;
            if (geoNamesSortedList.ContainsKey(key))
            {
                entities.GeographicAreas.AddObject(alternateName);
                alternateName.Parent = geoNamesSortedList[key];
            }
        }
        entities.SaveChanges();
    }

}

There is also Open Street Maps that you can download or use their API.

I do not suggest Yahoo's new API they are cutting products left and right and you never know how long it will be around. Also you cannot download a whole dump currently.

Answer 2

Jan. 29, 2013 Update: I've created a CSV dataset of all the cities and populated places in the world, along with a latitude/longitude area centroid, and placed into the public domain. I combined data from the USGS GNIS server for the US, and the NGA GNS server for all other countries. Below is the metadata for the CSV file layout and link to the dataset:

http://www.opengeocode.org/download.php#cities

Column 1: ISO 3166-1 alpha-2 country code.
Column 2: US FIPS 5-2 1st level administrative division code (e.g., state/province).
Column 3: NGA GNS Feature Description (DSG) code.
Column 4: NGA GNS Unique Feature Identifier (UFI).
Column 5: ISO 639-1 alpha-2/3 code for language corresponding to the feature name.
Column 6: Language script (e.g., latin, arabic, chinese, etc) corresponding to the feature name.
Column 7: Feature name.
Column 8: Latitude coordinate of the area centroid.
Column 9: Longitude coordinate of the area centroid.

---

I looked at the solution from Jonperl. It could use some comments. First, I believe geonames.org gets the US city data from the USGS GNIS server. One can directly get a download file from them.

http://geonames.usgs.gov/domestic/download_data.htm

A few points someone should know: ADM1 stands for first level administrative division. For the US, these are the 50 states, District of Columbia, the 5 US territories, and 4 freely associated states.

ADM2 stands for second level administrative division. For the US, these are counties, boroughs and census designated areas for Alaska, parishes for Louisiana, municipios for Puerto Rico, islands for the Virgin Islands, Marshall Islands, U.S. Minor Outlying Islands, districts for American Samoa, and municipalities for Nothern Mariana Islands.

PPL are populated places. I'm not sure how geonames.org sorts these, but this category includes with cities: large subdivisions, unicorporated areas and large trailer parks. Thney also include some historical places.

I can answer alot of these questions. I'm part of a public domain geospatial team at OpenGeoCode.Org

Andrew

Answer 3

Download the data from http://www.geonames.org/

Answer 4

I don't know if you're restricted to google maps or openstreet map but you might find taking a look at Yahoo's woeid interesting.

http://developer.yahoo.com/geo/geoplanet/

I've had a play around with this and it's extremely powerful.