KML scalability: Plotting hundred-thousands of GPS points in KML

Question

I have a rather large KML file with 75000 GPS fixes as KML Placemarks, and upon showing it in Google Earth, the program basically stops working/becomes unresponsive or painfully slow. I want to plot up to half a million fixes at a time.

The KML file looks something like this (which follows the style from https://developers.google.com/kml/documentation/time#example1):

    ...
    <Folder>
        <name>2014-03-17T08:47:15+01:00 to 2014-03-26T13:59:49+01:00</name>
        <Placemark>
            <TimeStamp>
                <when>2014-03-17T08:47:15+01:00</when>
            </TimeStamp>
            <styleUrl>#MarkerStyle</styleUrl>
            <Point>
                <coordinates>10.198022417724133,60.01997647900134</coordinates>
            </Point>
        </Placemark>
        <Placemark>
            <TimeStamp>
                <when>2014-03-17T08:47:16+01:00</when>
            </TimeStamp>
            <styleUrl>#MarkerStyle</styleUrl>
            <Point>
                <coordinates>10.19811805523932,60.020018765702844</coordinates>
            </Point>
        </Placemark>
        ...

My question is: Is there a KML feature that more efficiently shows this trace? I can't just use a Path, since I need the time information for animation.

Alternatively, is there a better tool to show large GPS traces over time?

Thanks!

Answer 1

The key to efficiently scale lots of features for a large KML file is using NetworkLinks and Regions with a collection of smaller KML files. Depth of nesting depends on the number of the features.

You're not likely going to be able to display half-million placemarks at once in Google Earth without restructuring the KML. You can break up the data into segments and display a subset at a time. Something more advanced would be to create heat-map like super-features that represent 100's or 1000's of nearby points with a given color for the number of points in a given area. Zooming into a given area could load the "raw" points for that region using combination of NetworkLinks and Regions.

Here is a discussion showing problems with KML in Google Earth when there is large # of features in a KML file where they all get loaded at once.

You probably need to parse the KML file or better yet to use the raw data to regenerate a series of new KML files with an appropriate grouping of the data.

The data is time based so if the data has a relatively normal distribution of "points" per interval of time then you can simply break the KML files into groups based on time intervals such that 10-50K places get added to each KML file. If the data covers a large geographic area then you can break the data into regions as regularly-spaced cells in a grid and/or also with time.

The master root KML file would have a series of NetworkLinks and reference each of the sub-files. To prevent Google Earth from loading all files at once you need to set the TimeSpan on each NetworkLink as well as a smaller initial interval on the outer-most Document in the root KML file.

The root KML would look something like this:

<kml xmlns="http://www.opengis.net/kml/2.2">
<Document>
    <name>Parent Document</name>
    <open>1</open>
    <!-- use interval for first time span as initial period to load -->
    <TimeSpan>
            <begin>2012-08-12T01:00:00Z</begin>
            <end>2012-08-12T01:59:59Z</end>
    </TimeSpan>

    <NetworkLink>
      <name>Group #1</name>
      <TimeSpan>
            <begin>2012-08-12T01:00:00Z</begin>
            <end>2012-08-12T01:59:59Z</end>
      </TimeSpan>
      <Link>
        <href>group_0100.kml</href>
      </Link>
    </NetworkLink>

    <NetworkLink>
      <name>Group #2</name>
      <TimeSpan>
            <begin>2012-08-12T02:00:00Z</begin>
            <end>2012-08-12T02:59:59Z</end>
      </TimeSpan>
      <Link>
        <href>group_0200.kml</href>
      </Link>
    </NetworkLink>
    ...
</Document>
</kml>

For consistency the group kml files should have the same time span matching the span as defined in the NetworkLink in the parent KML file.

<kml xmlns="http://www.opengis.net/kml/2.2">
 <Document>
  <name>Group #1</name>
  <TimeSpan>
        <begin>2012-08-12T01:00:00Z</begin>
        <end>2012-08-12T01:59:59Z</end>
  </TimeSpan>

  <Placemark>
      <TimeStamp>
        <when>2012-08-12T01:00:00Z</begin>
      </TimeStamp>
  </Placemark>
  ...
 </Document>
</kml>

Answer 2

ReThink your approach, usually only the locations of one vehicle of one day is exported / displayed as kml.

Kml is not suited for mass data display.

Answer 3

I ended up using the Google extensions gx:Track and gx:MultiTrack, where each gx:MultiTrack is put in a Placemark on its own, split per day. If visibility for each placemark is disabled, Google Earth handles millions of points neatly, if you don't display them all at once.