Structure to search efficiently pair of strings matching with pair of strings in different text files

Question

I have 2 files with several thousand of lines of coordinates in a txt/csv format. The format for each line of the first file is : "km, latitude, longitude". In the other file I just have "latitude, longitude" and I am trying to find an efficient way to search where are those thousand of points of the 2nd file. I am using for matching the coordinates just 4 decimals, but occasionally I will need to reduce to 3 for those which didn't match and interpolate them.

My question is if you know a structure alternative to make those searches and access no eternal.

I was thinking in (for both files) with a StreamReader, to access with ReadLine(), split() the line and compare them making a loop. If they match then write the km in my 2nd file. This could be a bit "improved" searching first if the longitude (it changes much more than latitude) of the line being iterated matches and so check the longitude.

Maybe storing all the first file in a Dictionary<<latitude,longitude>,km> would improve something?

Answer 1

Using Dictionary will be good approach as it will result in O(1) search for the file result but i think the memory usage will be very high and building the dictionary it will be costly.

I was thinking can you Binary Search after the sorting the data based on the latitude and longitude . In this case NLog(N) , you can find each matching on on logN time.

      struct D : ICompareable

     {
           decimal latitude,
           decimal longitude,
           int km ;
           public int CompareTo(object a)
          {
              D incoming = (D)a;
             if(incoming.latitude == this.latitude)
                     return this.longitude.CompareTo(incoming.longitude);
             return this.latitude.CompareTo(incoming.latitude);
          }
     }
     // load the first file data in the Struct array and do the Binary Search