How to create a Large Distance Matrix?

Question

How to allocate a huge distance matrix in an appropriate way to avoid "allocation is unable" error. Imagine you have a 100.000 points randomly spreaded over some space. How can one cleverly create a matrix or "dist"-object, which represents the the half of DistMatrix. Maybe it should be another object, which will be able efficiently allocate the large number of distances.

You can get the polygonial object from the following link: https://www.dropbox.com/sh/65c3rke0gi4d8pb/LAKJWhwm-l

# Load required packages
library(sp)
library(maptools)
library(maps)

# Load the polygonal object
x <- readShapePoly("vg250_gem.shp")

# Sample or Pick up a large Number of Points
# this command needs some minutes to be done. 
# "coord" is SpatialPoints Object
n <- 1e5
coord <- spsample(x, n, "random")
# Try to measure the distances by dist() 

DistMatrix <- dist(coord@coords)
Error: negative length vectors are not allowed

# Try to measure the distances by spDists()
DistMatrix <- spDists(coord)
Error: cannot allocate vector of size (some number) MB

# It seems that the problem lies on large matrix to be created. 

How is this problem solvable in R for great numbers of "n".

Answer 1

At this point R cannot allocate the random number of megabytes of RAM. At this point, your computer is using all of its memory somewhere else and there just isn't (some number) of MBytes available for your process to continue. You have several solutions at this point. Among them, get a machine with more RAM, close programs, or do your distance calculations in smaller batches. Try a smaller n; and when it works just repeat the process several times until you have your whole matrix of distances.