You have some infinite values in your matrix. This should work in the 2 cases ( with and without ff package)
df.dists.inv[is.infinite(df.dists.inv)] <- 0
Applying this with a small example:
require(ape)
set.seed(1)
df <- data.frame(
x = 1:10,
y = 1:10,
v = rnorm(20, mean=4) )
.....
df.dists.inv[is.infinite(df.dists.inv)] <- 0
Moran.I(df$v, df.dists.inv)
$observed
[1] -0.02246154
$expected
[1] -0.05263158
$sd
[1] 0.05399303
$p.value
[1] 0.5763143