When I need to do this, I limit my pool of data to check using a "box" around my current data point. For example, in psuedo-SQL:
SELECT DataID, lat, long, distance=fn_getdistance(@ThisLat, lat, @ThisLong, long)
FROM MyTable
WHERE lat BETWEEN (@ThisLat - @LatDistance) AND (@ThisLat + @LatDistance) AND
long BETWEEN (@ThisLong - @LongDistance) AND (@ThisLong + @LongDistance)
Variables:
@ThisLat, @ThisLong: latitude & longitude of your current location
@LatDistance, @LongDistance: the dimensions of the "box" around your current position (play around with different numbers depending on your application and needs)
(Obviously, you'll want an index on the lat & long fields.)