The buffer is around 500K lines as it contains work ouput over few days. I am logged into a server machine and am not sure about the access speed of the files being read by eldoc.
Either way, this fixed it for me
(setq ess-use-eldoc nil)
Manual says the following should work too, but haven't had luck with that yet
(setq ess-use-eldoc ’script-only)
VitoshKa, thanks for the response