Is there a sensible limit to “display:none” preloading?

Question

I'm designing a single page website. All the sections to the site pop up modally in DIV elements, so it never leaves the actual browser page. Initially I have all the sections hidden with css using "display:none", and the navigation reveals and hides them accordingly.

Now I've come to putting together the portfolio section, I'm starting to wonder whether it would be best to use AJAX for each portfolio item, as they include larger images. Is there any rough guide to what's a sensible limit for preloading? If all the items were preloaded, the HTML file would look quite monstrous.

I realise this is a bit of a general question and it feels like preloading everything is a bad idea but I don't want to go on a hunch. I'm not quite sure what sort of resources are taken up with the user's computer when elements are set to "display:none" (e.g. Is it a RAM issue?).

Cheers! :)