Joomla getItems running out of memory with big results set

Question

I've got just over 10,000,000 records in the database of my component and I think getItems/getListQuery is trying to load every single one of them into memory. The search form on the site extremely slow or comes back saying php is out of memory.

phpMyAdmin seems to be able to handle displaying this data - why not Joomla?

The strange thing is that the items are then displayed correctly using the globally set list limit of 5 to a page.

I've just looked and Joomla's cache is disabled - is that screwing me up here?

Many thanks in advance!

Answer 1

I fixed it in the end by copying the getPagination, getTotal, getItems etc. from the library's (list.php) and into my model (to override them). Then in each method, I made sure the results where returned instead of sending them to the cache.

The getTotal function seems to count the number of rows instead of doing a seperate count(*). That's ok with a few thousand records but over 1/2 million is asking for trouble!