No, it's not the number of pages.
This is related to PHP script (cache is usually to avoid theses kind of problems)
You should check what php script is executed, which part takes long.
You can use memory_get_usage
and use throw new Exception("too much memory")
if its over the limit you will define at some point of your script (and inside loops if you have).