PHP: Processing thousands of entries, script dies after certain amount

Question

I'm calling an MLS service that responds with 4000+ records ... and I need to process each and every one of them, as well as insert all of the meta data per listing.

I'm able to get to about 135 (* 150 meta records) and then the script apparently stops responding, or at least stops processing the rest of the data.

I've added the following to my .htaccess file:

php_value memory_limit 128M

But this doesn't seem to help me any. Do I need to process chunks of the data at a time, or is there another way to ensure that the script will indeed finalize?

Answer 1

It isn't the memory- it's most likely the script execution time.

Try adding this to your htaccess, then restart apache:

php_value max_execution_time 259200

Answer 2

You should probably enable display_errors and error_reporting to get a better analysis of why the script isn't processing.

However, you should also consider making sure the time limit isn't being hit by calling:

set_time_limit( 0 );

This will give you an unlimited time period. You can also just set it to something relatively high, like 600 (10 minutes)