I'm running really long task in php. It's a website crawler and It has to be polite and sleep for 5 seconds each page to prevent server overloading. Script starts with:
ignore_user_abort(1);
session_write_close();
ob_end_clean();
while (@ob_end_flush());
set_time_limit(0);
ini_set('max_execution_time',0);
After few hours (between 3-7h) script dies without any visible reason. I've checked
- apache error log (nothing)
- php_errors.log (nothing)
- output for errors (10 578 467b of debug output, no errors)
- memory consumption (stable, around 3M from
memory_get_usage(true)checked every 5 sec, limit set to 512M)
It's not browser, cause I was using wget and chrome to check with the similar reason. Output is sent to browser every 2-3 seconds, so I don't think that's the fault + I ignore user abort.
Is there any other place I can check to find the issue?