Mongodb loop with php through large-datasets

Question

I want to iterate through all of my data >10mio records, but this timeouts.
My query looks like this:

    db.my_collection.find();

For processing the data i am using php, so i am doing an foreach of the cursor. Any suggestions how to do this eventualy with limit and skip maybe?

Answer 1

why do you use the mongo shell syntax if you query from the php client?

anyway I don't think you can hope to find and print 10mio of records in a resonable workload time without split your queries, also catch the Calimero suggestion and create your own pagination algorithm using skip limit.

$cursor = $db->collection->find($criteria, $projection)->sort($order)->skip($skip)->limit($limit);

give also a look at this link

or you could simply implement an external class that does it for you, check here and find MongoDB Pagination

Answer 2

You could use skip/limit, but also disable the timeout for this specific query :

http://www.php.net/manual/en/mongocursor.timeout.php