Google app engine + jquery ajax deferred + cursors

Question

I use the google app engine to run certain projects of mine. Because of the 30 second time limit that it imposes on requests, I use cursors and memcache to throttle the data fetch process.

Problem statement:

1. I have an ajax call log which is basically a list of all the details I need from the server, created by some user interaction
2. Each ajax call in the call log may have data, that may cause the 30 second limit to hit. So I need to break it down and return it in parts.
3. Once all the calls in the log have been completed, I need to perform some custom logic, based on the return data.

Now, I already have a solution that works in this way:

1. Initiate the first ajax call with some random identifier
2. On return, check some parameter to identify whether all the required data has been acquired.
3. If not, then perform the call again, with the same random identifier as in step 1 + save whatever data has been fetched so far
4. If yes, remove the item from log and repeat steps for next item

But all of this code is currently being handled in the success event of the ajax call. I read about jQuery Deferreds and it seems like a really good candidate to fix the atrocious code that is currently being employed.

I've already figured out the following:

1. Use $.when.apply(null, arrayOfDeferreds) to create a dynamic call list of sorts - read jQuery.when
2. I can now use the the deferred.done function to save a data dump for each call
   The problem being, how do I tell the deferred that if the request needs to be called again, then don't resolve the ajax request - just save my data and perform the call again with the same random identifier as its initial call? That is, basically calling my done function repeatedly until all the data is saved ? If there is a better way to do this, please provide an explanation.

Answer 1

You can use pipe recursively as follows:

function myFetch( id ) {
    return $.ajax({
        url: serviceURL,
        data: {
            id: id,
            someOthers: data
        }
    }).pipe(function( data ) {
        saveData( data );
        if ( !isOK( data ) ) {
            return myFetch( id );
        }
    });
}

The ajax request is only defined once and you conditionaly re-issue it in the pipe handler.