Do I need Web Workers for looping AJAX-requests?

Question

Given: a php-script for parsing portions of data on a web-site. It parses about 10k products hence rather slow.

I need to make a web-frontend with html/css/js for it. I made a loop which makes ajax-requests and shows progress inforamtion. It uses syncronous ajax because it needs to wait until another request is done to perform another.

 do {
    var parseProductsActive = true;
    var counter = 0;
    myAjax('parseProducts.php?start='+counter, false, function(resp) {
        if (resp[0]=='s') {
            counter += Number(resp.substring(1));
            parseProductsActive = false;
        }
        else {
            counter += Number(resp);
        }   
        self.postMessage(counter);  
    });
 } while (parseProductsActive==true);

I'm doing it in a Web Worker because I'm afraid it's going to hang up the interface because of this endless loop and (a)synchronousness of ajax itself won't help to solve the prolem.

But when I tried to use ajax in a web worker I found it's hard though possible because jQuery doesn't work in a Web Worker at all. It uses DOM even for non-DOM operations and DOM isn't available in a Web Worker. And many developers doubt using Web Workers at all. I just wanted to ask if I am doing it right or wrong. Is there any more surface solutions to that I can't see?

Answer 1

You guessed right: a recursive callback is the way to do a bunch of asynchronous requests in sequence. It might look a bit like this:

var parseProductsActive = true;
var counter = 0;

//define the loop
function doNextAjax(allDone){
   //Instead of just returning, an async function needs to 
   //call the code that comes after it explicitly. Receiving a callback
   //lets use not hardcode what comes after the loop.

  if(!parseProductsActive){
    allDone();
  }else{
    //use async Ajax:
    myAjax('parseProducts.php?start='+counter, true, function(resp) {
        if (resp[0]=='s') {
            counter += Number(resp.substring(1));
            parseProductsActive = false;
        }
        else {
            counter += Number(resp);
        }   
        self.postMessage(counter);

        doNextAjax(); // <---
    });
}

//Start the loop
doNextAjax(function(){
  console.log("the code that runs after the loop goes here")
});

//BTW, you might be able to get rid of the "parseProductsActive" flag with a small
// refactoring but I'm keeping the code as similar as possible for now.
//It would be kind of equivalent to writing your original loop using a break statement.

Yes, its ugly and verbose but ints the only way to do it in raw Javascript. If you want to write a more structured version that looks like a loop instead of something with tons of gotos, have a look at one of the async control flow libraries or one of the compilers that compiles extensions of Javaascript with async support back into regular JS with callbacks.