How do I achieve non-linear non-dependent control flow using Promises (in server-side ES6)

Question

Coming over from the Java world, I am having trouble translating a multi-threaded approach to IO to the ES6 Promises concept of aysnc IO. Many of the examples I have seen on promises show a linear flow.

promiseFunction
    .then(functionThatReturnsAnotherPromise)
    .then(functionThatReturnsYetAnotherPromise)
    ...

The examples that show a non-linear flow show functions where I must have all the promises complete before I move on:

Promise.all([functionThatReturnsOnePromise, functionThatReturnsAnotherPromise])
    .then(functionThatUsesBothReturnValues);

What I am trying to do is a tree-based control flow where each branch has no dependency on another branch. Consider this chart showing my control flow:

• (1) Make an async REST request to get projects
• (2) 5 projects received. For each project received, (a) create a project instance and (b) make an async REST request to get teams of that project
• (3-7) x Teams received. For each team received, (a) create a team instance, (b) add the team as a member to its parent project instance, and (c) make a REST call to get all team-members of that team
• (8-21) x team-members received. For each team-member, (a) create a team-member instance and (b) add it as a member to its parent team instance.

What is important to note here is that in order for 8-21 to happen, 3-7 don't all have to be done. Basically, what I am trying to achieve here is once 2's response is received, do 3, 4, 5, 6, and 7. As soon as 3 is done (not caring about 4-7), do 8-10.

However, I am not sure which Promise constructs to achieve this tree-like control flow, since 8-21 is not all dependent on 3-7's completion.

getAllProjects()
    .then(function(responseJson) {
        var promises = [];
        for (var index in responseJson.value) {
            var project = new Project(responseJson.value[index]);
            promises.push(getTeamsForProject(project));
        }
        return promises;
    })
    .thenAsEachPromiseCompletes(function(responseJson) {
        var promises = [];
        for (var index in responseJson.value) {
            var team = new Team(responseJson.value[index]);
            promises.push(getTeamMembersForTeam(team));
        }
        return promises;
    })
    .thenAsEachPromiseCompletes(function(responseJson) {
        ...
    });

Answer 1

The main problem seems to be where to branch and where to join asynchronous executions. In your sample code you're trying to use a single point to join non-dependent operations. The trick here is to use many branches and joins; say you have an array of projects, and load teams. Then you have a list of promises, and for each you will start a new branch and have a new promise. When chained correctly, things start to clear up and look simpler:

getAllProjects().then(function(projects) {
    return Promise.all(projects.map(function (project) {
        return getTeamsForProject(project).then(function (teams) {
            return Promise.all(teams.map(function (team) {
                project.addTeam(team);
                return getTeamMembers(team).then(function (members) {
                    // ...
                });
            }));
        }).then(function () {
            return project;
        });
    }));
}).then(function (projects) {
    // all done
});

Which is a little messy, but can always be refactored to something cleaner like this:

getAllProjects().then(function(projects) {
    return Promise.all(projects.map(fetchProjectTeams));
}).then(function (projects) {
    // all done
});

function fetchProjectTeams(project) {
    return getTeamsForProject(project).then(function (teams) {
        return addProjectTeams(teams, project);
    }).then(function () {
        return project;
    });
}

function addProjectTeams(teams, project) {
    return Promise.all(teams.map(function (team) {
        project.addTeam(team);
        return fetchTeamMembers(team);
    }));
}

function fetchTeamMembers(team) {
    return getTeamMembers(team).then(function (members) {
        // ...
    });
}

Now in the last example, this complex operation is broken down into multiple comprehensible steps.