Javascript Data Layer Architecture Assistance

Question

I'm making a fairly complex HTML 5 + Javascript game. The client is going to have to download images and data at different points of the game depending on the area they are at. I'm having a huge problem resolving some issues with the Data Layer portion of the Javascript architecture.

The problems I need to solve with the Data Layer:

1. Data used in the application that becomes outdated needs to be automatically updated whenever calls are made to the server that retrieve fresh data.
2. Data retrieved from the server should be stored locally to reduce any overhead that would come from requesting the same data twice.
3. Any portion of the code that needs access to data should be able to retrieve it easily and in a uniform way regardless of whether the data is available locally already.

What I've tried to do to accomplish this is build a data layer that has two main components: 1. The portion of the layer that gives access to the data (through get* methods) 2. The portion of the layer that stores and synchronizes local data with data from the server.

The workflow is as follows:

When the game needs access to some data it calls get* method in the data layer for that data, passing a callback function.

bs.data.getInventory({ teamId: this.refTeam.PartyId, callback: this.inventories.initialize.bind(this.inventories) });

The get* method determines whether the data is already available locally. If so it either returns the data directly (if no callback was specified) or calls the callback function passing it the data.

If the data is not available, it stores the callback method locally (setupListener) and makes a call to the communication object passing the originally requested information along.

getInventory: function (obj) {

    if ((obj.teamId && !this.teamInventory[obj.teamId]) || obj.refresh) {
        this.setupListener(this.inventoryNotifier, obj);
        bs.com.getInventory({ teamId: obj.teamId });
    }
    else if (typeof (obj.callback) === "function") {
        if (obj.teamId) {
            obj.callback(this.team[obj.teamId].InventoryList);
        }
    }
    else {
        if (obj.teamId) {
            return this.team[obj.teamId].InventoryList;
        }
    }
}

The communication object then makes an ajax call to the server and waits for the data to return.

When the data is returned a call is made to the data layer again asking it to publish the retrieved data.

getInventory: function (obj) {
    if (obj.teamId) {
        this.doAjaxCall({ orig: obj, url: "/Item/GetTeamEquipment/" + obj.teamId, event: "inventoryRefreshed" });
    }
},
doAjaxCall: function (obj) {

    var that = this;

    if (!this.inprocess[obj.url + obj.data]) {
        this.inprocess[obj.url + obj.data] = true;
        $.ajax({
            type: obj.type || "GET",
            contentType: "application/json; charset=utf-8",
            dataType: "json",
            data: obj.data,
            url: obj.url,
            async: true,
            success: function (data) {
                try {
                    ig.fire(bs.com, obj.event, { data: data, orig: obj.orig });
                }
                catch (ex) {
                    // this enables ajaxComplete to fire
                    ig.log(ex.message + '\n' + ex.stack);
                }
                finally {
                    that.inprocess[obj.url + obj.data] = false;
                }
            },
            error: function () { that.inprocess[obj.url + obj.data] = false; }
        });
    }
}

The data layer then stores all of the data in a local object and finally calls the original callback function, passing it the requested data.

publishInventory: function (data) {

    if (!this.inventory) this.inventory = {};

    for (var i = 0; i < data.data.length; i++) {
        if (this.inventory[data.data[i].Id]) {
            this.preservingUpdate(this.inventory[data.data[i].Id], data.data[i]);
        }
        else {
            this.inventory[data.data[i].Id] = data.data[i];
        }
    }

    // if we pulled this inventory for a team, update the team
    // with the inventory
    if (data.orig.teamId && this.team[data.orig.teamId]) {
        this.teamInventory[data.orig.teamId] = true;
        this.team[data.orig.teamId].InventoryList = [];
        for (var i = 0; i < data.data.length; i++) {
            this.team[data.orig.teamId].InventoryList.push(data.data[i]);
        }
    }

    // set up the data we'll notify with
    var notifyData = [];

    for (var i = 0; i < data.data.length; i++) {
        notifyData.push(this.inventory[data.data[i].Id]);
    }

    ig.fire(this.inventoryNotifier, "refresh", notifyData, null, true);
}

There are several problems with this that bother me constantly. I'll list them in order of most annoying :).

1. Anytime I have to add a call that goes through this process it takes too much time to do so. (at least an hour)
2. The amount of jumping and callback passing gets confusing and seems very prone to errors.
3. The hierarchical way in which I am storing the data is incredibly difficult to synchronize and manage. More on that next.

Regarding issue #3 above, if I have objects in the data layer that are being stored that have a structure that looks like this:

this.Account = {Battles[{ Teams: [{ TeamId: 392, Characters: [{}] }] }]}
this.Teams[392] = {Characters: [{}]}

Because I want to store Teams in a way where I can pass the TeamId to retrieve the data (e.g. return Teams[392];) but I also want to store the teams in relation to the Battles in which they exist (this.Account.Battles[0].Teams[0]); I have a nightmare of a time keeping each instance of the same team fresh and maintaining the same object identity (so I am not actually storing it twice and so that my data will automatically update wherever it is being used which is objective #1 of the data layer).

It just seems so messy and jumbled.

I really appreciate any help.

Thanks

Answer 1

+1 for Backbone -- it does some great heavy lifting for you.

Also look at the Memoizer in Douglas Crockford's book Javascript the Good Parts. It's dense, but awesome. I hacked it up to make the memo data store optional, and added more things like the ability to set a value without having to query first -- e.g. to handle data freshness.

Answer 2

You should consider using jquery's deferred objects.

Example:

var deferredObject = $.Deferred();
$.ajax({
     ...
     success: function(data){
         deferredObject.resolve(data);
     }
});
return deferredObject;

Now with the deferredObject returned, you can attach callbacks to it like this:

var inventoryDfd = getInventory();
$.when(inventoryDfd).done(function(){
     // code that needs data to continue
}

and you're probably less prone to errors. You can even nest deferred objects, or combine them so that a callback isn't called until multiple server calls are downloaded.