Cubism not plotting updated data on callbacks

Question

So my goal is to plot some real-time metrics that are stored in a backbone collection. I have a backbone collection that stores metrics, it's updated every second when I poll the backend server. My collection has an array of historical metrics and a field 'most-recent-data' which is the latest data to come from the backend. Every second, 'most-recent-data' is updated to a different value. I pass a reference of the backbone collection to the function that creates the cubism metrics.

When the view containing the Cubism.js view is rendered, it should do a historical load of all the data points in the stored in the collection to the horizon chart. This is the part I've had success on. However, when the metric function does a callback, it is not plotting new/correct points from the 'most-recent-data' field which is updated every second.

Here is my code that interacts with the DOM (I don't believe this part is causing a problem):

       var context = cubism.context()
            .serverDelay(0)
            .clientDelay(0)
            .step(250)
            .size(1116);


       //Getting the metrics from the backbone collection   
       var models = this.dataSet.models;
       console.log('models in metric view',models);
        //aggregate 'metrics'. Each component has a 'metric' which contains its stats over time
        for(model in models){
            var attributes = models[model].attributes;

            if(!attributes['name'] || attributes['type']== "FLOW" || attributes['type'] == "SERVER"){
                continue;
            }
            if(attributes['name'] == null){
                continue;
            }
            var name = attributes['name'];
            var type = attributes['type'];
            var serverName = attributes['serverName'];
            var metName = name.concat(serverName);
            console.log(metName);

            //Getting the cubism metric for each stat in the backbone collection
            //Passing in a reference to the current model in the backbone collection (this.dataSet.models[model])
            var curContext = getMetric(metName, this.dataSet.models[model]);

            statsList.push(curContext);

        }

        d3.select(this.loc).selectAll(".axis")
            .data(["top", "bottom"])
          .enter().append("div")
            .attr("class", function(d) { return d + " axis"; })
            .each(function(d) { d3.select(this).call(context.axis().ticks(12).orient(d)); });

        //create rule
        d3.select(this.loc).append("div")
            .style("position", "fixed")
            .style("top", 0)
            .style("bottom", 0)
            .style("width", "1px")
            .style("pointer-events", "none")
            .call(context.rule());

        d3.select(this.loc).selectAll(".horizon")
            .data(statsList)
          .enter().insert("div", ".bottom")
            .attr("class", "horizon")
            .call(context.horizon().height(35));

It's essentially iterating through all the stats in the backbone collection, then calling 'getMetric' passing in a reference to the current stat in the backbone collection. 'getMetric' returns a cubism metric.

Here's the code that handles each metric (probably causing the problem):

        /* Keep a map of intialized metrics 
        When the visualization is initialized, we load the historical data which is in the
        'attributes' field of the model. The model is an individual set metrics */

        var initializedMetrics = {};
        function getMetric(name, model){
            var format = d3.time.format("%I-%M-%S");
            return context.metric(function(start, stop, step, callback){
                var statValues = [];

                if(initializedMetrics[name]){
                    /* If the metric has already been initialized and we loaded the historical data
                    from 'attributes' field, plot the newest data which is stored in 'most-recent-data'
                    This field should be updated every second on every API call to the server */
                    while(start<stop){
                        start+=step;
                        console.log(model.attributes['most-recent-data']['rate']);
                        statValues.push(model.attributes['most-recent-data']['rate']);
                    }

                } else{
                    /* Metric has not been initalized, so we load all the historical data in 'all-data'
                    for this stat and plot it over time*/
                    initializedMetrics[name]=true;

                    var lookup = {},
                    i = start.getTime();
                    //console.log('startTime', i);
                    var curStat = null;
                    var metricData = model.attributes['all-data']
                    for(stat in metricData){
                        curStat = metricData[stat];
                        //console.log(name, curStat);
                        var curDate = new Date(curStat['timeStamp']);
                        curDate = format(curDate);
                        lookup[curDate] = curStat;
                    }
                    var lastValue;
                    while((i+=step) < stop){
                        start+=step;
                        var key = format(new Date(i));
                        if(key in lookup){
                            lastValue = lookup[key]['valueLong'];
                        }
                        var curVal = key in lookup ? lookup[key]['valueLong'] : lastValue;
                        //console.log(name,curVal);
                        statValues.push(curVal);
                    }
                }
                 /* Callback with statValues*/
                 callback(null,statValues);
                }, name);
        }

I'm having success loading all the historical data (everything inside the else bracket). However, when a metric is initialized, I try to load new data which is stored 'most-recent-data'. This field is updated every second with the API calls, but with each callback, it's ONLY plotting the initial data stored in 'most-recent-data'. I can confirm that most-recent-data is indeed being updated in the backbone collection itself, but the reference being passed to cubism is not being updated at each callback. When I log 'most-recent-data' in the cubism code, it never updates to the latest value.

Is this a closure issue? Am I missing some cubism syntax for polling new data? Or would I need to pass in the backbone collection reference differently? Any hints would be very helpful. Thank you.

Answer 1

Found a solution. Instead of merging new data in the backbone collection I was resetting it and adding each data point over again. This meant the models I originally passed in to cubism weren't the same as the ones being updated. Issue was fixed by merging the collection on after each call to server as opposed to resetting.