How to do group.reduce in a flat data in crossfilter

Question

New to crossfilter. I've a flat data which is given below:

 id,name,patientId,conditionId,isPrimary,age,gender,race,Status,CGI
 1,M1,1,c1,Y,33,Male,White,Discharged,0
 2,M2,1,c1,N,33,Male,White,Discharged,0
 3,M3,1,c2,N,33,Male,White,Discharged,0
 4,M4,1,c2,N,33,Male,White,Discharged,0
 5,M5,1,c3,N,33,Male,White,Discharged,0
 6,M6,1,c3,N,33,Male,White,Discharged,0
 25,M1,5,c1,Y,33,Male,White,Discharged,1
 26,M7,5,c2,N,33,Male,White,Discharged,1
 27,M4,5,c4,N,33,Male,White,Discharged,1
 28,M4,5,c1,N,33,Male,White,Discharged,1
 29,M4,5,c2,N,33,Male,White,Discharged,1
 30,M5,5,c4,N,33,Male,White,Discharged,1
 29,M2,6,c1,Y,33,Male,White,Discharged,1
 30,M2,7,c1,Y,33,Male,White,Discharged,1

I want to do a count on conditionId but since there are multiple records belonging to the same person as identified by patientId, the count of value c1 should be 4 (belonging to patientId 1, 5, 6, 7) - because same patient may have multiple records (for eg. patientId of 1 is repeated 6 times and two of them has c1 which should be counted only once) . I'm struggling to write a group.reduce on conditionId but could not even start.

Thanks in advance.

Answer 1

Here's one way of doing it. In the example I assumed that the first value was the patientId and the second value the conditionId. The code keeps track of grouping keys (concatenation of the patientId and the conditionId) that were seen already and ignores them.

var countMap = [
    [1, 'c1'],
    [1, 'c1'],
    [2, 'c1'],
    [2, 'c2']
].reduce(function (r, v) {
    var condition = v[1],
        groupKey = v[0] + condition;

    if (!r.seen[groupKey]) {
        r.seen[groupKey] = true; 
        r.count[condition] = (r.count[condition] || 0) + 1;
    }

    return r;

}, {seen: {}, count: {}}).count;


countMap.c1; //2
countMap.c2; //1

I do not know about crossfilter or dc.js, that's why I gave you a vanilla JS solution.

Answer 2

It's a little complicated to do this in Crossfilter, but the solution is similar to that provided by @plalx.

Here is a helper function I am using in one of my projects. It's not perfect, and is a bit optimized to reduce dictionary lookups, so it's not the most readable. The basic idea is you need to keep a dictionary of values seen before for each group. You only need to remember patients, because the condition is already known based on the group your are in:

function reduceHelper(accessorFunction) {
        var internalCount;
        return {
            add: function (p, v) {
                if(p.unique.has(accessorFunction(v))) {
                    internalCount = p.unique.get(accessorFunction(v));
                    p.unique.set(accessorFunction(v), internalCount + 1);
                } else {
                    p.unique.set(accessorFunction(v), 1);
                    ++p.count;
                }
                return p;
            },
            remove: function (p, v) {
                if(p.unique.has(accessorFunction(v))) {
                    internalCount = p.unique.get(accessorFunction(v));
                    if(internalCount == 1) {
                        p.unique.remove(accessorFunction(v));
                        --p.count;
                    } else {
                        p.unique.set(accessorFunction(v), internalCount - 1);
                    }
                }
                return p;
            },
            init: function () {
                return {unique: d3.map(), count: 0};
            }
        };
    }

You'll need to create a Crossfilter (xfilter) on your data and then:

var helperFunctions = reduceHelper(function(d) { return d.patientId; });
var dim = xfilter.dimension(function (d) { return d.conditionId; };
var group = dim.group()
              .reduce(helperFunctions.add, helperFunctions.remove, helperFunctions.init);

Your group will now count the number of patients that have each condition. If a condition appears more than once for a given patient, that patient will still only be counted once. At least, it will if my solution works properly :-)