Aggregate data by timestamped date in mongodb

Question

I'm storing created time for item in unix timestamp format. And now i'd like to aggregate it, according to year, for example. But, unfortunately, it does not work, as should...

Here is source data:

db.invoice.find({id: '01f96c0d4254cdd69692d97c3909fb6dff75066b'}, {created: 1, amount: 1})
{ "_id" : ObjectId("534801e8e71709fab334bdd9"), "amount" : 345, "created" : 1397661552 }

But when i try to do:

db.invoice.aggregate(
{$match: {id: '01f96c0d4254cdd69692d97c3909fb6dff75066b'}}, 
{$group: 
    {_id: {  year: {$year: new Date('$created' * 1000)}  }, 
    total: {$sum: '$amount'}}
}
)

I get

"result" : [
    {
        "_id" : {
            "year" : 292278994
        },
        "total" : 345
    }
],
"ok" : 1

The year is strange...

P.s. If i point exact time in Date function:

db.invoice.aggregate(
{$match: {id: '01f96c0d4254cdd69692d97c3909fb6dff75066b'}}, 
{$group: 
    {_id: {  year: {$year: new Date(1397661552000)}  }, 
    total: {$sum: '$amount'}}
}
)

it seems to work and outputs year 2014

Answer 1

The $year operator and all of the date aggregation operators work exactly as designed. But the problem here is that your "created" is not actually a BSON date type and is actually just a number representing an epoch timestamp.

So if you actually had a BSON date which in the shell would look like this:

ISODate("2014-04-18T03:54:40.023Z")

Then the operators will work. But in your case they do not, which is a shame because the BSON date type is actually just an epoch timestamp internally and so would not take up any additional storage.

But since your "timestamps" are just numbers, all you need to do is a little "date math" in order to get the year boundaries that you want:

db.invoice.aggregate([
    { "$group": {
        "_id": {
            "$add": [
                { "$subtract": [
                    "$created",
                    { "$mod": [
                        "$created",
                        31536000         // 60 * 60 * 24 * 365
                    ]}
                ]},
                950400                   // correction
            ]
        },
        "total": { "$sum": "$amount" }
    }}
])

And that should set each date value as the first day of the year it falls in so you can then aggregate by year.

"result" : [
    {
        "_id" : {
            "year" : 1388534400
        },
        "total" : 345
    }
],
"ok" : 1

You can check the epoch date in the result at at site like epoch converter or otherwise just feed it into a Date object.