MongoDB Aggregate Query Equivalent To PostsgreSQL
-
12-12-2019 - |
Question
This question has two parts. The collection structure is:
_id: MongoID,
agent_id: string,
result: string,
created_on: ISO DATE,
...other fields...
Part one:
Desired Output: One result for each agent_id and result combination with a count: TUPLE representation with Equivalent SQL using PostgreSQL.
( "1234", "Success", 4 ),
( "1234", "Failure", 4 ),
( "4567", "Success", 3 ),
( "7896", "Failure", 2 ),
.....
SELECT agent_id, result, count(*)
FROM table
GROUP BY agent_id, result
HAVING created_on >= now()::date;
I have come up with the below mongo query....I think I have a conceptual or syntax error. The docs say to use $match early in the pipeline:, but although the $match limits the query when I run it by itself, as soon as I add the $group I get way to many results. Also I can't seem to understand how to group by more than one field. How can I edit the below query to get results like the SQL query above?
db.collection.aggregate(
{ $match :
{ created_on:
{ $gte: new Date('08-13-2012') //some arbitrary date
}
}, $group:
{ _id:"$agent_id" },
$project:
{_id:0, agent_id:1, result:1}
})
Part 2) The first result set would be adequate, but not optimal. With PostgreSQL I can achieve a result set like:
( "1234", { "Success", "Failure" }, { 4, 3 } ),
( "4567", { "Success", "Failure" }, { 3, 0 } ),
( "7896", { "Success", "Failure" }, { 0, 2 } )
I can do this in Postgresql with the array data type and a set_to_array function (custom function). The Pg specific SQL is:
SELECT agent_id, set_to_array(result), set_to_array( count(*) )
FROM table
GROUP BY agent_id, result
HAVING created_on >= now()::date;
I believe the equivalent data structure in mongodb would look like :
[
{ "1234", [ { "success": 4 }, { "failure": 4 } ] },
{ "4567", [ { "success": 3 }, { "failure": 0 } ] },
{ "7896", [ { "success": 0 }, { "failure": 0 } ] }
]
Is it possible to achieve these desired compressed results with mongodb aggregate framework ?
Solution
Here you go:
Created some test data:
db.test.insert({agent_id:"1234", result:"Failure", created_on:new Date()}); db.test.insert({agent_id:"1234", result:"Success", created_on:new Date()}); db.test.insert({agent_id:"1234", result:"Failure", created_on:new Date()}); db.test.insert({agent_id:"1234", result:"Success", created_on:new Date()}); db.test.insert({agent_id:"1234", result:"Failure", created_on:new Date()}); db.test.insert({agent_id:"1234", result:"Success", created_on:new Date()}); db.test.insert({agent_id:"1234", result:"Success", created_on:new Date()}); db.test.insert({agent_id:"1324", result:"Success", created_on:new Date()}); db.test.insert({agent_id:"1324", result:"Success", created_on:new Date()}); db.test.insert({agent_id:"1324", result:"Success", created_on:new Date()}); db.test.insert({agent_id:"1324", result:"Success", created_on:new Date()}); db.test.insert({agent_id:"1324", result:"Failure", created_on:new Date()}); db.test.insert({agent_id:"1324", result:"Failure", created_on:new Date()}); db.test.insert({agent_id:"1324", result:"Failure", created_on:new Date()}); db.test.insert({agent_id:"1324", result:"Failure", created_on:new Date()}); db.test.insert({agent_id:"1324", result:"Failure", created_on:new Date()}); db.test.insert({agent_id:"1324", result:"Failure", created_on:new Date()}); db.test.insert({agent_id:"1324", result:"Failure", created_on:new Date()});
db.test.aggregate(
{
$match:{ /* filter out the things you want to aggregate */
created_on:{$gte:new Date(1000000)}
}
},
{
$group: {_
_id: { /* the things you want to group on go in the _id */
agent_id:"$agent_id",
result:"$result"
},
count:{$sum:1} /* simple count */
}
},
{
$project: { /* take the id out into the separate fields for your tuple. */
_id:0,
agent_id:"$_id.agent_id",
result:"$_id.result",
count:"$count"
}
});
Gives:
{
"result" : [
{
"count" : 7,
"agent_id" : "1324",
"result" : "Failure"
},
{
"count" : 4,
"agent_id" : "1324",
"result" : "Success"
},
{
"count" : 4,
"agent_id" : "1234",
"result" : "Success"
},
{
"count" : 3,
"agent_id" : "1234",
"result" : "Failure"
}
],
"ok" : 1
}
Adding part 2--pretty similar to part 1, but the counting is a bit more complicated; basically you count only if it matches what you want to count:
db.test.aggregate(
{
$match: {
created_on: {$gte:new Date(1000000)}
}
},
{
$group: {
_id: {
agent_id:"$agent_id"
},
failure: {
$sum:{
$cond:[
{$eq:["$result","Failure"]},
1,
0
]
}
},
success: {
$sum: {
$cond:[
{$eq:["$result","Success"]},
1,
0
]
}
}
}
},
{
$project: {
_id: 0,
agent_id: "$_id.agent_id",
failure: "$failure",
success: "$success"
}
});
Gives:
{
"result" : [
{
"failure" : 7,
"success" : 4,
"agent_id" : "1324"
},
{
"failure" : 3,
"success" : 4,
"agent_id" : "1234"
}
],
"ok" : 1
}