Spring Data MongoDB Aggregation framework, exception accessing computed value in group

Question

I have a collection of trips and every trip has a startDateTime and a completionDateTime property.

I'm trying to use the aggregation framework to find the average duration of a users' trips.

My aggregation seems simple enough and but it throws an exception.

There are 3 steps below, the first just matches the trips to a specific userId. In the second step of the pipeline I project the data I'm interested in along with the newly calculated duration of each trip.

I expected the duration from the projection to be available in the group step but I get the IllegalArgumentException below. Duration Is not a property on my Trip class but from the examples I've seen in the docs I don't think it needs to be. Duration is a property on my analytics class.

TypedAggregation<Trip> aggregation = newAggregation(Trip.class, 
    match(Criteria.where("userId").is(aUserId)),

    project("completionDateTime", "startDateTime", "userId")
    .and("completionDateTime").minus("startDateTime").as("duration"),

    group("userId").avg("duration").as("averageDuration")
);
AggregationResults<Analytics> result = mongoTemplate.aggregate(aggregation, Analytics.class);

I believe the duration is getting calculated correctly as if I leave out the group step I get back a collection of documents and they all have the duration set.

java.lang.IllegalArgumentException: Invalid reference 'duration'!
    at org.springframework.data.mongodb.core.aggregation.ExposedFieldsAggregationOperationContext.getReference(ExposedFieldsAggregationOperationContext.java:78)
    at org.springframework.data.mongodb.core.aggregation.GroupOperation$Operation.getValue(GroupOperation.java:367)
    at org.springframework.data.mongodb.core.aggregation.GroupOperation$Operation.toDBObject(GroupOperation.java:363)
    at org.springframework.data.mongodb.core.aggregation.GroupOperation.toDBObject(GroupOperation.java:308)
    at org.springframework.data.mongodb.core.aggregation.Aggregation.toDbObject(Aggregation.java:247)

I'm using 1.3.4.RELEASE

Any Suggestions?

Answer 1

After upgrading to 1.4.1 I was able to get the desired result using:

.andExpression("completionDateTime - startDateTime").as("duration")

Answer 2

It looks like you are running into https://jira.spring.io/browse/DATAMONGO-838 so until it's fixed you may need to figure out a different way to do what you need.

In mongo shell using js you don't need to have a $project field, you can just substitute in the computed expression into the group directly like so:

db.collection.aggregate( {$match:{ your-match } },
     { $group : { 
         _id : "$userId",
         avgDuration : { $avg : {$subtract:["$completionDateTime","startDateTime"]}},
     } } );

It might be possible to return the expression to do it this way in Spring. If not, your other option is to pass the aggregation directly to MongoDB through the Java driver, bypassing Spring until the bug is fixed.