MongoDB Java client - why does `sort` seem to break my query?

Question

I have not been able to make a query work when using sort. I expect the results of a query to be exactly the same as if I were not using sort, except the results should come in sorted, of course, but what happens is that when using sort I get nothing back.

Here's a complete example to reproduce the problem:

DB db = fongo.getDB( "something" );
DBCollection collection = db.getCollection( "what" );
collection.insert( new BasicDBObject( "hello", 4 ) );
collection.insert( new BasicDBObject( "hello", 2 ) );
collection.insert( new BasicDBObject( "hello", 1 ) );
collection.insert( new BasicDBObject( "hello", 3 ) );

final DBCursor sorted = collection
    .find( new BasicDBObject( "hello", new BasicDBObject( "$exists", true ) ) )
    .sort( new BasicDBObject( "hello", 1 ) )
    .limit( 10 );

final DBCursor notSorted = collection
    .find( new BasicDBObject( "hello", new BasicDBObject( "$exists", true ) ) )
            .limit( 10 );

// both asserts below work!
assertThat( notSorted.size(), is( 4 ) );
assertThat( sorted.size(), is( 4 ) );

List<DBObject> notSortedAsList = notSorted.toArray();
List<DBObject> sortedAsList = sorted.toArray();

assertThat( notSortedAsList.size(), is( 4 ) );
assertThat( sortedAsList.size(), is( 4 ) ); // << BREAKS HERE!!!!
assertThat( sortedAsList.stream().map( obj -> obj.get( "hello" ) )
    .collect( Collectors.toList() ), is( Arrays.asList( 1, 2, 3, 4 ) ) );

As you can see, the notSortedAsList list contains 4 elements, as expected, but sortedAsList is empty!! The only difference being that the latter was created from a query containing sort.

Unless I am doing something wrong, it seems that this could be a bug in the MongoDB Java driver, even though it could also be related to Fongo as I am using it to test this.

Any ideas about what's happening??

EDIT

This is what is generated by the query containing sort shown above:

find({ "query" : { "hello" : { "$exists" : true}} , "orderby" : { "hello" : 1}}, null).skip(0).limit(10)

Without sort, the query looks like this:

find({ "hello" : { "$exists" : true}}, null).skip(0).limit(10)

I have also tried doing the following query:

final DBCursor sorted = collection
    .find( new BasicDBObject( "hello", new BasicDBObject( "$exists", true ) ) )
    .addSpecial( "$orderby", new BasicDBObject( "hello", 1 ) )
    .limit( 10 );

The generated query then is:

find({ "$orderby" : { "hello" : 1} , "query" : { "hello" : { "$exists" : true}}}, null).skip(0).limit(10)

Both have the same result, though the first uses orderby and the second uses $orderby (as suggested here: http://docs.mongodb.org/manual/reference/operator/meta/orderby/#op._S_orderby)

Answer 1

Well, I found the problem. As I mentioned in a comment on the question, when you call sort the MongoDB driver starts using a meta-query operators rather than just a simple object query, as you would usually do.

For example, the simple query, without sorting, would look like this:

find({ "hello" : { "$exists" : true}}, null)

The above will find any document that contains a field called "hello".

However, when you use sort, the query looks like this:

find({ "query" : { "hello" : { "$exists" : true}} , "orderby" : { "hello" : 1}}, null)

This is WRONG because it will match a document containing a field called "query" with the above value, as well as a field called "orderby"...

The correct query should be:

find({ "$orderby" : { "hello" : 1} , "$query" : { "hello" : { "$exists" : true}}}, null)

The only difference is the $ that makes the keys meta-operators.

To confirm this hypothesis, I modified my code to use addSpecial (which allows me to add meta-operators to the query) as well as tried to "cancel" the automatically-generated (and wrong) query, as follows:

final DBCursor sorted = collection
    .find( )
    .addSpecial( "$orderby", new BasicDBObject( "hello", 1 ) )
    .addSpecial( "$query", new BasicDBObject( "hello", new BasicDBObject( "$exists", true ) ) )
    .limit( 10 );

Because this will still keep the wrong "query" operator in the query, I had to add a query field to my documents to make it finally work:

collection.insert( new BasicDBObject( "hello", 4 ).append( "query", 1 ) );
...

Now all the tests above pass!

Clearly, this is a horrible hack and hopefully the MongoDB team will fix the Java client ASAP. In the mean time, I can get this to work at least.

I have tried to issue a bug in the Java Driver JIRA but I couldn't find out how to sign up.

EDIT 2

Actually, just found out I could sign up and created a ticket: JAVA-1176

** EDIT 3 **

Just found out with the MongoDB team that this problem does not exist anymore with the most recent version of Fongo (1.4.5 at the time of writing) and the Java driver vesion 2.11.4.

Unfortunately, we cannot use the latest version (2.12.0) of the Java driver yet because of compatibility issues, but the problem will not occur with the above versions!