The sorting defines the order in which your items are inserted in the layout. However, the actual positions are the result of the masonry algorithm. Now here lies the inherent problem: An ordered layout and a space-efficient layout are two orthogonal things.
As a consequence, you can't have a guaranteed ordering and masonry at the same time. You have to decide, whether you can drop one of the requirements. If you don't really need a guarantee for the ordering, you can still get away with your solution, because the ordering acts as some kind of priority (this means early items have a good chance to appear early in the layout).