I did it. Just create one more alias and specify the join type in both
final Criteria criteria = getSession().createCriteria(Product.class);
criteria.add(Restrictions.eq("id", searchFilter.getProductId()));
criteria.createAlias("attributes", "atrs", CriteriaSpecification.LEFT_JOIN);
criteria.createAlias("atrs.attribute", "atr", CriteriaSpecification.LEFT_JOIN);
criteria.addOrder(Order.asc("atr.name"));