Deferred execution the in repository pattern

Question

In my UserRepository I have a GetActive method:

public IEnumerable<User> GetActive()
{
    var users = context.Set<UserTbl>().Where(x => x.IsActive);

    foreach(var user in users)
    {
        yield return entityMapper.CreateFrom(user);
    }
}

The entityMapper is used to map from an EF-generated UserTbl to the User domain entity.

There exists thousands of users so I want the GetActive method to defer execution while returning the IEnumerable<User> so that the entire list isn't pulled unnecessarily. I have done that above with the foreach and yield.

When testing, it seems that the all the data is being fetched regardless. The following two calls take the same time:

// Get only 5 users in memory
var someUsers = UserRepository.GetActive().Take(5).ToList();

// Get all 100,000 users into memory
var allUsers = UserRepository.GetActive().ToList();

What am I doing wrong?

Answer 1

The moment you use foreach, the data is enumerated. You have to use IQueryable only until the ToList method. Your idea about deferring data using IEnumerable seem nice, but it is wrong. IEnumerable always returns all data, it just doesn't force the provider of the data to hold it all in memory. You have to use IQueryable if you want to provider to return pieces of data. But then, you can't use foreach and yield, because it always enumerates all data in it's parameter. Only way to do what you want is to pass the required query into the GetActive method.

public IEnumerable<User> GetActive(Func<IQueryable<User>, IQueryable<User>> modifier)
{
    var users = modifier(context.Set<UserTbl>().Where(x => x.IsActive));

    foreach(var user in users)
    {
        yield return entityMapper.CreateFrom(user);
    }
}

// Get only 5 users in memory
var someUsers = UserRepository.GetActive(q=>q.Take(5)).ToList();

// Get all 100,000 users into memory
var allUsers = UserRepository.GetActive(q=>q).ToList();

But I would really recommend not having repositories in your architecture at all. They introduce needless complexity over already complex ORM. See more in Repository pattern with Entity framework