I figured it out.
jobPortalInfos is Linq to Objects, but EmailRepository is still an IQueryable/Linq To Entites. It doesn't know how to convert the LinqToObject jobPortalInfos to a Sql Server object.
I ended up doing this:
List<JobPortalInfo> jobPortalInfos = uow.JobPortalInfoRepository.GetQuery()
.Where(x => x.Job.Id == emailHash.JobId)
.ToList();
List<long> contactIds = jobPortalInfos
.Where(x => x.Contact != null)
.Select(y => y.Contact.Id)
.ToList();
var temp = uow.EmailRepository.GetQuery()
.Where(x => contactIds.Contains(x.ContactId))
.ToList();
Not sure if there is a more concise way to do this or not, but it seems to work.