I came up with a solution that solves most of my problems. What I ended up doing was using the ObjectContext API and in my model and making the navigation property that was taking a long time to access private (for getting and setting). You can do this in the edmx file by right clicking on the nav property.
Then I made a partial class file for the class that contained the private navigation property and added something along the lines of this.
public ObjectQuery<NavPropType> NavPropName
{
get
{
if(privateNavProp != null) //in case lazy loading is disabled or something
return privateNavProp.CreateSourceQuery();
else
return null;
}
}
Now no user of the class can accidentally try to pull in all of the navprop items, which there are a lot of. And it's also easy to make a query on the nav prop instead of having to remember to call CreateSourceQuery every time.
I didn't add a setter because that navigation property is read only in my application. I'm sure there's a way to make one that fits in with this pattern, but I don't know enough about the ObjectContext API to say how to do it.
I only ended up doing this for the navigation properties that could have a ton of data behind them because leaving the other ones as is had no effect on performance.
EDIT: One drawback of making it private that I ran into later is that I could no longer do
db.EntryTable.OrderBy(e => e.privateNavProp.Count())
even though it would have been smart enough to not get all of the Entities