As I understand it, the virtual keyword is required to allow Entity Framework to make proxies of your classes, which assist in lazy loading situations.
I don't know the details of the proxy variant, but I'm guessing it brings back and stores child ids with the parent. Then, it you subsequently access child attributes (other than the id) it has an efficient way of fetching the child records.
I have had trouble with navigational properties on the parent, and left them out in favor of explicitly loading the child records when needed. However, my keys are two part (id and effective date) which may be the cause.