You need to set up Cascade Deletion on your entities.
See Entity Framework 4.3 delete cascade with code first (Poco)
{
//...
modelBuilder.Entity<Parent>()
.HasMany(e => e.ParentDetails)
.WithOptional(s => s.Parent)
.WillCascadeOnDelete(true);
//...
One thing to note, when you delete the parent the related child entities in the context will issue a delete statement as well. For example, a parent with 1000 children will issue 1001 statements (1 delete for the parent and 1000 for the children). This is done to keep the entities in the context up to date.
To avoid extra child deletes, detach the children before SaveChanges and allow the foreign key in the database to do the delete. If you are doing code first, you'll need to add that foreign key relation.
See How Cascade Delete Really Works in EF for more info.