You are changing the collection while using for each to loop through it; that is asking for trouble: once the foreach is started and acquired the enumerator the enumerator is tied to the collection as it was at the start.
One way to solve this is by first looping and collecting a list of controls to be deleted.
Then loop the list and remove these controls.
Another way is to use for
which doesn't create an enumerator.
Note that your code will not work if a control is nested within another control.