To help you debug I strongly suggest using the following, recommended by the MSDN blog.
document.getElementsByTagName("video")[0].error.code
MEDIA_ERR_ABORTED : 1
The fetching process for the media resource was aborted by the user.
MEDIA_ERR_DECODE : 3
An error has occurred in the decoding of the media resource, after the resource was established to be usable.
MEDIA_ERR_NETWORK : 2 A network error has caused the user agent to stop fetching the media resource, after the resource was established to be usable
MEDIA_ERR_SRC_NOT_SUPPORTED : 4 The media resource specified by src was not usable.
We can't really debug this for you if we don't have the code you are using, or the files you are trying to display. The above, however, should be able to tell you what is going wrong.
== Edit to extend ==
If you are having issue #3, you are likely breaking IE's memory management. What is likely happening is that IE is attempting to pre-load the videos into memory and then crashes when resources run out. You can fix this by destroying the video elements as they are no longer needed, and turning off preloading.