The video track in the MP4 uses a codec browsers can't play in the video element (MPEG-4 Visual/MPEG-4 Part 2). I imagine when you enter the URL directly into the browser IE is using Windows Media Player or some other plugin to play it. Other browsers can play the ogg version.
You should re-encode the MP4 using the h264 codec.
mediainfo video.mp4
General
Complete name : video.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom
File size : 27.4 MiB
Duration : 2mn 25s
Overall bit rate : 1 581 Kbps
Writing application : Lavf52.31.0
Video
ID : 1
Format : MPEG-4 Visual
Format profile : Simple@L1
Format settings, BVOP : No
Format settings, QPel : No
Format settings, GMC : No warppoints
Format settings, Matrix : Default (H.263)
Codec ID : 20
Duration : 2mn 25s
Bit rate : 1 580 Kbps
Width : 256 pixels
Height : 256 pixels
Display aspect ratio : 1.000
Frame rate mode : Constant
Frame rate : 10.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Compression mode : Lossy
Bits/(Pixel*Frame) : 2.412
Stream size : 27.4 MiB (100%)
Writing library : Lavc52.20.1