First of all, Chrome should be able to read the mp4 so perhaps you didn't use h264 compression. You should check on that.
Second, you must include compressed ogv, and webm versions of the video. Each browser supports a different video format at the moment so we have to include them all so that no one gets left out.
Here is a handy chart for browser video format support : http://www.longtailvideo.com/html5/#media_formats
Don't forget the flash fall-back either, just incase someone with a dinosaur browser that doesn't support the html5 video tag needs to see the video too.
Here is a handy chart for video tag support : http://caniuse.com/video
Let us know how you get on.