ffmpeg streaming video website -- converting on the fly vs. hosting multiple formats

Question

Most websites/hosting platforms will keep the same video in multiple HTML5 video compatible formats (mp4 and webm as of today) for on demand delivery. Processing video on the fly requires CPU/GPU and that can cause penalty on the server load if too many people watch videos at the same time. Upstream transocding + storage/CDN is often less expensive and better for scalability.

Live video streaming is another story because you can get MPEG2 (or else) input and this often needs to be converted on the fly to multiple H264/AAC renditions either with ffmpeg or adds on like Wowza Media Server Transcoder.

EDIT:

why do sites still host Flash and non-Flash (i.e. HTML5) versions of videos?

Most do not and you do not need to. They use MP4 files to be delivered on both Flash and HTML5. You can find FLV or F4V (Adobe flavor of MP4) but they are either legacy, ads oriented, or flash only environment files (FLV can be used to store cue points for example to trigger animation at certain point of a video). Youtube for say mostly used MP4 files but you will find they deliver both flv and 3gp files (mobile) as well.

HTML5 is universal and can play on any client. Flash is restricted to PCs. By now they should have dumped Flash.

You can find on SO or Google articles describing the pros and cons of Flash vs. HTML5. Momentum on the market today is HTML5 first with a flash fallback. HTML5 video is "only" available in 80% of current browsers. You would understand why major players like Youtube or Brightcove just won't let go of the remaining 20% audience. Vimeo has made the switch for an HTML5 video first strategy. Flash still holds some grounds HTML5 video has not yet covered (or is only beginning to):

DRM
Ads: you can understand now why Youtube and others have not made the HTML5 video first switch just yet.
Adaptive streaming (required for better user experience with video): though MPEG DASH is coming fast and HLS will work on Apple devices (and some others), browser adoption is still narrow for HTML5 video. Flash has well established RTMP and HDS support.
No need for cross browser tweaking: HTML5 video can be inconsistent across browser/version (events not firing as expected, prefix vendor required ...). Flash on the other hand only needs one type of video file format and will work as expected in any browser with the flash plugin installed.
Cost for technology switch: in the long run you will save money with HTML5 video (no license to pay for if you want to develop HTML5 video - web developers can do HTML5 video development and they can be less expensive than Flash expert). Short term moving to an HTML5 video first strategy is asking for major updates of a technology - hence cost.

Flash does not play on iOS and Android - that is a clear win for HTML5 video and a major drawback for Flash. One growing caveat at the moment about flash is it being a plugin. That throws security and performance concerns. For example Microsoft has announced they will drop support for Silverlight in the coming years and is moving towards HTML5 video. Java (client side) is getting blocked more an more.

Not a good day to be a web plugin nowdays ...