Real Time Streaming to HTML5 (with out webrtc) just using video tag

Question

I would like to wrap real time encoded data to webm or ogv and send it to an html5 browser.

Can webm or ogv do this, Mp4 can not do this due to its MDAT atoms. (one can not wrap h264 and mp3 in real time and wrap it and send it to the client) Say I am feeding the input from my webcam and audio from my built in mic. Fragmented mp4 can handle this but its an hassle to find libs to do that).

I need to do this cuz I do not want to send audio and video separably.

If I did send it separably, sending audio over audio tag and video over video>(audio and video are demuxed and sent) Can I sync them on client browser with javascript. I saw some examples but not sure yet.

Answer 1

Evren,

Since you have asked this question initially, the Media Source Extensions https://www.w3.org/TR/media-source/ have matured enough to be able to play very short (30ms) ISO-BMFF video/mp4 segments with just a little buffering.

Refer to HTML5 live streaming

So your statement

(one can not wrap h264 and mp3 in real time and wrap it and send it to the client)

is out of date now. Yes you can do it with h264 + AAC.

There are several implementations out there; take a look at Unreal Media Server. From Unreal Media Server FAQ: http://umediaserver.net/umediaserver/faq.html

How is Unreal HTML5 live streaming different from MPEG-DASH? Unlike MPEG-DASH, Unreal Media Server uses a WebSocket protocol for live streaming to HTML5 MSE element in web browsers. This is much more efficient than fetching segments via HTTP requests per MPEG-DASH. Also, Unreal Media Server sends segments of minimal duration, as low as 30 ms. That allows for low, sub-second latency streaming, while MPEG-DASH, like other HTTP chunk-based live streaming protocols, cannot provide low latency live streaming.

Their demos webpage has a live HTML5 feed from RTSP camera: http://umediaserver.net/umediaserver/demos.html Notice that the latency in HTML5 player is comparable to that in Flash player.

Answer 2

I did this with ffmpeg/ffserver running on Ubuntu as follows for webm (mp4 and ogg are a bit easier, and should work in a similar manner from the same server, but you should use all 3 formats for compatibility across browsers).

First, build ffmpeg from source to include the libvpx drivers (even if your using a version that has it, you need the newest ones (as of this month) to stream webm because they just did add the functionality to include global headers). I did this on an Ubuntu server and desktop, and this guide showed me how - instructions for other OSes can be found here.

Once you've gotten the appropriate version of ffmpeg/ffserver you can set them up for streaming, in my case this was done as follows.

On the video capture device:

ffmpeg -f video4linux2 -standard ntsc -i /dev/video0 http://<server_ip>:8090/0.ffm

• The "-f video4linux2 -standard ntsc -i /dev/video0" portion of that may change depending on your input source (mine is for a video capture card).

Relevant ffserver.conf excerpt:

Port 8090
#BindAddress <server_ip>
MaxHTTPConnections 2000
MAXClients 100
MaxBandwidth 1000000
CustomLog /var/log/ffserver
NoDaemon

<Feed 0.ffm>
File /tmp/0.ffm
FileMaxSize 5M
ACL allow <feeder_ip>
</Feed>
<Feed 0_webm.ffm>
File /tmp/0_webm.ffm
FileMaxSize 5M
ACL allow localhost
</Feed>

<Stream 0.mpg>
Feed 0.ffm
Format mpeg1video
NoAudio
VideoFrameRate 25
VideoBitRate 256
VideoSize cif
VideoBufferSize 40
VideoGopSize 12
</Stream>
<Stream 0.webm>
Feed 0_webm.ffm
Format webm
NoAudio
VideoCodec libvpx
VideoSize 320x240
VideoFrameRate 24
AVOptionVideo flags +global_header
AVOptionVideo cpu-used 0
AVOptionVideo qmin 1
AVOptionVideo qmax 31
AVOptionVideo quality good
PreRoll 0
StartSendOnKey
VideoBitRate 500K
</Stream>

<Stream index.html>
Format status
ACL allow <client_low_ip> <client_high_ip>
</Stream>

• Note this is configured for a server at feeder_ip to execute the aforementioned ffmpeg command, and for the server at server_ip so server to client_low_ip through client_high_ip while handling the mpeg to webm conversation on server_ip (continued below).

This ffmpeg command is executed on the machine previously referred to as server_ip (it handles the actual mpeg --> webm conversion and feeds it back into the ffserver on a different feed):

ffmpeg -i http://<server_ip>:8090/0.mpg -vcodec libvpx http://localhost:8090/0_webm.ffm

Once these have all been started up (first the ffserver, then the feeder_ip ffmpeg process then then the server_ip ffmpeg process) you should be able to access the live stream at http://:8090/0.webm and check the status at http://:8090/

Hope this helps.

Answer 3

Not 100% sure you can do this. HTML5 has not ratified any live streaming mechanism. You could use websockets and send data in real time to the browser to do this. But you have to write the parsing logic yourself and I do not know how you will feed the data as it arrives to the player.

As for video and audio tag: Video tag can play container files that have both audio and video. So wrap your content in a container that is compatible. If you modify your browser to write your live streaming to this video file as the live content keeps coming in and stream out that data for every byte requested by the browser this could be done. But it is definitely non trivial.