Video stream capture and frame rate control

Question

I'm currently developing a video streaming software using VP8 and V4L2 but I'm struggling with the key concepts of frame rating.

I have a basic working implementation that fetches frames in a loop, encodes it and sends it over RTP (as fast as it can). However, I don't understand how to control the frame rate of the video or regulate the sampling.

Basically it could be summarized as follows :

while (true) {
    ioctl(fd, VIDIOC_DQBUF, buf); // Get the V4L buffer
    vpx_codec_encode(...);        // VP8 encode using pts and timebase
    sendto();                     // Send through RTP with the correct timestamp
}

In particular, I don't get how to properly set :

• The V4L2 capture loop (does it need a timer to fetch frames on a regular basis ?)
• The FRAME INTERVAL setting from V4L2 (is it mandatory ?)
• The libvpx timebase (should I use 1/fps ? 1001/30000 ?)
• The pts value (Does it need to be frame num * (1/fps) * 90000 ?)
• The RTP timestamp (Can I use the pts here ?)
• Any other configuration settings that could be taken into account ...

Answer 1

The V4L2 capture loop (does it need a timer to fetch frames on a regular basis ?)

either that, or find a way to block the thread until new data arrives (e.g. using select())

The FRAME INTERVAL setting from V4L2 (is it mandatory ?)

it's not mandatory. when setting the FRAME_INTERVAL, you tell the device to sample the data at a specified interval. the device might simply ignore the request (e.g. because it cannot capture at the given samplerate). also note, that the internal clock of the device might be inaccurate and/or different from other clocks on your system.

The libvpx timebase (should I use 1/fps ? 1001/30000 ?)

obviously this depends on your framerate.

the vpx-documentation is quite clear here:

Indicates the smallest interval of time, in seconds, used by the stream. For fixed frame rate material, or variable frame rate material where frames are timed at a multiple of a given clock (ex: video capture), the RECOMMENDED method is to set the timebase to the reciprocal of the frame rate (ex: 1001/30000 for 29.970 Hz NTSC). This allows the pts to correspond to the frame number, which can be handy. For re-encoding video from containers with absolute time timestamps, the RECOMMENDED method is to set the timebase to that of the parent container or multimedia framework (ex: 1/1000 for ms, as in FLV).

since the timebase is of type vpx_rational, you need to express it as the ratio between two integers. e.g. you cannot use 1/fps for NTSC's weirdo rate.

The pts value (Does it need to be frame num * (1/fps) * 90000 ?)

no not really (see above). it can be as simple as frame_num.

The RTP timestamp (Can I use the pts here ?)

yes.