Capturing jpegs from an h264 stream with gstreamer on a Raspberry Pi

Question

I have one of the new camera add-ons for a Raspberry Pi. It doesn't yet have video4linux support but comes with a small program that spits out a 1080p h264 stream. I have verified this works and got it pushing the video to stdout with:

raspivid -n -t 1000000 -vf -b 2000000 -fps 25 -o -

I would like to process this stream such that I end up with a snapshot of the video taken once a second.

Since it's 1080p I will need to use the rpi's hardware support for H264 encoding. I believe gstreamer is the only app to support this so solutions using ffmpeg or avconv won't work. I've used the build script at http://www.trans-omni.co.uk/pi/GStreamer-1.0/build_gstreamer to make gstreamer and the plugin for hardware H264 encoding and it appears to work:

root@raspberrypi:~/streamtest# GST_OMX_CONFIG_DIR=/etc/gst gst-inspect-1.0 | grep 264
...
omx:  omxh264enc: OpenMAX H.264 Video Encoder
omx:  omxh264dec: OpenMAX H.264 Video Decoder

So I need to construct a gst-launch pipeline that takes video on stdin and spits out a fresh jpeg once a second. I know I can use gstreamer's 'multifilesink' sink to do this so have come up with the following short script to launch it:

root@raspberrypi:~/streamtest# cat test.sh
#!/bin/bash

export GST_OMX_CONFIG_DIR=/etc/gst

raspivid -n -t 1000000 -vf -b 2000000 -fps 25 -o - |  \
gst-launch-1.0 fdsrc fd=0 ! decodebin ! videorate ! video/x-raw,framerate=1/1 ! jpegenc ! multifilesink location=img_%03d.jpeg

Trouble is it doesn't work: gstreamer just sits forever in the prerolling state and never spits out my precious jpegs.

root@raspberrypi:~/streamtest# ./test.sh
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
[waits forever]

In case it's helpful output with gstreamer's -v flag set is at http://pastebin.com/q4WySu4L

Can anyone explain what I'm doing wrong?

Answer 1

We finally found a solution to this. My gstreamer pipeline was mostly right but two problems combined to stop it working:

• raspivid doesn't add timestamps to the h264 frames it produces
• recent versions of gstreamer have a bug which stop it handling untimestamped frames

Run a 1.0 build of gstreamer (be sure to build from scratch & remove all traces of previous attempts) and the problem goes away.

See http://gstreamer-devel.966125.n4.nabble.com/Capturing-jpegs-from-an-h264-stream-tt4660254.html for the mailing list thread.