Decode video in Raspberry Pi without using OpenMAX?

Question

I am looking for an example of decoding video on Raspberry Pi directly, without using OpenMAX.

This explains the different layers of multimedia software:

There is an additional layer which is not shown in here, the "MMAL" layer which is (I believe) a Broadcom wrapper around OpenMAX. (If not, it would be an OpenMAX alternative, sitting on top of the kernel driver) raspivid and raspistill for example are written using MMAL.

I want an example of video decode where the input is raw H.264, and the output is either video in memory or video on screen. I want to do this using VCHIQ directly, not using OpenMAX. (Mainly for performance and flexibility reasons)

This github repository: https://github.com/raspberrypi/userland/ contains the source for everything shown above (the orange and green boxes; source for VCHIQ itself, OpenMAX IL implementation on top of VCHIQ, also OpenGL and EGL implementations, ...). So in theory it should be enough to get started. The problem is that it is highly non-obvious how to use it, even if one is very familiar with OpenMAX and with multimedia frameworks in general.

For example: vchiq_bulk_transmit() seems to be the function that one would use to send video to the decoder. But how to initialize the first argument of type VCHIQ_SERVICE_HANDLE_T? Where do the results go, in the framebuffer, or in a result handle, or... ?

EDIT The bounty can be collected by either providing a working example of video decode using vchiq, an API walkthrough that shows the calling sequence (even though not a working example) or a pointer to sufficient documentation to write this. A working example will get a hefty extra bounty :)

Answer 1

I don't have a working example, but I have an API walkthrough. Sort of..

Link to the full source code

I found the following function that demonstrate how you can call vchiq_bulk_transmit

int32_t vchi_bulk_queue_transmit(VCHI_SERVICE_HANDLE_T handle,
    void *data_src,
    uint32_t data_size,
    VCHI_FLAGS_T flags,
    void *bulk_handle)
{
    SHIM_SERVICE_T *service = (SHIM_SERVICE_T *)handle;
    ..
    status = vchiq_bulk_transmit(service->handle, data_src,
        data_size, bulk_handle, mode);
    ..
    return vchiq_status_to_vchi(status);
}
EXPORT_SYMBOL(vchi_bulk_queue_transmit);

There is a function to create VCHI_SERVICE_HANDLE_T

int32_t vchi_service_create(VCHI_INSTANCE_T instance_handle,
    SERVICE_CREATION_T *setup,
    VCHI_SERVICE_HANDLE_T *handle)
{
    VCHIQ_INSTANCE_T instance = (VCHIQ_INSTANCE_T)instance_handle;
    SHIM_SERVICE_T *service = service_alloc(instance, setup);

    *handle = (VCHI_SERVICE_HANDLE_T)service;
    ..
    return (service != NULL) ? 0 : -1;
}
EXPORT_SYMBOL(vchi_service_create);

But you need a VCHI_INSTANCE_T which can be initialized here

int32_t vchi_initialise(VCHI_INSTANCE_T *instance_handle)
{
    VCHIQ_INSTANCE_T instance;
    VCHIQ_STATUS_T status;

    status = vchiq_initialise(&instance);

    *instance_handle = (VCHI_INSTANCE_T)instance;

    return vchiq_status_to_vchi(status);
}
EXPORT_SYMBOL(vchi_initialise);

Answer 2

I think openmax gives more performance in multimedia processing. You can compare performance for this two alternatives simple running respective pipelines for gstreamer. For this actions not needed programming and you can use gst-launch fot this purposes. Openmax plugins for gstreamer start with 'omx' prefix. Coding and decoding operations perfectly executing with omx while main CPU get no load. Proprietary implementation for h264 coding or decoding is very difficult problem and without using libraries you can spend for this many years.