Android processing a video, YCrCb frames to video

Question

I want to process a video that I record on an Android phone by changing the color of the pixels. I know how to do the processing that I want, but I don't know the best way of putting everything together.

So far the things that I can do are:

• Get a single frame from a video as YCrCb (YUV)
• Modify the bytes in the YCrCb how I want
• Convert the YCrCb to RGB array
• Convert the RGB array to a bitmap

How can I put everything together and save it back as a video that I can playback on Android? Can i go straight from the YCrCb information? Or do I need to convert to RGB or some other format first?

I don't need to do this in real time, the aim would be to record a video, save it, then process it and save the processed video. However I do have speed in mind. Are their libraries that I can use to speed up the process? I know Android provides use of OpenGL and I'v read about ffmpeg but I'm unsure of the best path to take.

If someone could point me in the right direction I would appreciate it! =D

Thanks!! =D

Answer 1

The frames that you get from the camera are uncompressed. This is typically not the way video is stored, since uncompressed frames are too big to be read and displayed in real time by a player. Instead, you want to do encode your yuv or rgb frames, so that the frames are much smaller and the I/O load during playback is small.

The encoding format that you should use must be one that your device can decode in hardware, as decoding in CPU is also prohibitively expensive. These days, you should be good if you use H.264 for the video and AAC for the audio, and use an MP4 container. Unfortunately if you include H.264/AAC encoders (even open source ones) in your app and plan to make money from it, then you may get a visit from the MPEG LA, who may want to collect from you, since both H.264 and AAC require licensing. I believe this is avoided if you use a codec that is already licensed and installed in the device, like the one that is used by the camera app to record. I do not know if the Android platform allows you to access the camera codecs.

As far as libraries, I think libavcodec and libavformat from the ffmpeg project are the most complete. With these libs you can encode H.264/AAC/MP4 files, assuming the licensing problem doesn't affect you.