Augmented Reality: Render Video on top of a marker

Question

I'd like to render a video (can be any format as long as it works) on top of a marker, but I am not sure how to do this. I am using Artoolkit for desktop-based AR and QCAR for mobile AR. Eventually I want to get it done for both systems, therefore I am preferably looking for a video-library that I could use on both platforms.

Basically I know the steps that have to be done:

 - load video file
 - extract image according to current time
 - use image as texture 
 - apply marker's transformation matrix to textured rectangle

Which libraries could I use to achieve the first two steps?

Answer 1

Which libraries could I use to achieve the first two steps?

Take a look into the libavcodec library (you can start here), but take into account that it is quite complex.