Error with java video decoding by Xuggler

Question

I am learning Xuggler(a library supports video streaming for Java) by following the code in a tutorial teaching how to decode and play video.

I supposed this snippet of code is reliable, but when I want to play the video read on my window, I got the error telling me

Exception in thread "main" java.lang.RuntimeException: got error decoding video in: C:/Users/swnmlab/1.mp4

This error happens when this line got executed

int bytesDecoded = videoCoder.decodeVideo(picture, packet,offset);

I used debugger to step into and find that xuggle-xuggler.jar has no source attachment, does anyone has encountered this problem before?

import java.awt.image.BufferedImage;

import com.xuggle.xuggler.ICodec.Type;
import com.xuggle.xuggler.IContainer;
import com.xuggle.xuggler.IPacket;
import com.xuggle.xuggler.IPixelFormat;
import com.xuggle.xuggler.IStream;
import com.xuggle.xuggler.IStreamCoder;
import com.xuggle.xuggler.IVideoPicture;
import com.xuggle.xuggler.IVideoResampler;
import com.xuggle.xuggler.Utils;
import com.xuggle.xuggler.demos.VideoImage;

public class DecodeAndPlayVideo {
    public static void main(String[] args) {
        String filename = "C:/Users/swnmlab/1.mp4";

        // Create a Xuggler container object
        IContainer container = IContainer.make();

        // Open up the container
        if (container.open(filename, IContainer.Type.READ, null) < 0)
            throw new IllegalArgumentException("could not open file: "
                    + filename);

        // query how many streams the call to open found
        int numStreams = container.getNumStreams();

        // and iterate through the streams to find the first video stream
        int videoStreamId = -1;
        IStreamCoder videoCoder = null;
        for (int i = 0; i < numStreams; i++) {
            // Find the stream object
            IStream stream = container.getStream(i);
            // Get the pre-configured decoder that can decode this stream;
            IStreamCoder coder = stream.getStreamCoder();

            if (coder.getCodecType() == Type.CODEC_TYPE_VIDEO) {
                videoStreamId = i;
                videoCoder = coder;
                break;
            }
        }
        if (videoStreamId == -1)
            throw new RuntimeException(
                    "could not find video stream in container: " + filename);

        if (videoCoder.acquire() < 0)
            throw new RuntimeException(
                    "could not open video decoder for container: " + filename);

        IVideoResampler resampler = null;
        if (videoCoder.getPixelType() != IPixelFormat.Type.BGR24) {
            // if this stream is not in BGR24, we're going to need to
            // convert it. The VideoResampler does that for us.
            resampler = IVideoResampler.make(videoCoder.getWidth(),
                    videoCoder.getHeight(), IPixelFormat.Type.BGR24,
                    videoCoder.getWidth(), videoCoder.getHeight(),
                    videoCoder.getPixelType());
            if (resampler == null)
                throw new RuntimeException("could not create color space "
                        + "resampler for: " + filename);
        }

        /*
         * And once we have that, we draw a window on screen
         */
        openJavaWindow();

        IPacket packet = IPacket.make();
        while (container.readNextPacket(packet) >= 0) {
            /*
             * Now we have a packet, let's see if it belongs to our video stream
             */
            if (packet.getStreamIndex() == videoStreamId) {
                IVideoPicture picture = IVideoPicture.make(
                        videoCoder.getPixelType(), videoCoder.getWidth(),
                        videoCoder.getHeight());

                int offset = 0;
                while (offset < packet.getSize()) {
                    /*
                     * Now, we decode the video, checking for any errors.
                     */
                    int bytesDecoded = videoCoder.decodeVideo(picture, packet,
                            offset);
                    if (bytesDecoded < 0)
                        throw new RuntimeException(
                                "got error decoding video in: " + filename);
                    offset += bytesDecoded;

                    /*
                     * Some decoders will consume data in a packet, but will not
                     * be able to construct a full video picture yet. Therefore
                     * you should always check if you got a complete picture
                     * from the decoder
                     */
                    if (picture.isComplete()) {
                        IVideoPicture newPic = picture;
                        /*
                         * If the resampler is not null, that means we didn't
                         * get the video in BGR24 format and need to convert it
                         * into BGR24 format.
                         */
                        if (resampler != null) {
                            // we must resample
                            newPic = IVideoPicture.make(
                                    resampler.getOutputPixelFormat(),
                                    picture.getWidth(), picture.getHeight());
                            if (resampler.resample(newPic, picture) < 0)
                                throw new RuntimeException(
                                        "could not resample video from: "
                                                + filename);
                        }
                        if (newPic.getPixelType() != IPixelFormat.Type.BGR24)
                            throw new RuntimeException("could not decode video"
                                    + " as BGR 24 bit data in: " + filename);


                        @SuppressWarnings("deprecation")
                        BufferedImage javaImage = Utils.videoPictureToImage(newPic);

                        // and display it on the Java Swing window
                        updateJavaWindow(javaImage);    
                    }

                }
            } else {
                /*
                 * This packet isn't part of our video stream, so we just silently drop it.
                 */
                do {
                } while (false);
            }
        }

        closeJavaWindow();

    }

    private static VideoImage mScreen = null;
    private static void updateJavaWindow(BufferedImage javaImage) {
        mScreen.setImage(javaImage);
    }

    private static void openJavaWindow() {
        mScreen = new VideoImage();
    }

    private static void closeJavaWindow() {
        System.exit(0);
    }
}

P.S. If you want to try this library, you can find the installing file here, and then follow the steps on this page finish installing this library on Windows.

---

I found the error happened because I cahnged the original code

        if (videoCoder.open() < 0)
            throw new RuntimeException(
                    "could not open video decoder for container: " + filename);

to

        if (videoCoder.acquire() < 0)
            throw new RuntimeException(
                    "could not open video decoder for container: " + filename);

since open() method causes deprecation warning, so I used auto complete to find a method that looks like open(), then changed to acquire(). I thought that was OK since no "could not open video decoder for container: " exception thrown out.So just follow the sample code.

Answer 1

I went through your code and found that you obtained a videoCoder, but you didn't open it before playing. Maybe that is why you couldn't decode it. So could you please have a try?

if (videoCoder.open() < 0)
    throw new RuntimeException(
            "could not open video decoder for container: "
                    + filename);
IVideoResampler resampler = null;

Answer 2

open() method is deprecated , you should use open(null,null) instead

if (videoCoder.open(null,null) < 0)
throw new RuntimeException(
        "could not open video decoder for container: "
                + filename);

Answer 3

I executed the same code with following code changes. These changes were required as below APIs are deprecated.

IMetaData params = IMetaData.make();
IContainerParameters params = IContainerParameters.make();

As shown, I used videoCoder to set the timeBase, Height, and Width.

  if (coder.getCodecType() == ICodec.Type.CODEC_TYPE_VIDEO)
  {
    videoStreamId = i;
    videoCoder = coder;

    // The timebase here is used as the camera frame rate    
    videoCoder.setTimeBase(IRational.make(30,1));

    // we need to tell the driver what video with and height to use
    videoCoder.setWidth(320);
    videoCoder.setHeight(240);

    break;
  }

However, I am facing a different problem that the webcam display occupying entire screen, rather than the specified Width and Height.

Is the code changes to set the height and width are incorrect? How should we control the size?