Resize (downsize) YUV420sp image

Question

I am trying to resize (scale down) an image which comes in YUV420sp format. Is it possible to do such image resizing without converting it into RGB, so directly manipulating the YUV420sp pixel array? Where can I find such algorithm?

Thanks

Answer 1

YUV 4:2:0 planar looks like this:

----------------------
|     Y      | Cb|Cr |
----------------------

where:

Y = width x height pixels
Cb = Y / 4 pixels
Cr = Y / 4 pixels

Total num pixels (bytes) = width * height * 3 / 2

And the subsamling used like this:

Which means that each chroma-pixel-value is shared between 4 luma-pixels.

One approach is just to remove pixels, making sure that corresponding Y-Cb-Cr relationship are kept/recalculated.

Something close to the Nearest-neighbor interpolation but reversed.

Another approach is to first convert the 4:2:0 subsampling to 4:4:4

Here you have a 1 to 1 mapping between luma and chroma data.

This is the correct way to interpolate chroma between 4:2:0 and 4:2:2 (luma is already at correct resolution) Code in python, follow html-link for c-dito. Code is not very pythonic, just a direct translation of the c-version.

def __conv420to422(self, src, dst):
    """
    420 to 422 - vertical 1:2 interpolation filter

    Bit-exact with
    http://www.mpeg.org/MPEG/video/mssg-free-mpeg-software.html
    """
    w = self.width >> 1
    h = self.height >> 1

    for i in xrange(w):
        for j in xrange(h):
            j2 = j << 1
            jm3 = 0 if (j<3) else j-3
            jm2 = 0 if (j<2) else j-2
            jm1 = 0 if (j<1) else j-1
            jp1 = j+1 if (j<h-1) else h-1
            jp2 = j+2 if (j<h-2) else h-1
            jp3 = j+3 if (j<h-3) else h-1

            pel = (3*src[i+w*jm3]
                 -16*src[i+w*jm2]
                 +67*src[i+w*jm1]
                +227*src[i+w*j]
                 -32*src[i+w*jp1]
                  +7*src[i+w*jp2]+128)>>8

            dst[i+w*j2] = pel if pel > 0 else 0
            dst[i+w*j2] = pel if pel < 255 else 255

            pel = (3*src[i+w*jp3]
                 -16*src[i+w*jp2]
                 +67*src[i+w*jp1]
                +227*src[i+w*j]
                 -32*src[i+w*jm1]
                 +7*src[i+w*jm2]+128)>>8

            dst[i+w*(j2+1)] = pel if pel > 0 else 0
            dst[i+w*(j2+1)] = pel if pel < 255 else 255
    return dst

Run this twice to get 4:4:4. Then it's just a matter of removing rows and columns.

Or you can just quadruple the chroma-pixels to go from 4:2:0 to 4:4:4, remove rows and columns and then average 4 Cb/Cr values into 1 to get back to 4:2:0 again, it all depends on how strict you need to be :-)

Answer 2

Here is a Java function I use to scale down a YUV 420 (or NV21) by a factor of two.

The function takes the image in a byte array along with the width and height of the original image as an input and returns an image in a byte array which has width and heigh both equal to the half of the original width and height.

As a basis for my code I used this: Rotate an YUV byte array on Android

public static byte[] halveYUV420(byte[] data, int imageWidth, int imageHeight) {
    byte[] yuv = new byte[imageWidth/2 * imageHeight/2 * 3 / 2];
    // halve yuma
    int i = 0;
    for (int y = 0; y < imageHeight; y+=2) {
        for (int x = 0; x < imageWidth; x+=2) {
            yuv[i] = data[y * imageWidth + x];
            i++;
        }
    }
    // halve U and V color components
    for (int y = 0; y < imageHeight / 2; y+=2) {
        for (int x = 0; x < imageWidth; x += 4) {
            yuv[i] = data[(imageWidth * imageHeight) + (y * imageWidth) + x];
            i++;
            yuv[i] = data[(imageWidth * imageHeight) + (y * imageWidth) + (x + 1)];
            i++;
        }
    }
    return yuv;
}

Answer 3

YUV420sp has the Y in one plane and the U&V in another. If you split the U& V into separate planes, you can then perform the same scaling operation on each of the 3 planes in turn, without first having to go from 4:2:0 -> 4:4:4.

Have a look at the source code for libyuv; it just scales the planes: https://code.google.com/p/libyuv/source/browse/trunk/source/scale.cc