How to read short (16bits) integers from an unsigned char (8bits) buffer

Question

As in the title I need to read short integers from a char buffer

The buffer

uint8_t *data[AV_NUM_DATA_POINTERS]

which is a field of the AVFrame frame structure, is filled by a call to the ffmpeg function

avcodec_decode_audio4(avctx,frame,got_frame_ptr,avpkt)

But, I need to read this buffer as a buffer of signed 16 bits integers because this is the sample format indicated by the codec context avctx->sample_fmt==AV_SAMPLE_FMT_S16

I tried to do this using a memcpy but I have not succeeded to get reasonable values so then I tried to use a union struct as suggested on some related questions here in StackOverflow. My code is as follows: union CharToStruct{ uint8_t myCharArray[2]; short value; } presentSound;

 audioRet=avcodec_decode_audio4(avctx,frame,got_frame_ptr,avpkt);
 if(got_frame_ptr){
     audioRet=audioRet/2;
     int b=0;
     for(int i=0;i<audioRet;i++){
         presentSound.myCharArray[0]=frame->data[0][2*i+1];
         presentSound.myCharArray[1]=frame->data[0][2*i]
         dbuf[((i-b)/2)*8+info->mLeft+b]=info->presentSound.value;//the reason of the offset by 8 here is because I will be writing the result to a multichannel device
 }

With this, the values are reasonable, but when I write this to a device using portaudio, I get just clicking noise. Am I doing the conversion in a wrong way? Can you help me maybe with some better way to do this reading?

Thank you very much for your help

Alba

Answer 1

Just think of a uint8_t array as a raw byte array. In C/C++ unsigned char (uint8_t) is as close to a "typeless" array as you can get. Any type of data can be written to any type array as raw bytes, but it is easiest to interact with an unsigned char array because each element is a value from 0x00 to 0xFF (one byte) and the user can interpret those bytes however they choose.

You may not need to do any interpretation of the data on your own, if you are simply passing the data from ffmpeg to PortAudio. PortAudio's callback (or write method if using the blocking API) requires that the user set a void pointer to the beginning of the data buffer being played. It does not matter what type that buffer is, as long as the bytes, when read in order, can be interpreted as the expected sample format. In fact you may not even need to copy the data as long as you are able to pass the buffer pointer to the callback and the buffer doesn't get deallocated before being processed by the callback. Watch out for other issues like reading a mono stream and writing a stereo stream. If you're output stream is expecting interleaved stereo audio, you'll have to write each sample to the output buffer twice (or once for each channel it is expecting).

On the other hand if you wish to manipulate the samples in the buffer, you may wish to reinterpret_cast the uint8_t* to a short*. Since the data in the buffer is already signed 16-bit samples, once you cast, each element in the array will be one sample of data. Just remember that the size of the array will only be half that of the original buffer since the elements are twice as big.

This should be entirely safe and you should not have any endianness issues moving samples between ffmpeg and PortAudio as long as you are working on a single system. If the system is big endian the samples will be big endian (high order byte in the lowest address, Motorolla), if the system is little endian (low order byte in lowest address, Intel) the samples will be little endian.

Answer 2

To me, this looks wrong:

     presentSound.myCharArray[0]=frame->data[0][2*i+1];
     presentSound.myCharArray[1]=frame->data[0][2*i]

I would expect to see:

     presentSound.myCharArray[0]=frame->data[0][2*i]
     presentSound.myCharArray[1]=frame->data[0][2*i+1];

It may be worth writing the data out to a file, and append a WAV header (take the first 40 bytes from an existing file of the right format [bits per sample, samples per second], then the number of samples in the output, and the samples after that).