Syncronization audio and video

Question

I need to display stream video using MediaElement in Windwso Phone application.

I'm getting from web-service a stream that contains frames in H264 format AND raw-AAC bytes (strange, but ffmpeg can parse with -f ac3 parameter only).

So, if try to play only one of stream (audio OR video) it plays nice. But I have issues when try it both.

For example, if I report video sample without timestamp and report audio with timestamp, my video plays 3x-5x faster then I need.

MediaStreamSample msSamp = new MediaStreamSample(
                    _videoDesc,
                    vStream,
                    0,
                    vStream.Length,
                    0,
                    _emptySampleDict);

                ReportGetSampleCompleted(msSamp);

From my web-service I getting a DTS and PTS for video and audio frames in following format:

120665029179960

but when I set it for sample, my audio stream plays too slow and with delays.

Timebase is 90khz.

So, could someone tell me how I can resolve it? Maybe I should calculate others timestamps for samples? If so, show me the way, please.

Thanks.

Answer 1

Okay, I solved it.

So, what I need to do for sync A/V:

Calculate right timestamps for each video and audio frames using framerate.

For example, for video I have 90 kHz and for audio 48 kHz and 25 frames per second - my frame increments will be:

_videoFrameTime = (int)TimeSpan.FromSeconds((double)0.9 / 25).Ticks;
_audioFrameTime = (int)TimeSpan.FromSeconds((double)0.48 / 25).Ticks;

And now we should add these values for each sample:

private void GetAudioSample()
{
... 
/* Getting sample from buffer */
MediaStreamSample msSamp = new MediaStreamSample(
                    _audioDesc,
                    audioStream,
                    0,
                    audioStream.Length,
                    _currentAudioTimeStamp,
                    _emptySampleDict);

_currentAudioTimeStamp += _audioFrameTime;

ReportGetSampleCompleted(msSamp);
}

For gettign video frame method will be the same with a _videoFrameTime incrementation instead.

Hope this will be helpfull for someone.

Roman.