FFmpeg library: modified muxing sample for FLTP input and FLTP audio, loses audio

Question

Based on muxing sample that comes with FFmpeg docs, I have modified it, from input format as S16 to FLTP (planar stereo), and outputting to webm format (stereo).

Since input is now FLTP, I am filling two arrays, then encoding again to FLTP. There are no obvious errors given on screen, but the resulting webm video does not play any audio (just the video content). This is just proof of concept in understanding things; here is an added (crude) function to fill up input FLTP stereo buffer:

static void get_audio_frame_for_planar_stereo(int16_t **samples, int frame_size, int nb_channels)
{
    int j, i, v[2];
    int16_t *q1 = (int16_t *) samples[0];
    int16_t *q2 = (int16_t *) samples[1];

    for (j = 0; j < frame_size; j++)
    {
        v[0] = (int)(sin(t) * 10000);
        v[1] = (int)(tan(t) * 10000);
        *q1++ = v[0];
        *q2++ = v[1];
        t     += tincr;
        tincr += tincr2;
    }
}

Which I am calling from inside write_audio_frame() function.

Note also, wherever code reffered AV_SAMPLE_FMT_S16 as input, I have changed to AV_SAMPLE_FMT_FLTP.

Whole workable source is here:

https://gist.github.com/anonymous/05d1d7662e9feafc45a6

When run with ffprobe.exe, with these instructions:

ffprobe -show_packets output.webm >output.txt

I see nothing out of ordinary, all pts/dts values appear to be in place:

https://gist.github.com/anonymous/3ed0d6308700ab991704

Could someone highlight cause of this mis-interpretation?

Thanks for your time...

p.s. I am using Zeranoe FFmpeg Windows builds (32 bit), built on Jan 9 2014 22:04:35 with gcc 4.8.2.(GCC)

Edit: Based on your guidance elsewhere, I tried the following:

    /* set options */
    //av_opt_set_int       (swr_ctx, "in_channel_count",   c->channels,       0);
    //av_opt_set_int       (swr_ctx, "in_sample_rate",     c->sample_rate,    0);
    //av_opt_set_sample_fmt(swr_ctx, "in_sample_fmt",      AV_SAMPLE_FMT_FLTP, 0);
    //av_opt_set_int       (swr_ctx, "out_channel_count",  c->channels,       0);
    //av_opt_set_int       (swr_ctx, "out_sample_rate",    c->sample_rate,    0);
    //av_opt_set_sample_fmt(swr_ctx, "out_sample_fmt",     c->sample_fmt,     0);

    av_opt_set_int(swr_ctx, "in_channel_layout",    AV_CH_LAYOUT_STEREO, 0);
    av_opt_set_int(swr_ctx, "in_sample_rate",       c->sample_rate, 0);
    av_opt_set_sample_fmt(swr_ctx, "in_sample_fmt", AV_SAMPLE_FMT_FLTP, 0);

    av_opt_set_int(swr_ctx, "out_channel_layout",    AV_CH_LAYOUT_STEREO, 0);
    av_opt_set_int(swr_ctx, "out_sample_rate",       c->sample_rate, 0);
    av_opt_set_sample_fmt(swr_ctx, "out_sample_fmt", AV_SAMPLE_FMT_FLTP, 0);

And the revised function:

static void get_audio_frame_for_planar_stereo(uint8_t **samples, int frame_size, int nb_channels)
{
    int j, i;
    float v[2];
    float *q1 = (float *) samples[0];
    float *q2 = (float *) samples[1];

    for (j = 0; j < frame_size; j++) 
    {
        v[0] = (tan(t) * 1);
        v[1] = (sin(t) * 1);
        *q1++ = v[0];
        *q2++ = v[1];
        t     += tincr;
        tincr += tincr2;
    }
}

Now it appears to be working properly. I tried changing function parameters from uint8_t** to float**, as well as src_samples_data from uint8_t** to float**, but did not make any difference, in a view.

Updated code: https://gist.github.com/anonymous/35371b2c106961029c3d

Thanks for highlighting the place(s) that result in this behavior!

Answer 1

With AV_SAMPLE_FMT_FLTP each sample must be a 32-bit float value (from -1.0 to 1.0). You also initialize the resampler to accept the floats:

av_opt_set_sample_fmt(swr_ctx, "in_sample_fmt", AV_SAMPLE_FMT_FLTP, 0);

but feeding it up with the array of ints:

get_audio_frame_for_planar_stereo( (int16_t **)src_samples_data, src_nb_samples, c->channels );