How to calculate FFT using NAudio in realtime (ASIO out)

Question

I am programming clone of guitar (violin) Hero as a final project for this school year.

The idea is to take input from my electric violin, analyse it via FFT, do some logic and drawing and output it through speakers. Perhaps some steps in parallel threads.

I already have Asio low latency input-output implemented but I am having a great problem implementing realtime FFT.

This is a code that sets up asioOut along with sampleAggregator. Sample aggregator should store samples that are added each time AudioAvailable() is called and trigger FFT calculation when the number of samples exceeds fftLength.

private static int fftLength = 8192;
private SampleAggregator sampleAggregator = new SampleAggregator(fftLength);

void asioStartPlaying(object sender, EventArgs e)
{
    sampleAggregator.PerformFFT = true;
    sampleAggregator.FftCalculated += new EventHandler<FftEventArgs>(FftCalculated);
    var asioOut = new AsioOut();
    BufferedWaveProvider wavprov = new BufferedWaveProvider(new WaveFormat(48000, 1));
    asioOut.AudioAvailable += new EventHandler<AsioAudioAvailableEventArgs> (asio_DataAvailable);
    asioOut.InitRecordAndPlayback(wavprov, 1, 25);
    asioOut.Play();
}

void asio_DataAvailable(object sender, AsioAudioAvailableEventArgs e)
{
    byte[] buf = new byte[e.SamplesPerBuffer*4];

    for (int i = 0; i < e.InputBuffers.Length; i++)
    {
        Marshal.Copy(e.InputBuffers[i], buf, 0, e.SamplesPerBuffer*4);
        Marshal.Copy(buf, 0, e.OutputBuffers[i], e.SamplesPerBuffer*4);
    }

    for (int i = 0; i < buf.Length; i=i+4)
    {
        float sample32 = BitConverter.ToSingle(buf, i);
        sampleAggregator.Add(sample32);
    }

    e.WrittenToOutputBuffers = true;
}

SampleAggregator is class taken from NAudio fft result gives intensity on all frequencies C#.

Asio outputs data in Int32LSB sample type. In buf there are values from 0 to 255.

This is function that should be called when fft is calculated (triggered from SampleAggregator class).

void FftCalculated(object sender, FftEventArgs e)
{
    for (var i = 0; i < e.Result.Length; i++)
    {
        Debug.WriteLine("FFT output.");
        Debug.WriteLine(e.Result[i].X);
        Debug.WriteLine(e.Result[i].Y);
    }
}

But the FFT always outputs NaN as a result.

I think there is a problem with the conversion to float.

Could someone point me in the right direction?

EDIT_1: I changed the loop in DataAvailable() to

for (int i = 0; i < e.SamplesPerBuffer * 4; i++)
{
    float sample32 = Convert.ToSingle(buf[i]);
    sampleAggregator.Add(sample32);
}

And FFT now outputs data. But I think they are not correct. The mistake must be in the conversion between asio samples and float values. But I am not much comfortable around byte operations.

Could e.GetAsInterleavedSamples somehow help?

Sample of raw data from FFT: X: -5,304741 Y: -0,7160959 X: 6,270798 Y: -0,4169312 X: -8,851931 Y: -0,4485725

I noticed, that first few and last few values in raw data from FFT are somehow bigger then other data. Making calculation of magnitude tricky.

Answer 1

The problem was as I thought in conversion between data about samples from Asio (4 bytes in a row in buf array) to float for fft. BitConvertor should do the trick but it somehow makes fft output NaN in my case. So I tried this conversion instead.

for (int i = 0; i < e.SamplesPerBuffer * 4; i=i+4)
{
    float sample = Convert.ToSingle(buf[i] + buf[i+1] + buf[i+2] + buf[i+3]);
    sampleAggregator.Add(sample);
}

And it works very well. Even with 192 000 sample rate.

Answer 2

I haven't used NAudio, but we have implemented pretty much similar thing with DirectSound. There's tools for this in LightningChart Ultimate SDK. AudioInput component captures waveform data from sound device, and the data is forwarded to FFT calculation (SpectrumCalculator component) and waveform monitors at same time. FFT data is then visualized as spectrograms in 2D or 3D. AudioOutput writes the data to sound device to be audible through speakers.

Overall the audio input/output, FFT calculation and visualization run with very low CPU load.

Our libraries are commercial, but I think even if you were not looking for any additional components, it might be a good idea to take a look at our audio examples, source code is visible in the demo application Visual Studio projects. You may get fresh ideas, at least :-) You can apply some methods for NAudio, I believe.

Download LightningChart demo from LightningChart web site, running it costs you nothing.

[I'm CTO of LightningChart components]