Extract audio frequency from instrument to find a musical note

Question

I'm trying to develop an Android app that extracts audio frequency from an instrument. I'm using Fast Fourier Transform method with Jtransforms. Here is what I have so far:

public class MainActivity extends Activity {

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);

    new readFrequencies().execute();
}

@Override
public boolean onCreateOptionsMenu(Menu menu) {
    // Inflate the menu; this adds items to the action bar if it is present.
    getMenuInflater().inflate(R.menu.main, menu);
    return true;
}

private class readFrequencies extends AsyncTask<Void,Integer,Integer> {

        @Override
        protected Integer doInBackground(Void... arg0) {
            AudioRecord recorder = null;
            int bufferSize = 0;
            boolean recording = true;

            int rate = 8000;
            short audioFormat = AudioFormat.ENCODING_PCM_16BIT;
            short channelConfig = AudioFormat.CHANNEL_IN_MONO;

            try {
                bufferSize = AudioRecord.getMinBufferSize(rate,channelConfig, audioFormat);

                recorder = new AudioRecord(AudioSource.DEFAULT, rate, 
                    channelConfig, audioFormat, bufferSize);

                if (recorder.getState() == AudioRecord.STATE_INITIALIZED) {
                    /*
                     *  Android 4.1.2
                     * 
                    int recorder_id = recorder.getAudioSessionId();
                    if (NoiseSuppressor.isAvailable()) NoiseSuppressor.create(recorder_id);
                    */
                }
                else {
                    Toast.makeText(getApplicationContext(), 
                            "Error en la inicialización", Toast.LENGTH_SHORT).show();
                }
            } catch (Exception e) {}

            short[] audioData = new short[bufferSize];

            if (recorder != null) {
                while (recording) {
                    if (recorder.getRecordingState() == AudioRecord.RECORDSTATE_STOPPED) {
                        recorder.startRecording();
                    }
                    else {
                        int numshorts = recorder.read(audioData,0,audioData.length);
                        if ((numshorts != AudioRecord.ERROR_INVALID_OPERATION) && 
                            (numshorts != AudioRecord.ERROR_BAD_VALUE)) {

                            //  Hann
                            double[] preRealData = new double[bufferSize];
                            double PI = 3.14159265359;
                            for (int i = 0; i < bufferSize; i++) {
                                double multiplier = 0.5 * (1 - Math.cos(2*PI*i/(bufferSize-1)));
                                preRealData[i] = multiplier * audioData[i];
                            }

                            DoubleFFT_1D fft = new DoubleFFT_1D(bufferSize);
                            double[] realData = new double[bufferSize * 2];

                            for (int i=0;i<bufferSize;i++) {
                                realData[2*i] = preRealData[i];
                                realData[2*i+1] = 0;    
                            }
                            fft.complexForward(realData);

                            double magnitude[] = new double[bufferSize / 2];

                            for (int i = 0; i < magnitude.length; i++) {
                                double R = realData[2 * i];
                                double I = realData[2 * i + 1];

                                magnitude[i] = Math.sqrt(I*I + R*R);
                            }

                            int maxIndex = 0;
                            double max = magnitude[0];
                            for(int i = 1; i < magnitude.length; i++) {
                                if (magnitude[i] > max) {
                                    max = magnitude[i];
                                    maxIndex = i;
                                }
                            }

                            int frequency = rate * maxIndex / bufferSize;
                            publishProgress(frequency);
                        }
                        else {
                            if (numshorts == AudioRecord.ERROR_BAD_VALUE) {
                                Toast.makeText(getApplicationContext(), 
                                        "ERROR_BAD_VALUE", Toast.LENGTH_SHORT).show();
                            }
                            else {
                                Toast.makeText(getApplicationContext(), 
                                        "ERROR_INVALID_OPERATION", Toast.LENGTH_SHORT).show();
                            }

                            return -1;
                        }
                    }
                }

                if (recorder.getState() == AudioRecord.RECORDSTATE_RECORDING) 
                    recorder.stop(); //stop the recorder before ending the thread
                recorder.release();
                recorder=null;
            }
            return 0;
        }

        protected void onProgressUpdate(Integer... f) {
            TextView texto = (TextView) findViewById(R.id.texto);
            texto.setText(String.valueOf(f[0]));
        }

        protected void onPostExecute(Integer f) {
            TextView texto = (TextView) findViewById(R.id.texto);
            int frecuencias = f.intValue();
            texto.setText(String.valueOf(frecuencias));
        }
}

}

With this code I am able to obtain the exact frequency from a frequency generator which produces pure signals. However, when I try the same with an instrument, I obtain random frequencies. I know that, when it comes to real instruments, the signals produced contain harmonics which may influence on the final result, but I don't know to obtain the real frequency in this case. Could anybody help me?

I've used TarsosDSP and I also have tried the Autocorrelation method, but I haven't been able to get what I wanted.

Thanks in advance.

Answer 1

There is a huge difference between the most powerful frequency component in a spectrum and the perceived pitch by a human listener.

This academic paper is probably the definitive review of approaches to solving some of the problems of pitch detection, but does not address the perceptual issues you will need to deal with using real signals. At the very least you'll need to account for string and wind-column instruments that jump octave at the beginning and end of notes, and missing fundamentals. This will be a particular lottery on Android where microphones on devices are not in any way consistent.

Searching for the peak frequency in the spectrum is no better than using autocorrelation and frequently wrong with real signals.

The FFT approach (more properly, using STFT and frequency estimation from the phase differences) can be made to work, but only with a lot of post-processing of the spectrum. Several commercial chromatic tuner apps use an FFT-based approach (I can attest to this having written one).

You might like to have a look how some of the feature extraction plugins for Sonic Visualiser go about solving this problem.

I also recommend reviewing the vast number of previous questions on here asking pretty much the same thing - usually questioners wanting to build Guitar tuners.

Answer 2

For what it's worth - I believe electronic tuners for musical instruments (e.g. guitar tuners, etc.) don't do it this way. Instead of doing a FFT like you're doing, they are simply measuring the period of the waves - (i.e. the time between the zero crossings), then computing the frequency from the period.