manipulate/process audio signal in android?

Question

I'm new to android and new to anything related to audio processing.

So I will need step by step guidance from where to start.

I have already used android AudioRecord to input sound from the microphone and the AudioTrack class to output it through the speakers in realtime, which is working fine.

What I'm trying to do is to change the amplitude/frequency of the input signal and add distortion effect before outputting it.

This is what I've done so far:

    // Calculate minimum buffer size
    int bufferSize = AudioRecord.getMinBufferSize(44100,
        AudioFormat.CHANNEL_IN_STEREO,AudioFormat.ENCODING_PCM_16BIT);
    // AudioRecord Lib
    final AudioRecord record = new AudioRecord(MediaRecorder.AudioSource.MIC, 44100,
            AudioFormat.CHANNEL_IN_STEREO, AudioFormat.ENCODING_PCM_16BIT,
            bufferSize);
    // AudioTrack Lib
    final AudioTrack audioPlayer = new AudioTrack(
            AudioManager.STREAM_MUSIC, 44100,
            AudioFormat.CHANNEL_IN_STEREO, AudioFormat.ENCODING_PCM_16BIT,
            bufferSize, AudioTrack.MODE_STREAM);
    record_button = (Button) findViewById(R.id.start_record);
    stop_recording = (Button) findViewById(R.id.stop_record);
    stop_recording.setOnClickListener(new OnClickListener() {
        @Override
        public void onClick(View v) {
            System.out.println("Stopped record");
            stopped = true;
            // TODO Auto-generated method stub
            // recorder.stop();
            // recorder.release();
            record.stop();
            record.release();
            audioPlayer.stop();
            audioPlayer.release();

        }
    });

    record_button.setOnClickListener(new OnClickListener() {

        @Override
        public void onClick(View v) {
            // TODO Auto-generated method stub
            System.out.println("Recording");
            record.startRecording();
            audioPlayer.play();
            new AsyncBuffer(record,audioPlayer).execute();
            }
    });

}
class asyncBuffer extends AsyncTask<Void, Void, Void>{
    AudioRecord record;
    AudioTrack audioPlayer;

    public AsyncBuffer(AudioRecord record, AudioTrack audioPlayer) {
        // TODO Auto-generated constructor stub
        this.record = record;
        this.audioPlayer = audioPlayer;
    }

    @Override
    protected Void doInBackground(Void... params) {
        System.out.println("asynch task");

        short[] buffer = new short[160];
        while (!stopped) {
            Log.i("Map", "Writing new data to buffer");
            int n = record.read(buffer, 0, buffer.length);
            audioPlayer.write(buffer, 0, n);

        }
        return null;
    }   
}

I'm already reading the input signal, I just need to know how to process it in the AsyncBuffer class.

Is it possible to add distortion in the above mentioned code by manipulating the signal in the java code itself? If so, how?

If not, are there any java libraries that you can recommend?

Also, can this be achieved with plain Android SDK using the included classes, or I will have to go deeper and use NDK (Kissfft, etc.)?

Answer 1

There is two general approaches to audio programming in Android. You have found the first one, which is to stay in the SDK in java, with AudioTrack.

The downside of this approach is that your audio processing also remains in java code, which could potentially be slower than compiled C code. The audio latency, without processing, is practically the same though.

If you stick with Java, you will probably need a solid FFT library with support for Java (through a wrapper), such as this.

If you choose the NDK approach (C with OpenSL) over SDK (Java with AudioTrack), the setup will be more complex than what you have right now. There is a very good example, with step by step instructions here. That should give you the basic setup for playback of recorded audio and a starting point for adding your processing. When implementing the OpenSL approach, you will still benefit from FFT libraries such as the one linked above, as it can be hard to write and debug audio processing code yourself.

Answer 2

If you need to do real time audio analysis on Android, the NDK is the way to go.

You can read a little bit more here: https://stackoverflow.com/a/22156746/1367571