Playing back audio with a delay

Question

currently i'm working on a project regarding "delayed auditory feedback" (DAF). Basically i want to record sounds from a microphone, delay it by a specific amount of time and then play it back. Using a delay around 200ms and a person with a headset, this feedback shuts down the persons ability to speak fluently. (Pretty much fun: DAF on youtube)

Right now i am trying to make this loop with SourceDataLine and TargetDataLine using a byte[]-buffer with 256 bytes. If the buffer gets bigger, so does the delay. My problem is now: I can't tell what the delay in milliseconds is.

Is there any way to calculate the real delay in ms from the buffer size? Or is there maybe another approach to get this result?

This is what my loop looks like at the moment:

private int mBufferSize; // 256
private TargetDataLine mLineOutput;
private SourceDataLine mLineInput;
public void run() {

    ... creating the DataLines and getting the lines from AudioSystem ...

    // byte buffer for audio
    byte[] data = new byte[mBufferSize];

    // start the data lines
    mLineOutput.start();
    mLineInput.start();

    // start recording and playing back
    while (running) {
        mLineOutput.read(data, 0, mBufferSize);
        mLineInput.write(data, 0, mBufferSize);
    }

    ... closing the lines and exiting ...

}

Answer 1

You can calculate the delay easily, as it's dependent on the sample rate of the audio. Assuming this is CD-quality (mono) audio, the sample rate is 44,100 samples per second. 200 milliseconds is 0.2 seconds, so 44,100 X 0.2 = 8820.

So your audio playback needs to be delayed by 8820 samples (or 17640 bytes). If you make your recording and playback buffers exactly this size (17640 bytes) it will make your code pretty simple. As each recording buffer is filled you pass it to playback; this will achieve a playback lag of exactly one buffer's duration.

Answer 2

There is some delay inherent in Android that you should account for, but aside from that...

Create a circular buffer. Doesn't matter how big, as long as it is more than big enough for N 0 samples. Now write it with N '0' samples.

N in this case is (delay in seconds) * (sample rate in hertz).

Example: 200ms with 16kHz stereo:

0.2s*16000Hz*(2 channels)=3200*2 samples = 6400 samples

You will probably be working with pcm data too, which is 16-bit, so use short instead of byte.

After filling the buffer with the right amount of zeroes, start reading data for the speaker while filling with data from the microphone.

PCM Fifo:

public class PcmQueue
{
    private short                mBuf[] = null;
    private int                  mWrIdx = 0;
    private int                  mRdIdx = 0;
    private int                  mCount = 0;
    private int                  mBufSz = 0;
    private Object               mSync  = new Object();

    private PcmQueue(){}

    public PcmQueue( int nBufSz )
    {
        try {
            mBuf = new short[nBufSz];
        } catch (Exception e) {
            Log.e(this.getClass().getName(), "AudioQueue allocation failed.", e);
            mBuf = null;
            mBufSz = 0;
        }
    }

    public int doWrite( final short pWrBuf[], final int nWrBufIdx, final int nLen )
    {
        int sampsWritten   = 0;

        if ( nLen > 0 ) {

            int toWrite;
            synchronized(mSync) {
                // Write nothing if there isn't room in the buffer.
                toWrite = (nLen <= (mBufSz - mCount)) ? nLen : 0;
            }

            // We can definitely read toWrite shorts.
            while (toWrite > 0)
            {
                // Calculate how many contiguous shorts to the end of the buffer
                final int sampsToCopy = Math.min( toWrite, (mBufSz - mWrIdx) );

                // Copy that many shorts.
                System.arraycopy(pWrBuf, sampsWritten + nWrBufIdx, mBuf, mWrIdx, sampsToCopy);

                // Circular buffering.
                mWrIdx += sampsToCopy;
                if (mWrIdx >= mBufSz) {
                    mWrIdx -= mBufSz;
                }

                // Increment the number of shorts sampsWritten.
                sampsWritten += sampsToCopy;
                toWrite -= sampsToCopy;
            }

            synchronized(mSync) {
                // Increment the count.
                mCount = mCount + sampsWritten;
            }
        }
        return sampsWritten;
    }

    public int doRead( short pcmBuffer[], final int nRdBufIdx, final int nRdBufLen )
    {
        int sampsRead   = 0;
        final int nSampsToRead = Math.min( nRdBufLen, pcmBuffer.length - nRdBufIdx );

        if ( nSampsToRead > 0 ) {
            int sampsToRead;
            synchronized(mSync) {
                // Calculate how many shorts can be read from the RdBuffer.
                sampsToRead = Math.min(mCount, nSampsToRead);
            }

            // We can definitely read sampsToRead shorts.
            while (sampsToRead > 0) 
            {
                // Calculate how many contiguous shorts to the end of the buffer
                final int sampsToCopy = Math.min( sampsToRead, (mBufSz - mRdIdx) );

                // Copy that many shorts.
                System.arraycopy( mBuf, mRdIdx, pcmBuffer, sampsRead + nRdBufIdx, sampsToCopy);

                // Circular buffering.
                mRdIdx += sampsToCopy;
                if (mRdIdx >= mBufSz)  {
                    mRdIdx -= mBufSz;
                }

                // Increment the number of shorts read.
                sampsRead += sampsToCopy;
                sampsToRead -= sampsToCopy;
            }

            // Decrement the count.
            synchronized(mSync) {
                mCount = mCount - sampsRead;
            }
        }
        return sampsRead;
    }
}

And your code, modified for the FIFO... I have no experience with TargetDataLine/SourceDataLine so if they only handle byte arrays, modify the FIFO for byte instead of short.

private int mBufferSize; // 256
private TargetDataLine mLineOutput;
private SourceDataLine mLineInput;
public void run() {

    ... creating the DataLines and getting the lines from AudioSystem ...


    // short buffer for audio
    short[] data = new short[256];
    final int emptySamples = (int)(44100.0 * 0.2); 
    final int bufferSize = emptySamples*2; 
    PcmQueue pcmQueue = new PcmQueue( bufferSize );

    // Create a temporary empty buffer to write to the PCM queue
    {
        short[] emptyBuf = new short[emptySamples];
        Arrays.fill(emptyBuf, (short)emptySamples );
        pcmQueue.doWrite(emptyBuf, 0, emptySamples);
    }

    // start recording and playing back
    while (running) {
        mLineOutput.read(data, 0, mBufferSize);
        pcmQueue.doWrite(data, 0, mBufferSize);        
        pcmQueue.doRead(data, 0, mBufferSize);        
        mLineInput.write(data, 0, mBufferSize);
    }

    ... closing the lines and exiting ...

}