How to record perfect loops in iOS and Xcode

Question

I've been struggling with this for about a year now trying to pin down my problem and represent it for others to see.

I've been writing an app that depends on 'GarageBand' like recording. That is, I want to record the user for exactly 8 beats, and then I want them to be able to loop this. I am playing a metronome for the user at the same time (User would be wearing head phones hearing the metronome, recording into the mic on their device)

I can manage to turn on recording for about 4.8 seconds (.6*8 beats), and the timer says it ran for 4.8 seconds, however my audio recording is always a bit shorter than 4.8. It's like 4.78, or 4.71 which causes a the loop to go out of whack.

I've experimented with AVAudioRecorder,AudioQueue, and AudioUnits thinking one the latter methods might result in solving my problem.

I am using NSTimer to fire off every .6 seconds playing a short blip for the metronome. After 4 beats, the metronome timer's function, turns on the recorder metronnome which waits for 4.6 seconds the stops recording.

I'm using time intervals to time how long the metro runs (looks pretty tight at 4.800xxx) and comparing this to the duration of the audio file which is always different.

I wish I could attach my project, but I guess I'll just have to settle with attaching my header and implementation. To test you'll have to make a project with the following IB characteristics:

Record, Play, Stop buttons Song/Track duration label Timer duration label Debug label

If you launch the app, then hit record, you are 'counted in' with 4 beats, then the recorder starts. Tap your finger on the desk until the recorder stops. After 8 more beats (12 in total) the recorder stops.

You can see in the displays that the recorded track is a little shorter than 4.8 seconds, and in some cases, a lot shorter, causing the audio to not loop properly.

Does anyone know what I can do to tighten this up? Thanks for reading.

Here's my code:

//
//  ViewController.h
//  speakagain
//
//  Created by NOTHING on 2014-03-18.
//

#import <UIKit/UIKit.h>
#import <Foundation/Foundation.h>
#import "CoreAudio/CoreAudioTypes.h"
#import <AudioToolbox/AudioQueue.h>
#import <AudioToolbox/AudioFile.h>
#import <AVFoundation/AVFoundation.h>

@interface ViewController : UIViewController
{
    IBOutlet UIButton *btnRecord;
    IBOutlet UIButton *btnPlay;
    IBOutlet UIButton *btnStop;
    IBOutlet UILabel *debugLabel;
    IBOutlet UILabel *timerDuration;
    IBOutlet UILabel *songDuration;

    //UILabel *labelDebug;

    struct AQRecorderState {
        AudioStreamBasicDescription  mDataFormat;
        AudioQueueRef                mQueue;
        AudioQueueBufferRef          mBuffers[kNumberBuffers];
        AudioFileID                  mAudioFile;
        UInt32                       bufferByteSize;
        SInt64                       mCurrentPacket;
        bool                         mIsRunning;                    // 8

    };
    struct AQRecorderState aqData;
    AVAudioPlayer *audioPlayer;

    NSString *songName;
    NSTimer *recordTimer;
    NSTimer *metroTimer;
    NSTimeInterval startTime, endTime, elapsedTime;

    int inputBuffer;
    int beatNumber;

}
@property (nonatomic, retain)   IBOutlet UIButton *btnRecord;
@property (nonatomic, retain)   IBOutlet UIButton *btnPlay;
@property (nonatomic, retain)   IBOutlet UIButton *btnStop;
@property (nonatomic, retain)   IBOutlet UILabel *debugLabel;
@property (nonatomic, retain) IBOutlet UILabel *timerDuration;
@property (nonatomic, retain) IBOutlet UILabel *songDuration;


- (IBAction) record;
- (IBAction) stop;
- (IBAction) play;

static void HandleInputBuffer (void *aqData,AudioQueueRef inAQ,AudioQueueBufferRef inBuffer,const AudioTimeStamp *inStartTime, UInt32 inNumPackets,const AudioStreamPacketDescription  *inPacketDesc);

@end

Implementation:

//
    //  ViewController.m
    //  speakagain
    //
    //  Created by NOTHING on 2014-03-18.
    //

    #import "ViewController.h"


    @interface ViewController ()

    @end

    @implementation ViewController
    @synthesize btnPlay, btnRecord,btnStop,songDuration, timerDuration, debugLabel;


    - (void)viewDidLoad
    {
        debugLabel.text = @"";
        songName =[[NSString alloc ]init];
        //NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
        //NSString *documentsDirectory = [paths objectAtIndex:0];
        songName = @"TestingQueue.caf";



        [super viewDidLoad];
        // Do any additional setup after loading the view, typically from a nib.
    }
    - (void)prepareAudioQueue
    {
        //struct AQRecorderState *pAqData;
        inputBuffer=0;
        aqData.mDataFormat.mFormatID         = kAudioFormatLinearPCM;
        aqData.mDataFormat.mSampleRate       = 44100.0;
        aqData.mDataFormat.mChannelsPerFrame = 1;
        aqData.mDataFormat.mBitsPerChannel   = 16;
        aqData.mDataFormat.mBytesPerPacket   =
        aqData.mDataFormat.mBytesPerFrame = aqData.mDataFormat.mChannelsPerFrame * sizeof (SInt16);
        aqData.mDataFormat.mFramesPerPacket  = 1;

        //    AudioFileTypeID fileType             = kAudioFileAIFFType;
        AudioFileTypeID fileType             = kAudioFileCAFType;
        aqData.mDataFormat.mFormatFlags = kLinearPCMFormatFlagIsBigEndian| kLinearPCMFormatFlagIsSignedInteger| kLinearPCMFormatFlagIsPacked;

        AudioQueueNewInput (&aqData.mDataFormat,HandleInputBuffer, &aqData,NULL, kCFRunLoopCommonModes, 0,&aqData.mQueue);

        UInt32 dataFormatSize = sizeof (aqData.mDataFormat);

        // in Mac OS X, instead use
        //    kAudioConverterCurrentInputStreamDescription
        AudioQueueGetProperty (aqData.mQueue,kAudioQueueProperty_StreamDescription,&aqData.mDataFormat,&dataFormatSize);

        //Verify
        NSFileManager *fileManager = [NSFileManager defaultManager];
        NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
        NSString *documentsDirectory = [paths objectAtIndex:0];
        NSString *txtPath = [documentsDirectory stringByAppendingPathComponent:songName];

        NSLog(@"INITIALIZING FILE");
        if ([fileManager fileExistsAtPath:txtPath] == YES) {
            NSLog(@"PREVIOUS FILE REMOVED");
            [fileManager removeItemAtPath:txtPath error:nil];
        }


        const char *filePath = [txtPath UTF8String];
        CFURLRef audioFileURL = CFURLCreateFromFileSystemRepresentation ( NULL,(const UInt8 *) filePath,strlen (filePath),false );
        AudioFileCreateWithURL (audioFileURL,fileType,&aqData.mDataFormat, kAudioFileFlags_EraseFile,&aqData.mAudioFile );

        DeriveBufferSize (aqData.mQueue,aqData.mDataFormat,0.5,&aqData.bufferByteSize);

        for (int i = 0; i < kNumberBuffers; ++i)
        {
            AudioQueueAllocateBuffer (aqData.mQueue,aqData.bufferByteSize,&aqData.mBuffers[i]);
            AudioQueueEnqueueBuffer (aqData.mQueue,aqData.mBuffers[i], 0,NULL );
        }

    }

    - (void) metronomeFire
    {
        if(beatNumber < 5)
        {
            //count in time.
            // just play the metro beep but don't start recording
            debugLabel.text = @"count in (1,2,3,4)";
            [self playSound];
        }
        if(beatNumber == 5)
        {
            //start recording
            aqData.mCurrentPacket = 0;
            aqData.mIsRunning = true;
            startTime = [NSDate timeIntervalSinceReferenceDate];
            recordTimer = [NSTimer scheduledTimerWithTimeInterval:4.8 target:self selector:@selector(killTimer) userInfo:nil repeats:NO];
            AudioQueueStart (aqData.mQueue,NULL);
            debugLabel.text = @"Recording for 8 beats (1,2,3,4 1,2,3,4)";
            [self playSound];
        }
        else if (beatNumber < 12)
        {   //play metronome from beats 6-16
            [self playSound];
        }
        if(beatNumber == 12)
        {
            [metroTimer invalidate]; metroTimer = nil;
            [self playSound];
        }

        beatNumber++;

    }
    - (IBAction) play
    {
        NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
        NSString *documentsDirectory = [paths objectAtIndex:0];
        NSString *txtPath = [documentsDirectory stringByAppendingPathComponent:songName];
        NSURL *url = [NSURL fileURLWithPath:[NSString stringWithFormat:@"%@",txtPath]];

        if (audioPlayer)
        {
            [audioPlayer stop];
            audioPlayer = nil;
        }
        NSError *error;
        audioPlayer = [[AVAudioPlayer alloc] initWithContentsOfURL:url error:&error];

        if (audioPlayer == nil)
        {
            NSLog(@"%@",[error description]);
        }
        else
        {
            [audioPlayer play];
            [audioPlayer setNumberOfLoops:-1];
        }
    }
    - (void) killTimer
    {
        //this is the timer function.  Runs once after 4.8 seconds.
       [self stop];

    }
    - (IBAction) stop
    {
        if (audioPlayer)
        {
            [audioPlayer stop];
            audioPlayer = nil;



        }
        else
        {

            if(metroTimer)
            {
                [metroTimer invalidate];metroTimer = nil;
            }
            //Stop the audio queue
            AudioQueueStop (aqData.mQueue,true);
            aqData.mIsRunning = false;
            AudioQueueDispose (aqData.mQueue,true);
            AudioFileClose (aqData.mAudioFile);

            //Get elapsed time of timer
            endTime = [NSDate timeIntervalSinceReferenceDate];
            elapsedTime = endTime - startTime;

            //Get elapsed time of audio file
            NSArray *pathComponents = [NSArray arrayWithObjects:
                                       [NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) lastObject],
                                       songName,
                                       nil];
            NSURL *audioFileURL = [NSURL fileURLWithPathComponents:pathComponents];
            AVURLAsset* audioAsset = [AVURLAsset URLAssetWithURL:audioFileURL options:nil];
            CMTime audioDuration = audioAsset.duration;
            float audioDurationSeconds = CMTimeGetSeconds(audioDuration);

            //Log values
            NSLog(@"Track Duration: %f",audioDurationSeconds);
            NSLog(@"Timer Duration: %.6f", elapsedTime);

            //Show values on GUI too
            songDuration.text = [NSString stringWithFormat: @"Track Duration: %f",audioDurationSeconds];
            timerDuration.text = [NSString stringWithFormat:@"Timer Duration: %@",[NSString stringWithFormat: @"%.6f", elapsedTime]];
            debugLabel.text = @"Why is the duration of the track less than the duration the timer ran?";
        }


    }
    -(void) playSound
    {
        NSString *path = [[NSBundle mainBundle] pathForResource:@"blip2" ofType:@"aif"];
        SystemSoundID soundID;
        AudioServicesCreateSystemSoundID((__bridge CFURLRef)[NSURL fileURLWithPath:path],  &soundID);
        AudioServicesPlaySystemSound (soundID);
    }

    - (IBAction) record
    {
        [self prepareAudioQueue];
        songDuration.text = @"";
        timerDuration.text = @"";
        //debugLabel.text = @"Please wait 12 beats (The first four are count in)";
        //init beat number
        beatNumber = 1;

        //safe guard
        if(aqData.mIsRunning)
        {
            AudioQueueStop (aqData.mQueue,true);

            aqData.mIsRunning = false;

            AudioQueueDispose (aqData.mQueue,true);
            AudioFileClose (aqData.mAudioFile);
        }

        //start count in (metro will start recording)
        //aqData.mCurrentPacket = 0;
        //aqData.mIsRunning = true;
        startTime = [NSDate timeIntervalSinceReferenceDate];
        metroTimer = [NSTimer scheduledTimerWithTimeInterval:.6 target:self selector:@selector(metronomeFire) userInfo:nil repeats:YES];
        //recordTimer = [NSTimer scheduledTimerWithTimeInterval:4.8 target:self selector:@selector(killTimer) userInfo:nil repeats:NO];
        //AudioQueueStart (aqData.mQueue,NULL);

    }
    static void HandleInputBuffer (void *aqData,AudioQueueRef inAQ,AudioQueueBufferRef inBuffer,const AudioTimeStamp *inStartTime,UInt32 inNumPackets,const AudioStreamPacketDescription *inPacketDesc)
    {
        //boiler plate
        NSLog(@"HandleInputBuffer");

        struct AQRecorderState *pAqData = (struct AQRecorderState *) aqData;

        if (inNumPackets == 0 && pAqData->mDataFormat.mBytesPerPacket != 0)
            inNumPackets = inBuffer->mAudioDataByteSize / pAqData->mDataFormat.mBytesPerPacket;

        if (AudioFileWritePackets (pAqData->mAudioFile,false,inBuffer->mAudioDataByteSize,inPacketDesc,pAqData->mCurrentPacket,&inNumPackets,inBuffer->mAudioData) == noErr)
        {
            pAqData->mCurrentPacket += inNumPackets;
        }

        if (pAqData->mIsRunning == 0)
            return;

        AudioQueueEnqueueBuffer (pAqData->mQueue,inBuffer,0,NULL);
    }

    void DeriveBufferSize(AudioQueueRef audioQueue,AudioStreamBasicDescription ASBDescription,Float64 seconds,UInt32 *outBufferSize)
    {
        //boiler plate
        static const int maxBufferSize = 0x50000;
        int maxPacketSize = ASBDescription.mBytesPerPacket;
        if(maxPacketSize == 0)
        {
            UInt32 maxVBRPacketSize = sizeof(maxPacketSize);
            AudioQueueGetProperty(audioQueue, kAudioQueueProperty_MaximumOutputPacketSize, &maxPacketSize, &maxVBRPacketSize);
            NSLog(@"max buffer = %d",maxPacketSize);
        }
        Float64 numBytesForTime = ASBDescription.mSampleRate * maxPacketSize * seconds;
        *outBufferSize = (UInt32)(numBytesForTime < maxBufferSize ? numBytesForTime : maxBufferSize);
    }

    OSStatus SetMagicCookieForFile (AudioQueueRef inQueue, AudioFileID inFile)
    {
        //boiler plate
        OSStatus result = noErr;
        UInt32 cookieSize;
        if (AudioQueueGetPropertySize (inQueue,kAudioQueueProperty_MagicCookie,&cookieSize) == noErr)
        {
            char* magicCookie =(char *) malloc (cookieSize);
            if (AudioQueueGetProperty (inQueue,kAudioQueueProperty_MagicCookie,magicCookie,&cookieSize) == noErr)
            {
                result =    AudioFileSetProperty (inFile,kAudioFilePropertyMagicCookieData,cookieSize,magicCookie);
            }

            free (magicCookie);
        }
        return result;

    }













    - (void)didReceiveMemoryWarning
    {
        [super didReceiveMemoryWarning];
        // Dispose of any resources that can be recreated.
    }
    @end

Answer 1

This is a big topic so I doubt you'll get an answer big enough to re-architect the code that you've provided. However, I can give you links that will supply the vast majority of what you require.

First thing is NSTimer would never work due to synchronisation issues. Also, forget AudioQueue and AVAudioRecorder. Only AudioUnit is low level enough for your needs.

Have a look at my answer here:

iOS Stream Audio from one iOS Device to Another

But the true goldmine - and knowledge that you will need to be intimately familiar with - is Tasty Pixel's blog. Tasty Pixel being the vendor of Loopy HD, but also someone that is kind enough to share some pretty in depth knowledge.

See:

A simple, fast circular buffer implementation for audio processing

Developing Loopy, Part 2: Implementation

and

Using RemoteIO audio unit

Finally, make sure you are familiar with packets, frames, samples, etc. Everything needs to sync perfectly.