scaleTimeRange has no effect on audio type AVMutableCompositionTrack

Question

I am working on a simple video editing app. I want to add slow motion to my app. I notice there is a scaleTimeRange method in the AVMutableCompositionTrack class, so I use it to achieve my purpose. I found scaleTimeRange works very well on video track, but has no any effect on audio track. This means the audio track still plays in original speed.

Follow is my code:

    CMTime insertionPoint = kCMTimeZero;

    double wholeDuration = CMTimeGetSeconds([asset duration]);
    double doubleDuration = CMTimeGetSeconds([asset duration])*2.0;
    CMTime trimmedDuration = CMTimeMakeWithSeconds(wholeDuration, 600.0);

    // Create a new composition
    self.mutableComposition = [AVMutableComposition composition];

    // Insert video and audio tracks from AVAsset
    if(assetVideoTrack != nil) {
        AVMutableCompositionTrack *compositionVideoTrack = [self.mutableComposition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
        [compositionVideoTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, trimmedDuration) ofTrack:assetVideoTrack atTime:insertionPoint error:&error];

        [compositionVideoTrack scaleTimeRange:CMTimeRangeMake(kCMTimeZero, trimmedDuration) toDuration:CMTimeMakeWithSeconds(doubleDuration, 600.0)];
    }
    if(assetAudioTrack != nil) {
        AVMutableCompositionTrack *compositionAudioTrack = [self.mutableComposition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];
        [compositionAudioTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, trimmedDuration) ofTrack:assetAudioTrack atTime:insertionPoint error:&error];

        [compositionAudioTrack scaleTimeRange:CMTimeRangeMake(kCMTimeZero, trimmedDuration) toDuration:CMTimeMakeWithSeconds(doubleDuration, 600.0)];
    }

Can anyone give me some advice about the problem? Thank you!

Answer 1

ScaleTimeRange does not work for audio due to a bug in apple api (Bug ID : 14616144). However, time scaling audio could be done using resampling. If audio is sampled at 48 kHz, resample to 96 kHz and play the audio at 48 kHz this will take twice as long to play. Generally:

scaledSampleRate = (orignalSampleRate / playRate);
playRate = (originalSampleRate / scaledSampleRate);

The pitch will be lowered though, if this is not desired. The solution might be a third party api. If you can extract the audio to a separate player i guess you could use AVAudioPlayer, where you can set the play-rate:

player = [[AVAudioPlayer alloc] initWithContentsOfURL:
                  [NSURL fileURLWithPath:path] error:&err];
        player.volume = 0.4f;
        player.enableRate=YES;
        [player prepareToPlay];
        [player setNumberOfLoops:0];
        player.rate=2.0f;
        [player play];

Answer 2

To be able to play slow mo with sound just set appropriate audioTimePitchAlgorithm for AVPlayerItem

playerItem.audioTimePitchAlgorithm = AVAudioTimePitchAlgorithmVarispeed;

Answer 3

If you are working with scaleTimeRange and wondering why is it not working or playing any audio then you should probably add this to your playerItem.

playerItem.audioTimePitchAlgorithm = AVAudioTimePitchAlgorithm.varispeed