Question

I'm working on some Android platforms (>= 4.1.1) where the openSL ES audio driver exhibits quite irregular callback patterns.

My expectation was that if I configure the audio driver to give me callbacks every 10ms, I would get them approximately every 10ms (give or take some milliseconds). Ideally, the callback pattern would look something like this:

t = 0ms : speaker callback

t = 1ms : mic callback

t = 10ms : speaker callback

t = 11ms : mic callback

t = 20ms : speaker callback

t = 21ms : mic callback

t = 30ms : speaker callback

t = 31ms : mic callback

The microphone callback takes the received microphone data and writes it to a ring buffer. It then sends a 'signal' to another thread to wake up and process the microphone data. The processing of the microphone data results in the generation of 10ms of speaker data. That speaker data is written to a speaker ring buffer, which the speaker callback reads from.

If the callback pattern looks like what I've described above where the speaker and microphone callback takes turns, everything works out fine.

However, if the callback pattern is irregular things start to get messed up. For example : A burst of microphone callbacks will drive the size of the speaker ring buffer through the roof. If I - for whatever reason - don't get the same kind of burst of speaker callbacks, I suddenly have large latency in the speaker path.

Another problem is if I get a burst of speaker callbacks. In that case the speaker ring buffer will run dry of samples and I'll have to return silence packets instead.

So I'm wondering if there's some kind of standard solution for this type of problem? I can't think of any.

The following link is an example of the callback pattern:

http://wikisend.com/download/143908/timestamps.txt

where '1' is microphone callback and '2' is speaker callback.

No correct solution

OTHER TIPS

you might check out audio-echosample, in README.md, irregularity of the callback is a factor mentioned there too. Put your audio into the platform fast audio path will improve the callback irregularity, it is platform dependent, might worth a try.

I'm just starting to use OpenSL ES so this may turn out to be a dumb/incorrect answer, but... Don't use the speaker callbacks. Then your ring buffer goes away too. Just enqueue your playback buffers directly in the microphone callback.

This may at least get around some of the issues caused by irregular callbacks. I think you'll still need some kind of synchronization logic to control latency between the mic and speaker. The irregular callbacks sound like a sign of performance issues, try raising thread priority. Of course since Android/Linux is not a realtime system your app will still occasionally have to deal with gaps or bursts in callbacks (throw away data if too much, pad if too little). Read the buffer queue status to see how full its buffer is.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top