I came across the same issue and it seems to be a bug (or maybe by design, for caching purposes) with reusing a CIDetector.
I was able to get around it by not reusing the CIDetector, instead instantiating one as needed and then releasing it (or, in ARC terms, just not keeping a reference around) when the detection is completed. There is some cost to doing this, but if you are doing the detection on a background thread as you said, that cost is probably worth it when compared to unbounded memory growth.
Perhaps a better solution would be, if you a detecting multiple images in a row, to create one detector, use it for all (or maybe, if the growth is too large, release & create a new one every N images. You'll have to experiment to see what N should be).
I've filed a Radar bug about this issue with Apple: http://openradar.appspot.com/radar?id=6645353252126720