Render CIImage to a specific rect in IOSurface

Question

I'm trying to render a CIImage to a specific location in an IOSurface using [CIContext render:toIOSurface:bounds:colorSpace:] by specifying the bounds argument r as the destination rectangle.

According to the documentation this should work, but CoreImage always render the image to the bottom-left corner of the IOSurface.

It seems to me like a bug in CoreImage.

I can overcome this problem by rendering the image to an intermediate IOSurface with the same size of the CIImage, and then copy the content of the surface to another surface. However, I would like to avoid the allocation and the copying in the solution.

Any suggestion?

Answer 1

What you want to happen isn't currently possible with that API (which is a huge bummer).

You can however wrap your IOSurface up as a texture (using CGLTexImageIOSurface2D) and then use CIContext's contextWithCGLContext:…, and then finally use drawImage:inRect:fromRect: to do this.

It's a huge hack, but it works (mostly): https://github.com/ccgus/FMMicroPaintPlus/blob/master/CIMicroPaint/FMIOSurfaceAccumulator.m

Answer 2

Since macOS 10.13 you can use CIRenderDestination and CIContext.startTask(toRender:from:to:at:) to achieve the same result without having to provide an intermediate image.

In my case I used a combination of Metal and Core Image to render only a subpart of the output image as part of my pipeline as follow:

let renderDst = CIRenderDestination(mtlTexture: texture, commandBuffer: commandBuffer)
try! context.startTask(toRender: ciimage,
                           from: dirtyRect, to: renderDst, at: dirtyRect.origin)

As I'm already synchronizing against the MTLCommandBuffer I didn't need to synchronize against the returned CIRenderTask.

If you want to more details you can check the slides (starting from 83) of Advances in Core Image: Filters, Metal, Vision, and More (WWDC video from 2017).