Is it possible to handle GPU priority between processes

Question

I know from CUDA 5.5 that it is possible to have high-priority kernels, but I understand that this is only for calls issues by the same context on the GPU, i.e. it does not effect the priority for another process' kernel launches, as long as they have enough CPU time to be issued.

Is it possible to have high priority applications on the GPU, similarly to how you can set the OS to give priority to specific threads?

Thanks

Henrik Andresen

Answer 1

Technically, CUDA 5.5 added the ability to set individual stream priorities to the CUDA work scheduling system for compute capability 3.5 devices. This means that the CUDA driver will allow any operation kernel in a higher priority stream to preempt the driver level scheduling of operations in competing lower priority streams. This can include launching of kernels, copy operations and events to the device. I don't, however, believe that extends to any difference in execution priority once a kernel has been launched on the device itself, it is purely a driver side stream scheduling heuristic control.. This extends to preemption of running kernels from lower priority streams on the device when conditions for concurrent kernel execution exist.

To the best of my knowledge, there is currently no ability to extend that mechanism beyond streams within a single context, and there is no way to influence how processes competing for access to the GPU will be prioritised at the driver level. The only caveat I add to this is that CUDA 6 might have changed this, and I haven't yet had the opportunity to look at everything that is new in that release.