Regarding this question:
What about specific ... CUDA Driver or API functions that help me to find out which ... NVENC hardware is Idle?
There are none.
NVENC and CUDA are orthogonal. The APIs to access and manage CUDA are completely separate from the APIs to access and manage NVENC. The code you've shown here identifies what GPUs are in the system, and happens to use the CUDA API to do that, but your question otherwise has nothing to do with CUDA. There are no CUDA API functions that will allow you to do anything with NVENC hardware.
Ultimately, managing the NVENC hardware is the responsibility of the client application that is using the NVENC API. This client application can easily make a busy/non-busy status available upstream to other applications if desired (since it knows the individual status of the hardware it is managing.