I think it is a good practice for two reasons:
- From the cuBLAS Library User Guide, "cublasCreate() [...] allocates hardware resources on the host", which makes me think that there is some overhead on its call.
- Multiple cuBLAS handle creation/destruction can break concurrency by unneeded context synchronizations.