I have trouble understanding this concept
It's not something that can be thoroughly covered in a short description, especially with all the clarification questions that are likely to occur to you.
My suggestion is to take one of these webinars:
GPU Computing using CUDA C – Advanced 1 (2010)
CUDA Global Memory Usage & Strategy + Live Q&A with Dr Justin Luitjens, NVIDIA
Then come back when you have specific questions that are based on a general understanding of the topic.