CPU memory for each program

Question 1

I'm assuming you are referring to registers.

Each CPU core only has one set of the registers. When the OS changes which process is running on the processor right now it saves the state of the registers somewhere is main RAM memory, and restores their value when beginning execution again.

You have to remember that programs on one CPU core are not executing in parallel, they are just being run a little and switched between so fast that it seems parallel to a human. That's why there can be only one set of registers.

Per the comments: read more here.

Question 2

You're probably thinking about cache, which all CPUs have a few KB to a few MB of near them, depending on the level (L1 cache, L2 cache, etc.). Ideally the CPU can use data in cache, since it is orders of magnitude faster to access than main memory. But because they are so small, they do indeed fill quickly. If the CPU can't find what it needs in one level of cache, it goes to the next level up. If it hasn't found the data in the highest level of cache available, it'll go out to main memory, swap that data into some portion of cache, and continue working.

There are various algorithms used to determine how to manage the data in cache, what to swap out, etc.

Question 3

Typically, a CPU has a single set of registers.

In order to use them over many processes ('programs' as you mention in your question), the OS saves the entire context of the current process in execution, whenever it switches to another process.

This procedure is called Context-Switch, and it includes saving the values of these registers.

When the OS switches back to a process, it reloads the registers with the previous values.

Question 4

Actually on bigger CPUs like intel x86 the registers and also instructions implementation and internal CPU logic are stored in special high-speed memory, this is Microcode layer.

Look: http://en.wikipedia.org/wiki/Microcode