Can we say that CLR assigns to App.exe process some amount of physical memory
You cannot say that, a process on Windows only allocates virtual memory. The mapping of virtual memory to physical memory (RAM) is strictly an operating system duty. RAM needs to be shared by all processes running on the machine and is done dynamically. The sum of the virtual memory allocations of all processes normally greatly exceeds the amount of RAM. When a process accesses virtual memory, a page fault gets the virtual memory page mapped to RAM. If necessary, data in RAM is discarded or stored in the paging file to make room.
AppDomains share a single set of GC heaps (generations 0 through 2 and the Large Object Heap) and their allocations are intermingled. They are kept apart by their roots, every AppDomain has its own heap for static variables, its own GCHandles and its own set of thread stack frames with local variables.