How the graphics is displayed is machine dependent, so therte is no definite answer. For example on the C64, the graphics hardware was mapped into the normal address space, So you had to write to a specific part of the memory in order to print characters on the screen. If you wanted to display graphics, then you had to switch the mode, by writing to registers of the display hardware, and also the mapped memory could change. Because of this, the normally accessible memory of the C64 was lower than the 64KB. You could switch off the memory mapping though, and thus get access to the full memory below the graphics memory, so it turned into a machine with no display.
However, on the PC you had e.g. VGA, EGA, Hercules cards and so, which were written to by accessing a specific port and sending commands to it via these ports. A totally different approach. But this is a system design decision, and doesn't depend on the CPU.
As a matter of fact, what would happen if a program compiled for an Apple II was somehow executed on a C64? The machine should be able to read the instructions, right?
Well, the answer is pretty clear. It would most likely crash, because even if the instruction set might be the same (I don't know which CPU the Apples had), the hardware details would differ.