In ARM, the page tables have multiple levels. The first level has 4096 entries as far as I can remember - each describing a 1MB section.
From there you can break up the 1MB section into 4KB sizes (or larger - I can't remember what other sizes) by giving a pointer to a second level page table containing 256 entries describing 4KB sections.
The top level 1MB sections all have to be mapped - whether they have to be mapped to something is decided to the OS. If an area isn't needed, some OS's will just fill in the entry to page fault.
You certainly don't want to divide up the entire memory space into 4KB chunks - the page table size would be enormous.