The vector's addresses, as any executable code location, are subject to MMU translation so you can put them anywhere in physical memory. Probably you can even have different vectors for different cores (via different MMU tables). Linux uses "high vectors" bit which puts them at 0xFFFF0000, but it's unlikely that a simple ARM machine would have over 3GB of RAM.
See also Relocate the ARM exception vectors?