The Java Virtual Machines are themselves applications that need to run on top of a hardware architecture and operating system and most likely they are not implemented in Java themselves.
That is the case of the popular Java HotSpot Virtual Machine (the default implementation from Oracle) which is mostly implemented in C/C++.
That means you need a compiled version of it for every hardware architecture and operating system in which you intend to use it, and so that explains that there are versions of it for 32-bit and 64-bit hardware architectures.
This is also the case of other JVM implementations, like JRockit, IBM J9, Azul Systems Zulu and probably many others.
Other helper programs aside the Java Virtual Machine, typically included for programmers to develop applications as part of what is know as the JDK (Java Development Kit) may also be developed in this way. That is the case of tools like the compiler (javac), the documentation generator (javadoc), the RMI compiler (rmic), the Java disassembler, (javap), etc.
So, those JDK tools would also require a hardware dependent implementation. And so that's why you are offered a choice when downloading the JDK.
--Edit--
Addressing the questions on the "portability" subject
It depends on what you mean by "portability". If you mean the Java's WORA (write once, run anywhere), then it has to be through a virtual machine like Java, Python or Ruby. But languages like C/C++ compile to machine code, and therefore, they are not run through a virtual machine, but by the hardware itself. This does not mean they are not portable, you may write your code in way that can be run in multiple architectures, it is just that you cannot use the same binaries. You have to recompile for every case, since the program must be written/compiled in a way a particular hardware/os understands.
This gap is what the virtual machines intend to close.
Now, portability can mean more than just using the same binaries. Even with Java you may write code that is not portable, perhaps because you use OS dependent features or because you mistakenly programmed paths using literals (i.e Linux /
vs Windows \
), etc.