Because the operating system also needs RAM, for cache, buffers, daemons such as syslogd, page tables, kernel data structures etc.
Also, the JVM designers have no idea what other applications you might want to start after the JVM starts up. So it is sensible for the JVM to not hog all the RAM by default.