Do all Class entries in the constant pool table refer to a class that is used somewhere in the class file?
No, you can always put in constant pool entries that aren't actually used. A class compiled with a standard compiler will only contain entries that are actually used though.
Or are there other mechanisms how a Class entry may get into the constant pool?
No, the constant pool is fixed. Though the file on your disk may not represent the actual class that is loaded, since it's always possible for a custom class loader or Java agent to manipulate things at runtime.
Assuming I wold not like to implement some kind of "lazy class loading" on first use of a class, am I therefore loading the correct list of used classes by using the class entries in the constant pool table?
For static analysis, that's the best you can do. Note that you will miss dependencies used through reflection, but there's not much you can do about that.