Java alone does not enable this kind of separation. As soon as a class needs to be public, its public to the world.
I feel with you, as I don't like packages with hundreds of classes, so I like to substructure them into subpackages. Unfortunately this interferes with the only hiding mechanism for classes, using package private classes.
The way one can solve this in practice is usually by separating the code into separate build units (projects, essentially). You can make a separate project for each layer, and only add the layers intended for access as dependecies.