This is a fairly subjective question and isn't really a good fit for SO but i'll give it a shot.
Depending on your implementation, you may find yourself far more limited by things like bandwidth and latency. If I were you, I'd write it as much as possible in a language you were comfortable with, profile it, and come back and rewrite the slow parts in something faster, with optimization in mind.
This is the procedure of optimization I'd follow for projects expected to be CPU bound in general.
Since you're doing it to grasp C++ better, it might benefit you to use C++ instead of java for the whole thing, and optimize parts of it later, but a neural net might not be a very "beginner friendly" project. It's possible you'd end up with a train wreck caused by a design better suited for another language. You'd still probably learn from it, but you wouldn't be able to use it after you finished.