You can use a MatteBorder
for the JToolBar
and just set the top edge
JToolBar toolBar = new JToolBar("ToolBar");
MatteBorder matteBorder = new MatteBorder(1, 0, 0, 0, Color.BLACK);
toolBar.setBorder(matteBorder);
This will give you a line on top of the tool bar, one pixel color black
Disregard the complete image. Its just some code I had handy with a menu bar in it.
Notice the line at the top of the toolbar to give more distinction in separation. You can play around with the pixel thickness and color. If you want it a little more subtle, tou can use a color like light_gray or gray. Your choice