If I understand correctly, you would like to have your button panel at the south of your frame, and your buttons to be at the right of the button panel.
To do that, I would use a horizontal BoxLayout, containing a glue as its first element, to layout the buttons inside their button panel. Then I would add this button panel at the south of the content pane.
Read how to use those two layout managers in the swing tutorial, because you're confused about how a BorderLayour works: you can't add something at the south and at the east of such a layout.
Note that the tutorial page about the BoxLayout has an example that does exactly what you want.