How do I remove the space that is automatically created between the panels?
Use a different layout manager for the panel. The GridLayout will always resize components to take up all the space in the panel.
Maybe you can use a BoxLayout or a GridBagLayout. You can also nest panels with different layout managers to get your desired effect.
Read the section from the Swing tutorial on Layout Managers for more information and examples.