The basic code when you add/remove a component from a visible GUI is:
panel.remove(...);
panel.add(...);
panel.revalidate();
panel.repaint();
The above code should be done in the ActionListener, NOT in the paintComponent() method. Painting methods are for painting only.