JPopupMenu change size dynamically if there is a JPanel inside

Question 1

Thanks all, but I've found the solution after continuous debugging. The problem was in layout used in JPanel (between Popup and items). It called the getPreferredSize() of JPanel which called directly menuItem.getPrefferedSize(), which called BasicMenuItemUI.getPrefferedSize(). The last method uses the MainMenuLayoutHelper class to get the preferred size. This class stores the data about size in Properties static object.

The default JPopupMenu layout - DefaultMenuLayout - clears this static data each time its preferredLayoutSize() method called, with call MenuItemLayoutHelper.clearUsedClientProperties(popupMenu). We will do the same - call this method with our JPanel parameter with further revalidate() call:

public class PopupTest2 {
    public static void main(String[] a) {
        final JFrame frame = new JFrame();
        frame.setSize(500, 500);
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);

        final JPanel panel = new JPanel(new BorderLayout());

        final JPopupMenu menu = new JPopupMenu();

        final JPanel itemPanel = new JPanel();
        itemPanel.setLayout(new BoxLayout(itemPanel, BoxLayout.Y_AXIS));

        final JMenuItem[] items = new JMenuItem[1];
        for (int i = 0; i < 1; i++) {
            JMenuItem item = new JMenuItem("Item #"+String.valueOf(i));
            itemPanel.add(item);
            items[i] = item;
        }
        menu.updateUI();

        menu.add(itemPanel);

        JToggleButton button = new JToggleButton("Press me");
        button.addActionListener(new ActionListener() {
            boolean pressed = false;
            @Override
            public void actionPerformed(ActionEvent e) {
                pressed = !pressed;
                if (pressed) {
                    for (JMenuItem item : items) {
                        item.setText(item.getText()+" changed");
                    }
                } else {
                    for (JMenuItem item : items) {
                        item.setText(item.getText().substring(0, item.getText().length() - 8));
                    }
                }
            }
        });

        panel.add(button, BorderLayout.NORTH);

        panel.addMouseListener(new MouseAdapter() {
            @Override
            public void mouseClicked(MouseEvent e) {
                if (e.getButton() == MouseEvent.BUTTON3) {
                    MenuItemLayoutHelper.clearUsedClientProperties(itemPanel);
                    itemPanel.revalidate();
                    menu.show(panel, (int) (e.getX() - menu.getPreferredSize().getWidth()), e.getY());
                }
            }
        });
        frame.setContentPane(panel);
        frame.setUndecorated(true);
        frame.setBackground(new Color(50, 50, 50, 200));

        SwingUtilities.invokeLater(new Runnable() {
            @Override
            public void run() {
                frame.setVisible(true);
            }
        });
    }
}

PS: Dear reader, it is not the best way to write your code. I do that because I really need that (I have such requirements). If you can build your JPopupMenu without JPanel inside, do not use code in this answer, please.

Question 2

Here is one way to do this:

public static void main(String[] a) {
    final JFrame frame = new JFrame();
    frame.setSize(500, 500);
    frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);

    final JPanel panel = new JPanel(new BorderLayout());
    panel.setBorder(BorderFactory.createLineBorder(Color.RED));

    final JPopupMenu menu = new JPopupMenu();

    // critical step
    final JPanel itemPanel = new JPanel();
    itemPanel.setLayout(new BoxLayout(itemPanel, BoxLayout.Y_AXIS));

    final JMenuItem[] items = new JMenuItem[10];
    for (int i = 0; i < 10; i++) {
        JMenuItem item = new JMenuItem("Item #"+String.valueOf(i));
        itemPanel.add(item);
        items[i] = item;
    }

    menu.add(itemPanel);

    JToggleButton button = new JToggleButton("Press me");
    button.addActionListener(new ActionListener() {
        boolean pressed = false;
        @Override
        public void actionPerformed(ActionEvent e) {
            pressed = !pressed;
            if (pressed) {
                for (JMenuItem item : items) {
                    item.setText(item.getText()+" changed");
                    item.setMaximumSize(new Dimension(70, 50));
                    item.setPreferredSize(new Dimension(70, 50));
                    item.setMinimumSize(new Dimension(70, 50));
                    itemPanel.invalidate();
                }
            } else {
                for (JMenuItem item : items) {
                    item.setText(item.getText().substring(0, item.getText().length() - 8));
                    item.setMaximumSize(new Dimension(130, 50));
                    item.setPreferredSize(new Dimension(130, 50));
                    item.setMinimumSize(new Dimension(130, 50));
                    itemPanel.invalidate();
                }
            }
        }
    });

    panel.add(button, BorderLayout.NORTH);

    panel.addMouseListener(new MouseAdapter() {
        @Override
        public void mouseClicked(MouseEvent e) {
            if (e.getButton() == MouseEvent.BUTTON3) {
                menu.show(panel, (int) (e.getX() - menu.getPreferredSize().getWidth()), e.getY());
            }
        }
    });
    frame.setContentPane(panel);
    frame.setUndecorated(true);
    frame.setBackground(new Color(50, 50, 50, 200));

    SwingUtilities.invokeLater(new Runnable() {
        @Override
        public void run() {
            frame.setVisible(true);
        }
    });
}

Question 3

You have to add

menu.updateUI();

after add menu items.

    final JMenuItem[] items = new JMenuItem[10];
    for (int i = 0; i < 10; i++) {
        JMenuItem item = new JMenuItem("Item #"+String.valueOf(i));
        itemPanel.add(item);
        item.setUI(new MyUI());
        items[i] = item;
    }
    menu.updateUI(); <<<<<<--------

    menu.add(itemPanel);