The problem is that when I have selected values and click on the list again the old selection is lost, new item is selected.
Alt+mousePressed is used to change the selection. This is a common usage of the Alt key. Try playing with this key combination on any application (ie. Windows Explorer) to see how it works.
So a better implementation would probably be to use a different mouse+key combination. I'm not sure what the standards are but I would suggest that for maybe just the "Delete" key could be used delete the selected items. Of course you should popup a confirm dialog. For using the mouse I would use a popup menu that would display on a right click and the menu would contain a Delete action. Again a confirm dialog should display.
Remember any time you build an application the user should be able to use the Keyboard or the mouse to achieve an Action.