Welcome to the joy of embedded and electronics!
What you are experiencing is called contact bounce: http://www.elexp.com/t_bounc.htm
It is typical in all uses of buttons and is caused by the electromechanical characteristics of the button.
There are 2 ways of dealing with this. Either add a debouncing circuitry or in software. Typically the software way this is to ignore all interrupts that occurs withing a few milliseconds of the first. There is a very good paper on debouncing strategies here: http://cseweb.ucsd.edu/classes/sp09/cse140L/slides/debouncing.pdf