When you create an SSH key you have the option to add a passphrase to it. This is not mandatory, but it does provide an added layer of security.
You are being prompted for a passphrase because your key has has one.
It is possible to remove the passphrase if you want, but I'd recommend using an agent instead. When using an agent, you can give your passphrase once and you won't be promoted to enter it again for a certain amount of time.
Depending on your operating system and desktop environment you have a number of options for running an agent. OpenSSH provides this method:
ssh-add -t 5h
This will prompt you for your passphrase immediately, and then let you work for five hours without having to re-enter your passphrase. See the manpage for details.
The time formats understood by OpenSSH can be found in the manpage for sshd_config
.