The GRANT privilege is a separate privilege. As the MySQL documentation says, the GRANT privilege "Enable privileges to be granted to or removed from other accounts".
If you don't give your friend the GRANT privilege, he won't be able to change your privileges. It is only because you have the GRANT privilege that you can change his privileges.
See http://dev.mysql.com/doc/refman/5.1/en/grant.html#grant-privileges for details on how privileges work in MySQL.