The first for
loop has a problem. The condition is checking for i > keylen
when it should be checking for i < keylen
.
Also when computing the next output value, the steps should be
(p[i]-'A') results in a number between 0 and 25
adding (key[i % keylen]) results in a number between 0 and 50
apply modulo 26 so the number is between 0 and 25 (this is the missing step)
then add 'A' to get the output
Note: Avoid using hard-coded numbers for character constants. For example, use 'A'
instead of 65, and 'a'
instead of 97.