You are almost there the sole problem is focus. However you can still achieve the same without focusing the control by using the PreviewKeyDown event so just change your code to use the same.
PreviewKeyDown += PreviewKeyDownHandler;
public void PreviewKeyDownHandler(object sender, PreviewKeyDownEventArgs e)
{
switch (e.KeyData)
{
case Keys.Right:
panel1.Location = new Point(panel1.Location.X + 5, panel1.Location.Y);
Invalidate();
break;
}
}
this will work regardless of focus on the control.