I solved this by adding a passphrase parameter to the AddIdentityFile()
command.
You might have added a passphrase when you generated your private key, include it in your call.
Sftp sftp = new Sftp(host, user);
//Include it here
sftp.AddIdentityFile(keyFileName, "yourpassphrase");
sftp.Connect(22);