Delete the empty keystoreFile.txt, then run your 2nd command again
[aniket@localhost bin]$ keytool -genkey -alias myKeyStore -keyalg RSA -keystore
/home/aniket/keystore/keystoreFile.txt
It's trying to create the file you specify with the -keystore
flag. The reason your first one command failed is that you gave it an existing directory name.