It would have been better if you had posted the keytool sentences you've used to generate the certificates, but I guess that you were trying to generate a selfsigned certificate. For that, first you should generate a public - private key pair, the command to achieve it looks like:
keytool -genkey -alias yourAliasOrDomainName -keyalg RSA -keystore yourKeyStoreName.jks -keysize 2048
To get the certificate (public key certificate) you've to use the export command (that's the file you've to issue to the client)
keytool -export -alias yourAliasOrDomainName -keystore youKeyStoreName.jks -file youServer.cer
And finally, you've to place the keystore file in your server, and set up the HTTP / SSL Connector, specifying the keystore file and its password (but not the domain password). Something like:
<Connector protocol="HTTP/1.1" SSLEnabled="true"
port="8443" address="${jboss.bind.address}"
scheme="https" secure="true" clientAuth="false"
keystoreFile="${jboss.server.home.dir}/conf/yourKeyStoreName.jks"
keystorePass="yourKeyStorePassWord" sslProtocol = "TLS" />
Note that in the Connector you've to specify the Keystore file, not the certificate!!!