How to convert certificate from PEM to JKS?

Question 1

You aren't clear which files you combined, but it should work to use openssl to combine the cert and private key to a PKCS#12:

cat cert_public_key.pem cert_private_key.pem >combined.pem
openssl pkcs12 -export -in combined.pem -out cert.p12

or on the fly but (update:) the privatekey must be first:

cat cert_private_key.pem cert_public_key.pem | openssl pkcs12 -export -out cert.p12

If your cert needs any chain cert(s) -- the CA should have told you this when you submitted the CSR and they issued the cert -- it's easiest to also include it(them) now.

Then (1) some Java programs can actually use a pkcs12 directly as a keystore, but (2) if you need or prefer a JKS use keytool:

keytool -importkeystore -srckeystore cert.p12 -srcstoretype pkcs12 -destkeystore cert.jks

If you care about the alias in the resulting JKS, easiest to fix it after converting.

Also: just changing the labels in an encrypted PEM doesn't unencrypt it, nor does changing the label from generic PKCS#8 to RSA actually change the data to match (and they are different, though only a little). If you do want a separate PEM file with the decrypted private key:

openssl pkey -in encryptedpk8 -out clearpk8.pem # 1.0.0 up
openssl pkcs8 -in encryptedpk8 -out clearpk8.pem # 1.0.0 up 
openssl pkcs8 -topk8 -nocrypt -in encryptedpk8 -out clearpk8.pem # below 1.0.0
openssl rsa -in encryptedpk8 -out clearrsa.pem

Question 2

First question: you only have a certificate request? Not an actual certificate? It needs to be signed, you can self-sign it or have it signed by an external party.

If you have the actual cert you can use this to parse the private key file and the cert file:

// parse the private key
KeyFactory keyFactory = KeyFactory.getInstance("RSA"); // might not be RSA
PKCS8EncodedKeySpec spec = new PKCS8EncodedKeySpec(byteArray);
PrivateKey privateKey = keyFactory.generatePrivate(spec);

// parse cert
CertificateFactory factory = CertificateFactory.getInstance("X.509");
X509Certificate cert = factory.generateCertificate(certInputStream);

// add it to the keystore
store.setKeyEntry(alias, privateKey, password, new X509Certificate[] { cert });

UPDATE

As far as I know the command line keytool does not support any advanced options like signing a csr. Even standard java does not support this, you need an external library like bouncy castle. This is not easy. E.g:

JcaPKCS10CertificationRequest pkcs10 = new JcaPKCS10CertificationRequest(csrBytes);
X509v3CertificateBuilder builder = new JcaX509v3CertificateBuilder(
        issuer,
        generateSerialId(),
        new Date(),
        until,
        subject,
        pkcs10.getPublicKey()
);

X509CertificateHolder holder = builder.build(getContentSigner(privateKey, type));
X509Certificate cert = getCertificate(holder);

...

ContentSigner getContentSigner(PrivateKey privateKey) {
    AsymmetricKeyParameter keyParameter = PrivateKeyFactory.createKey(privateKey.getEncoded());
    AlgorithmIdentifier sigAlgId = new DefaultSignatureAlgorithmIdentifierFinder().find("SHA256WITHRSA"); // or what you want
    AlgorithmIdentifier digAlgId = new DefaultDigestAlgorithmIdentifierFinder().find(sigAlgId);
    return new BcRSAContentSignerBuilder(sigAlgId, digAlgId).build(keyParameter);
}