You need to validate the responce according to the SAML spec. There are some functionaliy for doing this in OpenSAML but i seems the safest bet is to write your own validation code. see. http://marc.info/?t=137354098500007&r=1&w=2
You must also validate signature. As with all signature verification you use the public key. Here is some I wrote on my blog about OpenSAML signatur verification. https://blog.samlsecurity.com/2012/11/verifying-signatures-with-opensaml.html
I have more on signing and encryption using OpenSAML in my book, A Guide to OpenSAML