i have never had any success with the "test federation connectivity" function in OpenAM/OpenSSO.
What I have used to test that the federation is up and going is spSSOInit.jsp. It starts a SAML authentication for the user making a request to it.
https://youropenamserver/openam/saml2/jsp/spSSOInit.jsp?metaAlias=<The SP meta alias>&idpEntityID=<The IdP entity id>
meta alias and entity id can be found on the configuration page for SP and IDP respectively.