The problem with this is that the certificate information is in the IDP metadata that is sent to the SP and the metadata normally only allows for one certificate for one task (can be different tasks e.g. for signing and encryption).
Coming back the other way e.g. signing the SP Authn request, all the SP's can have different cetificates or they can share.
Some products e.g. ADFS 2.0 before Rollup 3 do not allow SP to share certificates.