Does it make sense, security-wise? Sure. And in fact you can use the "xs:ID" portion of an assertion to assist you (my company's software does).
From Page 9 of CORE:
The xs:ID simple type is used to declare SAML identifiers for assertions, requests, and responses. Values declared to be of type xs:ID in this specification MUST satisfy the following properties in addition to those imposed by the definition of the xs:ID type itself:
• Any party that assigns an identifier MUST ensure that there is negligible probability that that party or any other party will accidentally assign the same identifier to a different data object.
• Where a data object declares that it has a particular identifier, there MUST be exactly one such declaration.
We snatch that ID from an assertion, and drop it into an array with the not-after time, and then throw it out after that time expires. This way the same assertion can't be replayed.
In other software (especially home-grown stuff), this is entirely managed with the Not-Before and Not-On-Or-After portion of the Audience Restriction. Since some software counts solely on these values, the suggested method is to set this period as short as is reasonable. In the perfect world, everyone is using time servers, and their clock skew isn't more than a couple of seconds. A minute prior, and a minute post issue time should be far more than sufficient. While there isn't as much "security" here, it can be "managed".