You're likely to validate by its mime type
(which might be useless, since if a mimetype isn't registered, it will pick the default one for a binary unknown type, from the browser perspective).
In the docs, its referred as such:
["starts-with", "$Content-Type", "image/"],
I'm not sure if sets are allowed. At the very least, you could combine several policies, one for each allowed mime type.
As for Extension, I guess you have no options, though.
Edit: I've wrote a post a while ago outlining how the whole process works and how to use it (Disclaimer: I write for Newvem)