The clipboard can only copy a file URI, not the file itself, and that too only if your application implements a ContentProvider and encodes an identifier on the URI.
Users cannot simply copy your file paths by default. You have to code this behaviour explicitly.
So, you don't need to clear the clipboard since file paths won't be copied to the clipboard in the first place.
Your files are safe.
Also see the official tutorial.