I need to be able to track additions/deletions of Documents to the android file system.
Why is this in the user's interest? Remember: you are not important; your user is.
I am aware of ContentProviders and ContentObservers.
Note that these only work while your app is in memory for other reasons.
I see that I can set up a ContentObserver for media file type i.e. Audio, Video and Photos. However, I do not see something that could tell me about Document types.
Correct, because there is no "Document types" in Android. This is your own invention. You could just as easily be referring to "Snicklefritz types", defined as "files whose 18th byte is 0x4F".
Is there a more efficient way to achieve what I want without watching for changes for all files?
Not really. You are welcome to use FileObserver
to monitor changes to a file or a directory, but that will only be for that file or that directory, not the entire filesystem.
when I open "My Files" on an android device
There is no "My Files" in Android. There may be something called this on your specific Android device, running your specific firmware.