The reason for the debugger stopping execution is that "Break on user-unhandled exceptions" will break on any exception that is not handled by a handler in user code ("My Code").
Since the TPL, which is the one catching the exception, is not considered user code, the debugger will consider the exception unhandled by user code.
The problem is, the TPL does not have any other way to trigger TaskContinuationOptions.OnlyOnFaulted, so I'm afraid you're probably stuck seeing the exceptions.
Hopefully, getting exceptions isn't part of your normal flow. If it's not, seeing the exceptions as unhandled in the debugger may actually be a good thing for debugging errors in your task flow.