MVVM encourages separation of the logic from presentation so I would recommend using an Event Aggregation system similar to the Messenger subsystem in MVVMLight. You will be building a system that can be reused across your application and not tying on modal dialog to a specific source.
Your View binds to a ViewModel Command, fire an event, a subscriber ViewModel listens and catches, displays Flyout as a View.