With NAudio, you can make use of the MultiplexingSampleProvider
to route specific inputs to outputs. Read more about it here. This may meet your needs for multi-channel WAV playback.
NAudio does not currently include support for ogg playback, so you'd need to find an extension to do that (there are a few based on NVorbis).