There is no clear answer for what is the "best" to use. If you're just getting started, just pick one and go. Cowboy and Yaws are under the most active development as "full featured" Erlang webservers, and so if your limitation is to use Windows, just use Cowboy.
Each server has its own configuration for tuning, and if you're comfortable tuning and developing for one, Nitrogen provides the mechanism to use your server of choice.
As for not providing a Windows download for Nitrogen/Yaws, Yaws is not done in "pure erlang" (there are some C files that need compiling), and as a result, compiling Yaws on Windows is more complicated, so in order to just get the build out there, Yaws was left out on Windows. Enough time was spent optimizing Nitrogen's build in Windows that I just wasn't going to spend any more time trying to get Yaws running with Nitrogen on Windows. I wanted the release done, and so Yaws was left to the wayside for a future version.