Even with a Bluetooth 4.0-compliant device, communication via standard ("high energy") Bluetooth with an iOS device is still only allowed with MFi-compliant devices. Bluetooth Low Energy communication with iOS devices is the part that's now completely open and unrestricted by Apple. If you wish to send data to another computing device (Windows, Mac, Android, etc.), Bluetooth LE is what you're going to need (short of someone reverse-engineering the Bonjour over Bluetooth PAN connections).
As of iOS 6.0, iOS devices can make themselves look like Bluetooth LE peripherals, so desktop computers set up as Bluetooth LE central devices can connect to them. You could put together your own profile for communication, since you'd control the iOS and Windows sides of things, or you could make your iOS device mimic a standard keyboard, heart rate sensor, etc.
As far as compatible dongles, the answers to this question list several Bluetooth LE dongles that are known to work with Core Bluetooth on the Mac. The CSR ones seem to be the most frequently cited there and among other people I've talked to. I can't speak for Windows support, but I'd assume there would be driver support there, and I hear Windows 8.1 expands support for Bluetooth LE.
I do have the BLED112 (the Bluegiga USB dongle), and that shows up as a comm port when its drivers are installed. You might talk to that in a slightly different way than you would one of these other Bluetooth LE dongles. I know Bluegiga uses it on the Windows side to capture a little more data than I think you'd normally get from one of these other dongles. The other dongles might present a more universal interface for interacting with Bluetooth LE on the Windows side.