There is a lot of functionality going on behind all the different types of controls and making them work in Page Editor you will be sinking far too much time into this imo. Even some of the ORM projects like Glass.Mapper fall back to using Sitecore controls for Page Editor mode.
I'd think very carefully before going down this path. Not just from a technological point of view, but also from a support, maintenance and future upgrade point of view.
I noticed you tagged the question with Sitecore 6, but the latest version of Sitecore is 7 with 7.1 due this month with SPEAK UI support. You don't know what changes will be coming up in the future versions, at least with a more standard approach you know Sitecore has an upgrade path for your project which will be less painful.
And what happens if you leave and someone else needs to take over the project? Sitecore is a pretty niche developer market, you've just made your candidate base even more niche. You also have to think from a Sitecore Support perspective, if you ever need to engage them then you might run into another issue.
Then you also need to take into account DMS, personalization, A/B and multivariant testing, all from within the Page Editor all of which just opens up a whole other can of worms.
There's no reason you can't use Knockout.js alongside Sitecore for some parts of your project though.
That's just my 2 cents though.
BUT don't let that stop you exploring the idea though, it's what drives progress and has led to some awesome shared source modules :)