Guys Got the Culprit here... :) :)
After doing lot of RnD on Server side and Client side, I took a look at "Developer Tool" in IE. You can find something like "Document Mode: Quirks". If you check the Wiki page for this Quirks mode, You can find a definition as "In computing, quirks mode refers to a technique used by some web browsers for the sake of maintaining backward compatibility with web pages designed for older browsers, instead of strictly complying with W3C and IETF standards in standards mode."
So this all becuase of checking for compatibility for all components on page. And hence the performance issue. :)
And When I changed this mode to IE 8 Standard. Things started working really well.
IE and its issues ...!!!! :D