The problem is not the DOCTYPE; it may be that Firefox 10 (which is pretty old now BTW) is interpreting the code differently depending on the DOCTYPE, but the real problem is your Javascript.
Your code onclick="doFunction(outputarea)"
is not really correct; some browsers will guess that you want the element with the ID outputarea
but that's not standard behavior. Your should do something like this instead:
<script type="text/javascript">
function doFunction(towindowId)
{
var towindow = document.getElementById(towindowId);
towindow.value='2';
}
</script>
...
<button type="button" onclick="doFunction('outputarea')">doFunction </button>
Also, you should always check for Javascript errors in Firefox's error console - I tested your original code in Firefox 10 and got the error "Error: toWindowId is not defined".
FYI, XHTML (strict mode or otherwise) is rarely needed; you might want to consider just the regular HTML5 doctype, <!DOCTYPE html>
, unless you have a specific reason for needing XHTML. See this article: http://www.webdevout.net/articles/beware-of-xhtml. Also note that the W3Schools info is often not up-to-date with current trends and practices.