Very large single page application design problems

Question 1

This is actually a very complicated question as it really gets down to the design of your architecture.

For large-scale single-page applications, it's best to use some sort of front-end MV* style framework such as backbone.js, which ties in to jQuery quite usefully. You should also think about using some sort of dependency management framework such as require.js in order to load your scripts and dependencies asynchronously, and even better -- use the AMD pattern in your application design to make your architecture modular and easier to manage.

As far as how this relates to your MVC4 project, you have some options:

Do you want to use MVC as a "service layer" of sorts, that simply returns JSON objects, allowing your front-end to do the markup/template creation (think handlebars.js), or
Do you want your MVC project to return partial views (HTML) as a response, where you leverage the Razor templating system and simply use the front end to display what comes back from the server?

Either way, you will have to devise a way to handle front-end events and navigation (backbone provides both of these things when coupled with jQuery). Even more complicated is the pattern you choose to notify one view of another view's activities (there are many ways to do this) -- a publish/subscribe pattern for example.

I hope I have helped a bit, I know I'm not fully answering the question, but the answer could get really long!

Question 2

Lots of things are wrong with your approach. What I'd recommend is to watch some presentations on how people build Single Page Applications and what tooling is mostly used.

This seems like something reasonable: http://singlepageappbook.com/

You will at least want

some kind of modules system (I recommend AMD – http://requirejs.org)
an MV* framework (Backbone, Ember.js etc.)
DOM/AJAX Framework (jQuery, Mootools etc.). Some frameworks offer this and all of the above (Dojo, YUI, Sencha)
build solution (to have different environment in development / production)

Couple of good links:

Question 3

If you don't need a complicated truly SOFEA Single Page App then I recommend you go the PJAX route.

Then you just write your app as a normal web 1.0 app with the performance benefits of a single page load. I urge you to consider this an option as it allows you to do most of your validation work server side.

The idea is very simple on every response your sending the whole page back minus the header and footer (which contains the javascript and css includes). DOM rendering time is incredible fast these days... whats not is a full page reload, So don't worry about the size of the HTML your returning back.

Also the "PJAX way" is much easier to cache, Google SEO friendly and is in fact what the new Basecamp does.

Question 4

Note: Wanted this to be a comment not an answer but don't have enough exp to post comments ;(
[any corrections by community members welcome!]

Important points to consider for single page apps:

Lazy loading is vital as you don't want hundreds of js files loaded straight away as user first loads the page (very slow load time).
Good File organisation,- helps making changes easier, reduces complexity a bit and promotes reusable components. Makes testing of components easier.
Testing,- Since single page apps have a lot of javascript going on under the hood you would need a test framework for automatically testing components. This testing is on top of the tests you would use to confirm if certain user controls are rendered etc as you wouldn't want a viewless component making an ajax call to server when it shouldn't etc.

+1 for gryzzly's point about using a framework.

Sencha have a nice MVC like framework for their ExtJs product. They have data stores, ajax, lazy loading, class hierarchies and a lot more all bundled into the package. They have a good api page also to lookup object properties and methods (handy since there doesn't seem to be any intellisense for javascript :/ ). Their api page is; as far as I know, an example of a single page app. I know much of the stuff ExtJs does you can find an open source alternative but I like that it's just the one library and I don't have to download a couple of different frameworks to do various operations. [note: I have no affiliation to Sencha except for being a customer of theirs and like their stuff.]

Conclusion:

I'd say it would be quite difficult to manage a large single page app without using some client side framework; whether open source or not, and without using some architectural pattern like a client side MVC.

Single page apps I think are more complicated so your team would need to be quite handy at understanding the concept of a single page app and how to implement it. If you pull it off the site will be amazing in terms of user experience.

Question 5

I would recommend to use Sammy.js and split various viewmodels in knockout to a separate url. And if you're using asp.net mvc 4, use partial views (user controls) so that you put all code in one file. And name all and split all js-code in a meaningful way, filename and namespace in js. That will help a lot in maintainability and your own sanity in the long run. And use common sense!

Question 6

The way I did it was to include javascript code related to the template along with the template itself. And then load the whole template+script thing using ajax. If you want to try this be warned that most browsers don't execute <script> tags injected into the page. Especially if done using innerHTML. As such you should eval the script tags yourself (alternatively you can use document.createElement to inject the script but this does not offer any additional advantages compared to eval since the browser will blindly execute the script anyway).

In my case, to make it easier to grab the html and script portions of the template I store my templates in XML files instead of plain old HTML. That way I can simply use .responseXML of the ajax request to parse the template. My template has the following basic structure:

<template options...>
    <title>Optional</title>
    <html><![CDATA[

        Template body

    ]]></html>
    <init><![CDATA[

        // code that only needs to execute once

    ]]></init>
    <script><![CDATA[

        // code that needs to run every time
        // the template loads

    ]]></script>
</template>

You also need to remember to configure your server to reply with the correct content type for you templates. Otherwise resultXML won't work. This is not the only way to implement this system of course. You could simply save your templates as HTML and then parse that HTML to extract scripts to execute.

The main bulk of your code, the functions, constructors, objects etc. can be included in a js file. The template script only need to call those functions to tell the rest of the page how to work with the template.

If you further separate your data from your template and only populate the template with data on the page or make a separate ajax request for JSON data then you can configure your server to make the browser cache your templates. This is especially useful for often used templates (such as templates for dialog boxes). This allows the browser to download the template only once and use the cached version the next time you call the template.

Anyway, that's how I did it last time. It scaled well enough to serve facebook users (the web app was a facebook app). Just sharing my experience. Hope it helps.

Question 7

I've written a few huge single page application using Dojo Toolkit. I'm pretty sure whatever JavaScript framework you choose will probably work for you. I use Dojo because it provides me with features that makes huge single page application development easier to manage.

You can use Dojo's widget system to help you define all your screens and forms as widgets and then when you need them you can just instantiate and insert them wherever you need it. When ever you want to get rid of it, you can simply call destroy or destroyRecursive on that particular widget and its gone. Dojo's widget system also help you separate your HTML from your JavaScript, but still keep it together so that they are not located all over the place.

I've include a simple widget definition for a Login form.

This is the HTML template.

/* mine/forms/Login.html */
<div>
    ${form_name}
    <label>Username</label>
    <input data-dojo-type="dijit.form.TextBox"
           data-dojo-attach-point="_usrfld" />
    <br />
    <label>Password</label>
    <input data-dojo-type="dijit.form.TextBox"
           data-dojo-props="type: 'password'"
           data-dojo-attach-point="_pwdfld" />
    <br />
    <input data-dojo-type="dijit.form.Button"
           data-dojo-props="label: 'Login'"
           data-dojo-attach-event="onClick:_handleLogin" />
</div>

This is the JavaScript portion for the widget.

/* mine/forms/Login.js */
define([
    "dojo/_base/declare",
    "dijit/_Widget",
    "dijit/_Templated",
    "dijit/_WidgetsInTemplateMixin",
    "dijit/form/Button",
    "dijit/form/TextBox",
    "dojo/text!./Login.html"
], function(
    declare,
    _Widget,
    _Templated,
    _WidgetsInTemplateMixin,
    Button,
    TextBox,
    template
) {

return declare("mine.forms.Login", [_Widget, _Templated, _WidgetsInTemplateMixin], {
    // assign the template
    templateString: template,

    // signal that we will have widgets within our template and the parser should 
    // locate them and instantiate them
    widgetsInTemplate: true,

    form_name: "My Login Form",

    // place holders that will be referencing the corresponding widgets
    // that I have placed a data-dojo-attach-point on
    _usrfld: null,
    _pwdfld: null,
    _lgbtn: null,

    // a call back function that will be trigger when the Login button is clicked
    _handleLogin: function() {
        var usr = this._usrfld.get('value');
        var pwd = this._pwdfld.get('value');

        // now you have the username & password
        // you can use it to login
    }
});

});

There are a few benefits that the widget system provides:

The HTML template will be loaded for you when needed
You can have widgets within widgets. Dojo takes care of instantiating and even destroying those widgets for you
You can use dojo's simple template language to help with inserting strings. The example above uses ${form_name}. If you want to use a more sophisticate facility dojo also supports a Django similar syntax templating language. This will allow you to use most of the tags available with Django like for, if-then-else, etc.
data-dojo-attach-point is very helpful. If you use this, you will never need to assign id to a DOM element anymore. You will not need to locate the element in your HTML DOM tree. The variable you name in data-dojo-attach-point will be auto assign to reference the widget or DOM element you have defined in your template. The example above uses the _usrfld, _pwdfld, and _lgbtn.
data-dojo-attach-event is also very helpful. If you use this, you will not have to manully add an event hook to the button, Dojo's widget system will hook it for you and will also clear the hook when your widget is destroyed.
If you use Dojo's build system, which is a system that takes all your JavaScript codes and compresses them, Dojo will replace your template with the actual HTML so that when you load your widget in production mode, Dojo doesn't have to make another AJAX request to grab your template.

These are just some of the features I use daily when I develop my projects. Hopefully it will provide you with some insight that you can use to make your decision when choosing the proper JavaScript framework for your project. As a side note, I'm not advocating Dojo or anything, just wanted to share what works for me.

Question 8

To handle large size single page application it's recommended to break the individual module into different apps.
example : apps like search page app and result page app.