Better management of Javascript and html template files: Edit as many files, but use as one file

Question

Background

Have a project that includes over 100 Javascript and HTML template files. Using a script loader (yepnope) that loads the files directly from our development environment so that during the development process, a small change can be saved to a file and instantly be seen after refreshing the browser.

The Problem

This worked great at first, but now that the project has become larger, it takes too long for the browser to load all of the files. For our production environment, we can concatenate all of the files together, but the development environment also needs to be snappy in order to speed up the development process and reduce developer frustration.

The Question

Looking for some kind of solution that allows us to keep our current file organization, but probably deliver the files to the browser as one big file. I'm not sure how this would work. Would such a solution update some kind of derived file each time it detects a change in a source file, or keep everything in one large file, but allow the file to be navigated within the IDE as if it were a directory structure?

Currently we use PyCharm.

Thanks so much!

Answer 1

You should concatenate your javascript files for sure. Serving many files is really really bad thing. What I'm using now is Grunt. It's a nodejs task runner. We have two environments

• development/local - in local environment Grunt concatenate the files, but don't minify them. It also adds a informative comment above every file, so if you see some error in DevTools you need to scroll a bit to see the exact file
• production - the final file is minified and the comments are removed.

Using this approach we still have everything split in different files, but serve only one file. We are actually using Grunt not only for javascript, but for the html templates, CSS, generating cache manifest and the main index.html file. Here is how our package.json file looks like:

{
  "name": "project",
  "version": "0.0.1",
  "description": "project",
  "repository": {},
  "devDependencies": {
    "grunt": "0.4.1",
    "grunt-contrib-concat": "0.3.0",
    "grunt-contrib-watch": "0.4.4"
  }
}

And our Gruntfile.js:

module.exports = function(grunt) {

    grunt.initConfig({
        // contcatenation
        concat: {
            // javascript
            js: {
                src: ['src/**/*.js'],
                dest: 'bin/scripts.js',
                options: {
                    process: function(src, filepath) {
                        // do something here
                        return "\n/* " + filepath + " */\n" + src;
                    }
                }
            },
            // CSS
            css: {
                src: ['src/**/*.css'],
                dest: 'bin/styles.css',
                options: {
                    process: function(src, filepath) {
                        return src;
                    }
                }
            },
            // HTML templates
            html: {
                src: ['src/**/*.html', '!src/index.html'],
                dest: 'tmp/templates.html',
                options: {
                    process: function(src, filepath) {
                        return src;                     
                    }
                }
            }
        },
        // custom task for generating index.html (it's a single page app)
        'generate-index': {
            index: {
                src: '<%= concat.html.dest %>',
                dest: 'bin/index.html',
                template: 'src/index.html'
            }
        },
        // custom task for generating cache manifest file
        'generate-manifest': {
            manifest: {
                dest: 'bin/cache.manifest'
            }
        },
        // watching all the files and performa the specific tasks
        watch: {
            js: {
                files: ['<%= concat.js.src[0] %>'],
                tasks: ['concat:js', 'generate-manifest']
            },
            css: {
                files: ['<%= concat.css.src[0] %>'],
                tasks: ['concat:css', 'generate-manifest']
            },
            html: {
                files: ['<%= concat.html.src[0] %>'],
                tasks: ['concat:html', 'generate-index', 'generate-manifest']
            },
            img: {
                files: ['bin/img/**/*.png', 'bin/img/**/*.jpg', 'bin/img/**/*.gif'],
                tasks: ['generate-manifest']    
            }
        }
    });

    // loading modules
    grunt.loadNpmTasks('grunt-contrib-concat');
    grunt.loadNpmTasks('grunt-contrib-watch');
    grunt.loadTasks('tasks');

    // grunt.registerTask('default', ['concat', 'less']);
    grunt.registerTask('default', ['concat', 'generate-index', 'generate-manifest', 'watch']);

}

Answer 2

One thing is JavaScript Source Maps this allows you to deliver one big file, with the information how they are combined. If the browser supports source maps, it will display the minified and combine file as if it was not minified and combined. To achieve this it loads the original source files using the informations of the source map. From what i have seen the original files are only loaded when they are required (displayed in the scripted view).

Another thing you could do is using expire headers of e.g. one day and prefix your asserts with a build number. If the code is rebuild upon code changes a new build number is generated. That way only when changes are done the js code is reloaded otherwise it is read from browser cache. That way only the initial load after a change would be slower but then for testing the page would run smooth again.