Node.js browserify slow: isn't there a way to cache big libraries?

Question

I'm creating a file that requires huge libraries such as jquery and three.js using browserify. The compiling process takes several seconds, probably because it's recompiling all the libs for each minor change I make. Is there a way to speed it up?

Answer 1

Have you tried using the --insert-globals, --ig, or --fast flags? (they're all the same thing)

The reason it's slow may be that it's scanning all of jquery and d3 for __dirname, __filename, process, and global references.

EDIT:

I just remembered: Browserify will take any pre-existing require functions and fall back to using that. more info here

This means you could build a bundle for your static libs, and then only rebuild the bundle for your app code on change.

This coupled with my pre-edit answer should make it a lot faster.

Answer 2

There are a few options that can help:

--noparse=FILE is a must for things like jQuery and three.js that are huge but don't use require at all.

--detect-globals Set to false if your module doesn't use any node.js globals. Directs browserify not to parse a file looking for process, global, __filename, and __dirname.

--insert-globals Set to true if your module does use node.js globals. This will define those globals without parsing the module and checking to see if they're used.

I was able to speed up my build by externalizing ThreeJS, using noparse with it, and setting it not to create a source map for it.

Use https://github.com/substack/watchify while developing.

Answer 3

If you use grunt, you can use my grunt task : https://github.com/amiorin/grunt-watchify

It caches the dependencies and watches the filesystem. Because of this the build is very fast. You can use it with grunt-contrib-watch and grunt-contrib-connect or alone. You can find a Gruntfile.js example in the github repository.

If you don't use grunt, you can use the original watchify from @substack : https://github.com/substack/watchify

Answer 4

I wrote this to solve the problem of slow builds with browserify and commonjs-everywhere. If you run it in "watch" mode then it will automatically watch your input files and incrementally rebuild just any files that changed. Basically instantaneous and will never get slower as your project grows.

https://github.com/krisnye/browser-build

Answer 5

Using watchify is practically a must, as it actually caches your deps between reloads.

My builds dropped from 3-8s to under 1s. (The >3s builds were still using ignoreGlobals, detectGlobals=false, and even noParseing jQuery).

Here's how I use it with gulp and coffeescript:

gutil = require("gulp-util")
source = require("vinyl-source-stream")
watchify = require("watchify")
browserify = require("browserify")
coffeeify = require("coffeeify")

gulp.task "watchify", ->
  args = watchify.args
  args.extensions = ['.coffee']
  bundler = watchify(browserify("./coffee/app.coffee", args), args)
  bundler.transform(coffeeify)

  rebundle = ->
    gutil.log gutil.colors.green 'rebundling...'
    bundler.bundle()
      # log errors if they happen
      .on "error", gutil.log.bind(gutil, "Browserify Error")
      # I'm not really sure what this line is all about?
      .pipe source("app.js")
      .pipe gulp.dest("js")
      .pipe livereload()
    gutil.log gutil.colors.green 'rebundled.'

  bundler.on "update", rebundle
  rebundle()

gulp.task "default", ["watchify", "serve"]

EDIT: here's a JS translation:

var gutil = require("gulp-util")
var source = require("vinyl-source-stream")
var watchify = require("watchify")
var browserify = require("browserify")
var coffeeify = require("coffeeify")

gulp.task("watchify", function() {
  var args = watchify.args
  args.extensions = ['.coffee']
  var bundler = watchify(browserify("./coffee/app.coffee", args), args)
  bundler.transform(coffeeify)

  function rebundle() {
    gutil.log(gutil.colors.green('rebundling...'))
    bundler.bundle()
      // log errors if they happen
      .on("error", gutil.log.bind(gutil, "Browserify Error"))
      // I'm not really sure what this line is all about?
      .pipe(source("app.js"))
      .pipe(gulp.dest("js"))
      .pipe(livereload())
    gutil.log(gutil.colors.green('rebundled.'))
  }

  bundler.on("update", rebundle)
  rebundle()
})

gulp.task("default", ["watchify", "serve"])

Answer 6

Update

You can also give it a try to persistify which can be used as a drop in replacement for watchify from the command line and from code.

Original answer below

=======

I'm currently using bundly: https://www.npmjs.com/package/bundly

FULL DISCLOUSURE: I wrote it

But the main difference of this wrapper is that it provides incremental building. It persists the browserify cache between runs and only parse the files that have changed without the need for the watch mode.

Currently the module does a bit more than only adding the cache, but I'm thinking that the logic that handles the incremental build part could be moved to a plugin, that way it can be used with browserify directly.

Check a demo here: https://github.com/royriojas/bundly-usage-demo