Meteor: The application is not spiderable

Question

My application is not spiderable both on local and production.

When I go to http://localhost:3000/?_escaped_fragment_=, I can see the following error appears (phantom is killed after 15 seconds):

spiderable: phantomjs failed: { [Error: Command failed: ] killed: true, code: null, signal: 'SIGTERM' }

It seems that many other people got this problem:

• https://github.com/gadicc/meteor-phantomjs/issues/1
• https://groups.google.com/forum/#!msg/meteor-talk/Lnm9HFs4MgM/YKDMR80fVecJ
• https://groups.google.com/forum/#!topic/meteor-talk/7ZbidddRGo4

The thing is I am not using observatory or select2 and all my publications return a cursor. According to me, the problem comes from the minification. I just read in this thread that someone succeed to display "SyntaxError: Parse error". How can I know more about what is going wrong with Phantom and which file is causing the problem?

Answer 1

My problem was coming from SSL. You can have a complete overview of what I did here.

Edit the spiderable source and add --ignore-ssl-errors=yes to the phantomjs command line, it will work.

Answer 2

This happens when spiderable is waiting for subscriptions that fail to return any data and end up timing out, as mentioned in some of the threads you linked.

Make sure that all of your publish functions are either returning a cursor, a (possibly empty) list of cursors, or sending this.ready().

Meteor APM may be useful in determining which publications aren't returning.

Answer 3

If you want to know more about what is wrong with phatomjs, you might try this code (1):

// Put your URL below, no "?_escaped_fragment_=" necessary
var url = "http://your-url.com/";
var page = require('webpage').create();
page.open(url);
setInterval(function() {
  var ready = page.evaluate(function () {
    if (typeof Meteor !== 'undefined'
        && typeof(Meteor.status) !== 'undefined'
        && Meteor.status().connected) {
      Deps.flush();
      return DDP._allSubscriptionsReady();
    }
    return false;
  });
  if (ready) {
    var out = page.content;
    out = out.replace(/<script[^>]+>(.|\n|\r)*?<\/script\s*>/ig, '');
    out = out.replace('<meta name=\"fragment\" content=\"!\">', '');
    console.log(out);
    phantom.exit();
  }
}, 100);

For use in local, install phantomjs. Then outside your app, create a file phantomtest.js with the code above. And run phantomjs phantomtest.js

Another thing that maybe you can try is to use UglifyJS to catch some errors in the minified JS file as Payner35 did.