Best headless web engine

Question

I'm working on a project where I need to talk to a headless web engine as a service - meaning I need the web engine to run as a daemon, and talk to it over some interface.

I've been using PhantomJS + GhostDriver. It works, but performs very slowly. It also fails in many occasions due to weird bugs. (For example, I can't navigate to amazon.com over ghostdriver because for some reason it doesn't recognize when pageload is finished, so the call never returns).

Is there a more robust solution for what I'm trying to do?

Answer 1

It all depends on what features you require. If you need a fast browser that doesn't support JavaScript (or doesn't support it well) and supports Selenium WebDriver then you can use HtmlUnit (or many others, but in my opinion HtmlUnit has the best WebDriver support).

However, if strong JavaScript and "normal" browser emulation is required, PhantomJS + GhostDriver is a great choice since it uses the JavaScriptCore engine with WebKit as a backend.

For my purposes I needed to use Chrome (and chromedriver) before GhostDriver was available. The latter, while not always perfect, has replaced Chrome for me in most cases solely because it's faster (largely since does not have to render pages) and acts very similarly to Chrome (due to WebKit - especially the JS engine portion).