Execute Javascript without using a browser

Question

In an application I'm working on, I am downloading a webpage of mine from the server and then parsing the code for information. I'm using an ASIHTTPRequest to get the code from the server. The problem is that the webpage uses Javascript to set some text on the page, and the ASIHTTPRequest pulls the code from the server, so the Javascript never gets executed, and therefore the webpage is missing information. Is there an easy way to execute the Javascript without a browser so as to let it update the code? Also, currently I'm developing the app for OSX 10.9 NOT iOS.

Answer 1

You cannot execute the javascript on a webpage without a browser. The browser is precisely the software that renders a webpage and lets code interact with it. This is not something ASIHTTPRequest will ever be able to do.

However, you can use PhantomJS which is basically a headless and scriptable browser. But packaging that within your app may be tricky.

Or possibly, to stay in Cocoa realm, you could load a WebView up with the page that you want, and never display it on screen. You can then execute javascript in that WebView to return data.

---

However, this seems like a bad idea. Downloading a page, all it's assets, parsing and rendering it, then executing javascript is a slow and memory intensive operation. It will be so much cleaner if you can make your server just give you the data you want without having to run any complex client side scripts.

It's hard to advise more without knowing what you are trying to do, but do yourself a favor and rethink your strategy here.

Answer 2

This problem also occurs with R programming. The R readLines() function will happily accept a URL and suck down the web page into an internal string vector. However, if that webpage is javascript, the script does not get run. Thus the text may be completely different from what would appear on the same web page in an actual browser.

PhantomJS may be a solution for this as well.