Is fetching remote data server-side and processing it on server faster than passing data to client to handle?

Question

I am developing a web app which functions in a similar way to a search engine (except it's very specific and on a much smaller scale). When the user gives a query, I parse that query, and depending on what it is, proceed to carry out one of the following:

• Grab data from an XML file located on another domain (ie: from www.example.com/rss/) which is essentially an RSS feed
• Grab the HTML from an external web page, and proceed to parse it to locate text found in a certain div on that page

All the data is plain text, save for a couple of specific queries which will return images. This data will be displayed without requiring a page refresh/redirect.

I understand that there is the same domain policy which prevents me from using Javascript/Ajax to grab this data. An option is to use PHP to do this, but my main concern is the server load.

So my concerns are:

1. Are there any workarounds to obtain this data client-side instead of server-side?
2. If there are none, is the optimum solution in my case to: obtain the data via my server, pass it on to the client for parsing (with Javascript/Ajax) and then proceed to display it in the appropriate form?
3. If the above is my solution, all my server is doing with PHP is obtaining the data from the external domains. In the worst (best?) case scenario, let's say a thousand or so requests are being executed in a minute, is it efficient for my web server to be handling all those requests?

Once I have a clear idea of the flow of events it's much easier to begin.

Thanks.

Answer 1

I just finish a project to do the same request like your req.

My suggestion is:

1. use to files, [1] for frontend, make ajax call to sen back url; [2] receive ajax call, and get file content from url, then parse xml/html

   • in that way, it can avoid your php dead in some situation

2. for php, please look into [DomDocument] class, for parse xml/html, you also need [DOMXPath]

Please read: http://www.php.net/manual/en/class.domdocument.php

No matter what you do, I suggest you always archive the data in you local server.

So, the process become - search your local first, if not exist, then grab from remote also archive for - 24 hrs.

BTW, for your client-side parse idea, I suggest you do so. jQuery can handle both html and xml, for HTML you just need to filter all the js code before parse it.

So the idea become :

1. ajax call local service

2. local php grab xm/html (but no parsing)

3. archive to local

4. send filter html/xml to frontend, let jQuery to parse it.

Answer 2

HTML is similar to XML. I would suggest grabbing the page as HTML and traversing through it with an XML reader as XML.