How to increase speed of application with lots of RSS

Question

Good Morning all...

I have web application with lots of RSS feeds. The flow is below

• Fetch the RSS from respective site(using PHP)
• Store it in DB (MySQL)
• Read it and display it to users browser.

Fetching is done at every one hour interval and store to DB.

Then whoever request, display these feeds to users browser. (Read from DB and display it.)

The reading process is not that fast. In other words if user have 20 feeds on same page then loading 5 articles for each feed. It is not that fast and currently not giving good User experience.

I am running on 8GB RAM VPS server, Technology - PHP, MYSQL, MOOTOOLS, javascripts

Then to make it fast i tried using Flat File - Read feeds from respective sites and write it to Feed File. (separate file for each feed.)

Then read the feed file and display it on user browsers. In this scenario it was slower than reading from DB.

So now i have no option.. and no clue what i can do to increase the speed of my website.

If any expert have any suggestion please let me know.

Regards, Mona

Answer 1

I would not consider myself an expert, but I could at least give you somethings to try: First it's a good thing you're showing the rss feeds from your own database, this should protect you if any of the rss-sources fails due to problems at the rss-provider. Nonetheless, I suggest you to move the loading part of the rss feeds to a separate file, which runs server-sided (and make it into a 'cronjob'). This makes sure that a user can never be bothered with the rebuilding of your data-source. This cronjob can then be called each hour to refresh your database.

The next step would be to find out where the process slows down the most, are there slow queries? Or is there just some sluggish code in your script?

To narrow down the causes, I really suggest you to install the XDebug extension (there are ready dlls for Windows here: http://pecl.php.net/package/Xdebug) and add the following lines to your php.ini: [XDebug] zend_extension = "C:\xampp\php\ext\php_xdebug.dll" xdebug.profiler_append = 0 xdebug.profiler_enable = 0 xdebug.profiler_enable_trigger = 1 xdebug.profiler_output_dir = "C:\xdebug" xdebug.profiler_output_name = "cachegrind.out.%t-%s" xdebug.remote_enable = 0 xdebug.remote_handler = "dbgp" xdebug.remote_host = "127.0.0.1" xdebug.trace_output_dir = "C:\xdebug"

After installing, adding ?XDEBUG_PROFILE to your url (see: http://www.xdebug.org/docs/profiler) will generate a file which you can examine with WinCacheGrind (http://sourceforge.net/projects/wincachegrind/). This program you can narrow down the execution time per function call.

I hope this helps you out :)

PS: Make sure to disable, or even better, not install XDebug on your production environment, since XDebug slows down your scripts...