mysql memory load from javascript live search

Question

I am running a live search through a text box that when a user types it returns matching rows from a mysql database. The problem is this is one of the main features of the site and has really increased the memory load on my mysql database. Because every key stroke sends a request to the php script to query the database.

I have php ignore any search term less than 3 characters long, but besides that what else could I do?

There are some options here: Live search optimisation in Javascript

But I was wondering if I should pull from a cached xml sheet, or is there somehow some way to cache mysql itself.

What does google, or some of the other large sites that rely on this feature heavily do?

Answer 1

I would try to optimize the SQL query as much as possible:

• no SELECT *
• no JOIN
• use the WHERE only in indexed fields

Also, in the PHP side:

• cache the search results for the most frequently searched terms (the more frequently the searched data get updated, the shorter the cache lifetime) in plain text files
• use redis, memcached if possible

Also, consider a parallel NoSQL db

And

Zend_Search_Lucene is great for low/medium traffic sites (reportedly has issues when scaling)

Answer 2

Use indexing engines to index your data and speed up your search results. Like: http://sphinxsearch.com/ or http://lucene.apache.org/core/

Setup cron job to index data, there is PHP API for sphinx, and Zend Framework Module. Indexing speed uo things a lot, if used correctly.