Query is cached on local setup, but never on server?

Question

I am having a lot of troubles with the query cache on a project: I am running a Percona flavor of MySQL, same versions both on my local development machine as on the production server. Now, enabling the query cache gives me excellent results on my local machine: almost all queries that should be cached, effectively are.

Now, exactly the same queries are not being cached on the production server. Everything is exactly the same; the mysql variables, the database contents, the codebase, the logged in user, .. but on production only a handful queries are being cached, the most important ones are all being skipped. And I can't figure out why :-)

So, looking for a solution, I am working with the following query, used to select the latest 3 topics from the topics table: (this is the most "heavy" query and is the one I definitely want to be cached!)

SELECT `topic`.* FROM `topics` AS `topic` 
LEFT OUTER  JOIN `topics` AS `topic_helper` 
 ON (`topic`.`id` = `topic_helper`.`id` 
      AND `topic_helper`.`created_on` < `topic`.`created_on`) 
GROUP BY `topic`.`id` HAVING COUNT(*) < 3 
ORDER BY `topic`.`created_on` DESC;

So, to start, the SHOW VARIABLES LIKE '%query_cache% give me the same results, both local as on production:

+------------------------------+----------+
| Variable_name                | Value    |
+------------------------------+----------+
| have_query_cache             | YES      |
| query_cache_limit            | 1048576  |
| query_cache_min_res_unit     | 4096     |
| query_cache_size             | 10485760 |
| query_cache_strip_comments   | OFF      |
| query_cache_type             | ON       |
| query_cache_wlock_invalidate | OFF      |
+------------------------------+----------+

Running the above query gets cached locally after the first run, as SHOW PROFILE clearly tells me near the end of it's trace:

| Waiting for query cache lock   | 0.000001 |
| Waiting on query cache mutex   | 0.000001 |
| freeing items                  | 0.000000 |
| storing result in query cache  | 0.000002 |
| logging slow query             | 0.000001 |
| cleaning up                    | 0.000006 |
+--------------------------------+----------+

Second call returns the query from the cache, as expected.

On the production server, running this query will never store it in the cache. The result set is exactly the same, and there are clearly no statements being used that would invalidate query caching (according to the manual at http://dev.mysql.com/doc/refman/5.1/en/query-cache-operation.html - I am certain the above query does comply with the requirements for it to be cached.)

For completeness sake, the full output of the SHOW PROFILE for that same query on the production server, is pasted here: http://pastebin.com/7Jm5rmVd

Also, it is worth noting that although configuration is exactly the same on both servers, my local version is 5.5.27, slightly newer than the one on production 5.5.17-55. Could it be that this is the problem .. ?

I compared the full SHOW VARIABLES; output from both my local server as production server to see if anything was missing, but nothing differs except for the system time zone and the path's to log files etc ..

So, might any of you know where to look for next? Or have any clue what could be causing this?

Answer 1

We use Percona server a lot here, and community MySQL too.

The query caches are powerful - and mightily complicated. The worse thing MySQL could do is return some stale cache data.

Not only does MySQL cache the queries, but also the database data too - and uses indexes for additional performance.

Anything which might invalidate the query cache, invalidates it.

As a rule of thumb - we don't focus too closely on whether it's being cached or not ... we trust that MySQL acts intelligently - if for any reason it thinks something shouldn't be cached, it doesn't cache it. What we do do though - is make sure our queries are as efficient and simple as possible.

If I may say this - I think you're going to hit serious scalability problems irrespective of the query cache if your example query is "one of your most used". It's going to run like a dog without legs once that server gets busy!

According to your pastebin entry - you've at least one temporary table being created, probably due to the outer join (or the GROUP BYs).

I'm all for normalisation - but sometimes performance demands an alternative route.

Can you not cache some of that data yourself, in a some sort of lookup/summary table? Triggers can be your friend here :)