Efficiently use mysql Table to cache complex queries

Question

I used to have quite a big number of multiple Joins in queries.

To be able to use (at least) the build-in MySql Cache features I wrote the following function, it simply encodes the original query into base64, checks if it's there and not expired.

This has improved the performance dramatically, and I have the advantage to contol the cache-time query-by-query in the source code.

But on busy times the table becomes unavailable due to deletions or the selections simply take too long. Are there any suggestion on what to do to make this run faster and to avoid the problem previous mentioned?

table:

CREATE TABLE `cachesql` (
`id` int(9) NOT NULL AUTO_INCREMENT,
`expire` int(15) NOT NULL,
`sql` text NOT NULL,
`data` mediumtext NOT NULL,
PRIMARY KEY (`id`,`sql`(360)),
KEY `sdata` (`sql`(767)) USING HASH
) ENGINE=InnoDB

function:

    function fetchRows_cache($sql,$cachetime,$dba){
    // internal function (called by fetchRows)
    global $Site;
    $expire = 0;
    $this->connect($dba);

    // check if query is cached
    $this->q = mysql_query("SELECT `expire`,`data` from cachesql where `sql`='".base64_encode($sql)."' limit 1;", $this->con) OR $this->error(1, "query$".$sql."$".mysql_error());
    $this->r = mysql_fetch_assoc($this->q);
    $expire = $this->r['expire'];
    $data = $this->r['data'];

    if (($expire < time())||($cachetime =="0")) { // record expied or not there -> execute query and store
        $this->query("DELETE FROM `cachesql` WHERE `sql`='".base64_encode($sql)."'",$dba); // delete old cached entries

        $this->q = mysql_query($sql, $this->con) OR $this->error(1, "query$".$sql."$".mysql_error());
        $this->r=array();
        $this->rc=0;
        while($row = mysql_fetch_assoc($this->q)){
            $arr_row=array();
            $c=0;
            while ($c < mysql_num_fields($this->q)) {        
                $col = mysql_fetch_field($this->q, $c);    
                $arr_row[$col -> name] = $row[$col -> name];            
                $c++;
            }    
            $this->r[$this->rc] = $arr_row;
            $this->rc++;
        }
        $out = $this->r;

        // write results into cache table
        if ($cachetime != "0") {
            // not store cache values for now (too many locks)
            $this->query("INSERT INTO `cachesql` (`sql`,`data`,`expire`) VALUES ('".base64_encode($sql)."','".mysql_real_escape_string(serialize($out))."','".(time()+$cachetime)."')",$dba);       
        }
        return $out;

    }
    else { // use Cached data
        return unserialize($data); 
    }
}

Answer 1

thanks to @HeatfanJohn - he pointet out something really Simple and efficient.

since the original Queries are not used for anything (except the matching of cache entries), it is enough to simply store a checksum identify the query in question uniquely.

the new structure simply stores the MD5 hash of the original Query (16 Bytes), expireUnixTime , and the serialized row-set

new Structure:

CREATE TABLE `cachesql` (
`sql` varchar(32) NOT NULL,
`expire` int(11) NOT NULL,
`data` text NOT NULL,
PRIMARY KEY (`sql`),
UNIQUE KEY `sql` (`sql`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='cache for db->fetchRows'

The primary index on the sql column reacts extremely fast since its very short, and can be indexed for searches much better.

I have not get a different speed results in using BLOB or TEXT fields for the datasets.

Answer 2

i think the main slowing down point is, that you use InnoDB for your caching table.

i found out, that you should use InnoDB for everything except for read-heavy cache-tables ;)

MyISAM is especially good for read-intensive (select) tables.

Answer 3

Lets try the inmemory table to make it faster.