mysql query takes too long

Question

This query takes too long to produce result. And in php page it create 500 Internal server error.

SELECT DISTINCT tower_mac_address AS MAC_Address, tower_survey_no AS Survey_No 
FROM tower 
WHERE 
SUBSTR(REPLACE(tower_mac_address,':',''),7,6) 
NOT IN 
    (SELECT DISTINCT deviceid FROM device_data 
    WHERE SUBSTRING(date_time,1,10)=CURRENT_DATE)

Please Help

Answer 1

Your query is really bad. Since you are using SUBSTR(REPLACE ..... it cannot use index on tower_mac_address column. Similar situation for that inner query, you have SUBSTR... again. Instead of doing that, you can have prepared new column which has that SUBSTR(REPLACE... already, and add index on that column.

Btw, when you calculate values that you should use for that inner WHERE, instead of doing SUBSTRING to get date part of datetime, use DATE() function.

Answer 2

You can definitely get rid of substring in subquery:

SELECT DISTINCT deviceid FROM device_data 
WHERE SUBSTRING(date_time,1,10)=CURRENT_DATE;

--is the same as 
SELECT DISTINCT deviceid FROM device_data
WHERE `date_time` >=CURRENT_DATE AND `date_time`<CURRENT_DATE+INTERVAL 1 DAY;

But second query can use index on date_time column.

Answer 3

Try this

SELECT DISTINCT tower_mac_address AS  MAC_Address, tower_survey_no AS Survey_No 
FROM tower LEFT JOIN
(SELECT DISTINCT deviceid FROM device_data WHERE SUBSTR(date_time,1,10)=CURRENT_DATE) d ON SUBSTR(REPLACE(tower_mac_address,':',''),7,6) = d.deviceid
WHERE d.deviceid IS NOT NULL

   

Answer 4

Your problem is the NOT IN. MySQL does a very poor job of optimizing this. At least the documentation sort of explains this.

SELECT DISTINCT tower_mac_address AS MAC_Address, tower_survey_no AS Survey_No 
FROM tower t
WHERE  not exists
       (select 1
        from device_data dd
        where dd.deviceid = SUBSTR(REPLACE(t.tower_mac_address,':',''),7,6) and
              SUBSTRING(date_time,1,10)=CURRENT_DATE
       )

Here are additional recommendations:

1. Build an index on device_data.deviceid to speed up the subquery.
2. Store date_time as a datetime data type, not as a string. This way, the comparison to CURRENT_DATE does not include an implicit conversion.
3. If "tow_mac_address" contains encoded information, consider splitting that out into separate fields.