Computing average values over sections of date/time

Question

Problem:

I have a database of sensor readings with a timestamp for the time the sensor was read. Basically it looks like this:

Sensor | Timestamp | Value

Now I want to make a graph out of this data and I want to make several different graphs. Say I want one for the last day, one for the last week and one for the last month. The resolution of each graph will be different so for the day-graph the resolution would be 1 minute. For the week graph it would be one hour and for the month graph it would be one day, or quarter of a day.

So I would like an output that is the average of each resolution (eg. Day = Average over the minute, Week = Average over the hour and so on)

Ex:

Sensor | Start | End | Average

How do I do this easily and quickly in mySQL? I suspect it invoves creating a temporary table or sorts and joining the sensor data with that to get the average values of the sensor? But my knowledge of mySQL is limited at best.

Is there a really clever way to do this?

Answer 1

SELECT  DAY(Timestamp), HOUR(Timestamp), MINUTE(Timestamp), AVG(value)
FROM    mytable
GROUP BY
        DAY(Timestamp), HOUR(Timestamp), MINUTE(Timestamp) WITH ROLLUP

WITH ROLLUP clause here produces extra rows with averages for each HOUR and DAY, like this:

SELECT  DAY(ts), HOUR(ts), MINUTE(ts), COUNT(*)
FROM    (
        SELECT  CAST('2009-06-02 20:00:00' AS DATETIME) AS ts
        UNION ALL
        SELECT  CAST('2009-06-02 20:30:00' AS DATETIME) AS ts
        UNION ALL
        SELECT  CAST('2009-06-02 21:30:00' AS DATETIME) AS ts
        UNION ALL
        SELECT  CAST('2009-06-03 21:30:00' AS DATETIME) AS ts
        ) q
GROUP BY
        DAY(ts), HOUR(ts), MINUTE(ts) WITH ROLLUP

2, 20, 0, 1
2, 20, 30, 1
2, 20, NULL, 2
2, 21, 30, 1
2, 21, NULL, 1
2, NULL, NULL, 3
3, 21, 30, 1
3, 21, NULL, 1
3, NULL, NULL, 1
NULL, NULL, NULL, 4

2, 20, NULL, 2 here means that COUNT(*) is 2 for DAY = 2, HOUR = 20 and all minutes.

Answer 2

Not quite the result table you wanted, but here's a starter for doing a 1 minute resolution:

SELECT sensor,minute(timestamp),avg(value) 
FROM table 
WHERE <time period specifier limits to a single hour>
GROUP BY sensor, minute(timestamp)

Answer 3

I've used code very similar to this (untested, but it's taking from working code)

set the variables:

$seconds = 3600;
$start = mktime(...);  // say 2 hrs ago
$end   = .... // 1 hour after $start

then run the query

SELECT MAX(`when`) AS top_When, MIN(`when`) AS low_When,
   ROUND(AVG(sensor)) AS Avg_S,
   (MAX(`when`) - MIN(`when`)) AS dur, /* the duration in seconds of the actual period */
   ((floor(UNIX_TIMESTAMP(`when`) / $seconds)) * $seconds) as Epoch
   FROM `sensor_stats`
   WHERE `when` >= '$start' AND `when` <= '$end' and duration=30
   GROUP BY Epoch/*((floor(UNIX_TIMESTAMP(`when`) / $seconds)) * $seconds)*/

The advantage of this is that you can have whatever time periods you want - and not even required to have them on 'round numbers', like a complete clock-hour (even a clock-minute, 0-59).