MySQL add total column

Question

I need to query this DB to get each row, but also the SUM of one of the column values of the results. I could use php to get the total value, but then I'd need to run two loops, one to get the total (which goes at the top above the results). So I'd prefer the query to catch it and just make a "total" row, but the only way I've gotten it to work is with a subquery that is essentially a repeat of the original query. Is there a better way?

SELECT 
CONCAT(u.firstname, ' ', u.lastname ) name, u.id, s.description, s.shiftstart, s.shiftend, 
    (SELECT 
    SUM( TIME_TO_SEC( TIMEDIFF( shiftend, shiftstart ) ) ) /3600
    FROM shifts
    WHERE id =  '$user'
    AND DATE( shiftstart )
    BETWEEN '$start'
    AND '$end') total
FROM shifts s
INNER JOIN users u ON ( s.id = u.id )
WHERE s.id = '$user'
AND DATE( shiftstart )
BETWEEN '$start'
AND '$end'
ORDER BY shiftstart

The above works and outputs:

name        id     description  shiftstart             shiftend               total
Joe User    joeuser    Stuff    2009-01-05 07:45:00    2009-01-05 12:15:00    39.5000
Joe User    joeuser    Stuff    2009-01-05 13:00:00    2009-01-05 17:00:00    39.5000
Joe User    joeuser    Stuff    2009-01-06 07:45:00    2009-01-06 10:45:00    39.5000
Joe User    joeuser    Stuff    2009-01-06 10:45:00    2009-01-06 12:45:00    39.5000
Joe User    joeuser    Stuff    2009-01-06 13:30:00    2009-01-06 14:30:00    39.5000
Joe User    joeuser    Stuff    2009-01-06 14:30:00    2009-01-06 17:00:00    39.5000
Joe User    joeuser    Stuff    2009-01-07 09:45:00    2009-01-07 14:00:00    39.5000
Joe User    joeuser    Stuff    2009-01-07 15:00:00    2009-01-07 17:00:00    39.5000
Joe User    joeuser    Stuff    2009-01-08 08:00:00    2009-01-08 12:15:00    39.5000
Joe User    joeuser    Stuff    2009-01-08 13:15:00    2009-01-08 17:00:00    39.5000
Joe User    joeuser    Stuff    2009-01-09 07:45:00    2009-01-09 10:45:00    39.5000
Joe User    joeuser    Stuff    2009-01-09 11:45:00    2009-01-09 15:15:00    39.5000
Joe User    joeuser    Stuff    2009-01-09 15:15:00    2009-01-09 17:00:00    39.5000

Which is what I need, but probably not the best way to get it.

Answer 1

MySQL supports a special group-by modifier called ROLLUP.

SELECT CONCAT(u.firstname, ' ', u.lastname ) name, u.id, 
  s.description, s.shiftstart, s.shiftend, 
  SUM( TIME_TO_SEC( TIMEDIFF( shiftend, shiftstart ) ) ) /3600 total
FROM shifts s INNER JOIN users u ON ( s.id = u.id )
WHERE s.id = ? AND DATE( shiftstart ) BETWEEN ? AND ?
GROUP BY u.id, s.shiftstart WITH ROLLUP
ORDER BY shiftstart;

Answer 2

The better way is to do this with code. People keep insisting on using SQL, which is a relational algebra, for doing procedural duties. Trying to shoehorn procedural-ness onto SQL is always a bad idea, in terms of both complexity and performance. Take this advice from a professional DBA.

Run two queries from your code. Output the larger set first then the total line in whatever format you desire. Your queries will be smaller and simpler, your performance will improve and you'll get the output you desire.

Some other advice - disk space is cheap and most database tables are read far more often than they're written. Set up an insert/update trigger (if possible in MySQL) to populate a separate column with calculated fields like "CONCAT(u.firstname,' ',u.lastname)" and use that for queries. Per-row functions are not scalable and will kill your DBMS performance as it gets bigger.

Answer 3

Anything can be better by comparison; the question is comparison to what, right?

The key issue you indicated is that you wanted the total at the top of the results; Crystal Reports, and other apps, manage this sort of magic by being a 2 pass engine. First pass gets the totals.

There are trade-offs with any solution. If you get a separate "total" row, then the receiving app will need to either strip it out of the results or play some other trick to hide it.

One possibility, which may be an option if you aren't writing for a 1 million hit/hr website, is to simply make 2 calls -- one for the overhead information, such as the name, TOTAL Time, etc, then for the details ... It appears from the query you are selecting a single person's results.

We're all for saving overhead and bandwidth but sometimes, simple is better ...

EDIT: Pax beat me to the "save" button ... lol ...

Answer 4

Use the mysql extension provided exactly for this purpose, as described in Bill Karwin's response (which I've upvoted myself).

If this weren't available, option 2 would be another SQL statement: SELECT SUM ...) SQL really is optimized to be extremely efficient for this sort of thing, compared to any procedural-code looping you're likely to write.