mysql many-to-one and group by giving unusual results

Question

im having difficulty with the following fairly simple setup:

CREATE TABLE IF NOT EXISTS invoices (
id int(11) NOT NULL auto_increment,
PRIMARY KEY (id)
);

CREATE TABLE IF NOT EXISTS invoices_items (
id int(11) NOT NULL auto_increment,
invoice_id int(11) NOT NULL,
description text NOT NULL,
amount decimal(10,2) NOT NULL default '0.00',
PRIMARY KEY (id)
);

CREATE TABLE IF NOT EXISTS invoices_payments (
id int(11) NOT NULL auto_increment,
invoice_id int(11) NOT NULL,
amount decimal(10,2) NOT NULL default '0.00',
PRIMARY KEY (id)
);

some data:

INSERT INTO invoices (id) VALUES (1);

INSERT INTO invoices_items (id, invoice_id, description, amount) VALUES
(1, 1, 'Item 1', '750.00'),
(2, 1, 'Item 2', '750.00'),
(3, 1, 'Item 3', '50.00'),
(4, 1, 'Item 4', '150.00');

INSERT INTO invoices_payments (id, invoice_id, amount) VALUES
(1, 1, '50.00'),
(2, 1, '1650.00');

and the sql yielding unusual results:

select  invoices.id,
        ifnull(sum(invoices_payments.amount),0) as payments_total, 
        ifnull(count(invoices_items.id),0) as item_count 
from    invoices
            left join invoices_items on invoices_items.invoice_id=invoices.id 
            left join invoices_payments on invoices_payments.invoice_id=invoices.id 
group by invoices.id

results in the (erroneous) output

id  payments_total  item_count
1   6800.00     8

now, as evidenced by there being infact only four 'invoice_item' rows, i dont understand why mysql is not grouping properly.

EDIT

i know i can do something like this:

select x.*, ifnull(sum(invoices_payments.amount),0) as payments_total from (
    select  invoices.id,
            ifnull(count(invoices_items.id),0) as item_count 
    from    invoices
                left join invoices_items on invoices_items.invoice_id=invoices.id 
    group by invoices.id
) as x left join invoices_payments on invoices_payments.invoice_id=x.id
group by x.id

but i want to know if im doing something wrong in the first query - i cant immediately see why the first query is giving incorrect results! :(

Answer 1

Your join logic is incorrect. In your join, you specify invoices_items.invoice_id = invoices.id. You also specify invoices_payments.invoice_id = invoices.id. Because of transitivity, you end up with:

invoices_items.invoice_id    = invoices.id
invoices_payments.invoice_id = invoices.id
invoice_items.invoice_id     = invoices_payments.invoice_id

The sum of the 2 invoice payments is $1700. For every invoice payment, there are 4 invoice_items that satisfy the above relations. $1700 * 4 = $6800.

For every invoice item, there will be two invoice payments that satisfy the above relations. 4 invoice items * 2 = 8 count.

Answer 2

There are two tables with a many:one relationship with invoices. Your count is the cartesian product.

The payments should be applied to the invoice, not the invoice items. Get the invoice total first, then join the payments to it.

This may be similar to what you are looking for:

SELECT
    invoice_total.invoice_id,
    invoice_total.amount as invoice_amount,
    payments_total.amount as total_paid
FROM
    (
        SELECT
            invoice_id,
            SUM(amount) as amount
        FROM
            invoices_items
        GROUP BY
                invoice_id
        ) invoice_total
    INNER JOIN
            (
            SELECT 
                invoice_id,
                SUM(amount) as amount
            FROM
                invoices_payments
            GROUP BY
                invoice_id
            ) payments_total
    ON invoice_total.invoice_id  = payments_total.invoice_id;

Answer 3

edit:

ah, sorry - see your point now. The reason you're getting unexpected results is that this query:

SELECT * 
FROM invoices
    LEFT JOIN invoices_items ON invoices_items.invoice_id = invoices.id
    LEFT JOIN invoices_payments ON invoices_payments.invoice_id = invoices.id;

results in this:

id  id invoice_id   description amount  id  invoice_id  amount
1    1  1            Item 1      750.00  1   1           50.00
1    1  1            Item 1      750.00  2   1           1650.00
1    2  1            Item 2      750.00  1   1           50.00
1    2  1            Item 2      750.00  2   1           1650.00
1    3  1            Item 3      50.00   1   1           50.00
1    3  1            Item 3      50.00   2   1           1650.00
1    4  1            Item 4      150.00  1   1           50.00
1    4  1            Item 4      150.00  2   1           1650.00

As you can see you get every invoices_items record once each for every invoices_payments record. You're going to have to grab (i.e. group) them separately.

Note that the GROUP BY clause in your initial query is redundant.

Here's what you need:

SELECT
    invoices.id,
    payments_total.payments_total, 
    IFNULL(COUNT(invoices_items.id),0) AS item_count 
FROM invoices
    LEFT JOIN invoices_items ON invoices.id = invoices_items.invoice_id
    LEFT JOIN (
        SELECT invoice_id,
            IFNULL(SUM(invoices_payments.amount),0) AS payments_total
            FROM invoices_payments
            GROUP BY invoice_id
        ) AS payments_total ON invoices.id  = payments_total.invoice_id
;