Why "find . -print0 | xargs -r0 du -chx" and "du -chx" different results?

Question

When i try get total size of placed files with find . -print0 | xargs -r0 du -chx it's return: 61G total. When i use standalone du -chx it give me: 2.8T total

% df -h give same result: Filesystem Size Used Avail Use% Mounted on /dev/md7 3.6T 2.8T 599G 83% /opt

Where is mistake?

Answer 1

61G is the total of the last item.

If you scroll back or use grep you'll see more totals.

find . -print0 | xargs -r0 du -chx|grep total

You are requesting 'du -chx' for every subsequent dir and subdir instead of all together.

Answer 2

The 61G total you see from the 'find' command is just the total from the last "batch" of arguments that xargs saw. (xargs can only accept a limited number of arguments, so the "total" you're seeing is not a "grand total".)

Try this:

find pjproject-2.1.0/  -print0 | xargs -r0 du -chx | grep "total"

Assuming there's no files called "total", you'll see a list of totals which will more or less add up to the same as the output of the du command.

Answer 3

One mistake is that the find|du is going to add up duplicate results, eg with a directory structure foo/bar/baz it will effectively do 'du -chx foo foo/bar foo/bar/baz'. That's counting the foo/bar/baz dir 3 times. However, this would miscount on the high side, so doesn't explain what you see.

A second issue is the number of arguments you're passing to du. In the find case, you'll have thousands of arguments, and xargs will process them in chunks. You're actually not seeing the total of all the files, but the subtotal for a chunk of them. There would have been several other subtotals printed along the way. Read the man page for xargs, particularly -n (max args) and -s (max length of command line)

The normal thing to do here is not to rely on sums for a single invocation of 'du' but do the sums in another command in your pipeline. However, once you try this, you'll hit the first problem I mentioned - you're counting directories multiple times.