Is it possible to get the compressed and uncompressed sizes of a file on a btrfs file system?

Question 1

there is a third party tool that can do this.

usage:

ayush@devbox:/code/compsize$ sudo compsize /opt
Processed 54036 files, 42027 regular extents (42028 refs), 27150 inline.
Type       Perc     Disk Usage   Uncompressed Referenced  
Data        82%      5.3G         6.4G         6.4G       
none       100%      4.3G         4.3G         4.3G       
zlib        37%      427M         1.1G         1.1G       
lzo         56%      588M         1.0G         1.0G

Question 2

I am not able to answer on a file by file basis, and @catlover2 gave the answer for a filesystem. But you should differentiate between the block size on disk, and the size in the (virtual) filesystem, ls and du can not go beyond filesystem, so they give no informations on how many disk blocks are used, and @jiliagre --apparent-size is here useless.

To better illustrate this question, I have made a test with a single 23G file btrfs filesystem; first uncompressed, then lzo compressed. The example file is a virtual machine image and as only a compression level of 0.5. It shows that only df, and btrfs filesystem df can show the compression.

$   lvcreate vg0 test_btrfs -L 30G
Logical volume "test_btrfs" created
$ mkfs.btrfs /dev/vg0/test_btrfs
...
fs created label (null) on /dev/vg0/test_btrfs
    nodesize 16384 leafsize 16384 sectorsize 4096 size 30.00GiB
$ mount /dev/vg0/test_btrfs /tmp/test_btrfs
$ btrfs filesystem df /tmp/test_btrfs
Data, single: total=8.00MiB, used=256.00KiB
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, DUP: total=1.00GiB, used=112.00KiB
Metadata, single: total=8.00MiB, used=0.00
$ cp bigfile /tmp/test_btrfs
$ btrfs filesystem df /tmp/test_btrfs
Data, single: total=24.01GiB, used=22.70GiB
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, DUP: total=1.00GiB, used=23.64MiB
Metadata, single: total=8.00MiB, used=0.00
$ btrfs filesystem df /tmp/test_btrfs
... unchanged!
$ cd /tmp/test_btrfs/
$ ls -l bigfile
-rw------- 1 root root 24367940096 May  4 15:03 bigfile
$ du -B1 --apparent-size bigfile
24367940096 bigfile
$ du -B1 bigfile
24367943680 bigfile
$ btrfs filesystem defragment -c bigfile
$ ls -l bigfile
-rw------- 1 root root 24367940096 May  4 15:03 bigfile
$ du -B1 --apparent-size bigfile
24367940096 bigfile
$ du -B1 bigfile
24367943680 bigfile
$ btrfs filesystem df /tmp/test_btrfs
Data, single: total=24.01GiB, used=12.90GiB
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, DUP: total=1.00GiB, used=39.19MiB
Metadata, single: total=8.00MiB, used=0.00
$ df -BG /tmp/test_btrfs
Filesystem                 1G-blocks  Used Available Use% Mounted on
/dev/mapper/vg0-test_btrfs       30G   13G       16G  47% /tmp/test_btrfs

The question of @gandalf3 is still unanswered, and may be we need to wait for development of btrfs (or to help to develop it!) to get a proper underlying disks block du for a peculiar file. It would be very useful, I find very frustrating when I mount a btrfs fs with compression (without force) not knowing if my files are compressed or not and at which level.

Question 3

In Ubuntu-18

apt install btrfs-compsize
compsize /mnt/btrfs-partition

Question 4

The on-disk size of a file, regardless of the file system type, is given¹ by the du command, eg:

$ du -h *
732K    file
512 file1
4.0M    file2
$ du -B1 *
749568  file
512 file1
4091904 file2

The on-disk size is equal to the size of the file plus the size of its metadata, rounded to the file system block size. Non compressed files will usually have a slightly larger on disk size than their actual (bytes count) size.

As already stated, the uncompressed size is shown by ls -l. It can also be reported by du with the --apparent-size option;

$ du --apparent-size -h *
826K    file
64M file1
17M file2
$ du --apparent-size -B 1  *
845708  file
67108864    file1
16784836    file2

Note that -B1 and --apparent-size are GNU specific du extensions.

¹ It seems btrfs doesn't follow this rule. If this is really/still true, my understanding is that should be considered a bug or at least, a POSIX non conformance.

Question 5

I was trying to answer this question too, and here's what I've found:du -s and df produce different numbers. So I did some testing:

I've put a test directory in /home with size about 3TB. It is a partial copy of whole /home directory with typical mix of documents, text files, images and programs
I compressed this directory using .tar.gz which resulted in file of size

# du -s ./test.tar.gz 1672083116 ./test.tar.gz

With this file existing in the file system I did this:

# du -s /home 11017624664 /home

# du --apparent-size -s /home 11010709168 /home

# df /home Filesystem 1K-blocks Used Available Use% Mounted on /dev/md2 31230406656 9128594488 22095200200 30% /home

Which means that we have ((11017624664/(1024**2))/(9128594488/(1024**2))-1)*100 = 20% compression ratio

then I deleted this file and I got this:

# du -s /home 9348284812 /home

# du --apparent-size -s /home 9340957158 /home

# df /home Filesystem 1K-blocks Used Available Use% Mounted on /dev/md2 31230406656 7455549036 23764949364 24% /home

Yielding a compression ratio of 25%. Also from this information I concluded that the test.tar.gz file with real size 1592 G occupied on the disk 1595 G. Also I noted that using --apparent-size flag produces an insignificant difference, probably due to blocksize rounding.

Side note, my fstab line for mounting this partition is:

UUID=be6...07fe /home btrfs defaults,compress=zlib 0 2

Summary:

To check compression ratio on whole partition use these two commands:

du -s /home df /home

Then divide the outputs. I guess that my 25% compression ratio is a typical result to expect from a zlib compressor.

Question 6

You can create a Btrfs filesystem in a file, mount it, copy the files there and run df:

$ dd if=/dev/zero of=btrfs.data size=1M count=1K
$ mkdir btrfs
$ mount btrfs.data btrfs -o compress
... copy the files to ./btrfs
$ sync
$ cd btrfs
$ btrfs filesystem df .

Example of a single file compressed from 17MiB to 5MiB:

$ cd btrfs
$ ls -l
-rwx------ 1 atom atom 17812968 Oct 27  2015 commands.bin
$ btrfs filesystem df .
Data, single: total=1.01GiB, used=5.08MiB
System, DUP: total=8.00MiB, used=16.00KiB
Metadata, DUP: total=1.00GiB, used=112.00KiB
GlobalReserve, single: total=16.00MiB, used=0.00B

Question 7

Run btrfs filesystem df /mountpoint.

Example output:

Data: total=2.01GB, used=1.03GB
System, DUP: total=8.00MB, used=4.00KB
System: total=4.00MB, used=0.00
Metadata, DUP: total=1.00GB, used=2.52MB
Metadata: total=8.00MB, used=0.00

The key line starts with Data:; used= is the compressed size, and total= is the total size as if on an uncompressed filesystem. I created a test filesystem, mounted it with the compress_force=zlib option, and copied 1GB of zeroes to a file on the filesystem; at that point the Data: line was Data: total=1.01GB, used=32.53MB (zeroes are quite compressable!). Then I re-mounted the filesystem with compression disabled, copied another GB of zeroes to it, and at that point the Data: line read Data: total=2.01GB, used=1.03GB.

As nemequ mentioned above, ls -l, on the contrary, shows the uncompressed size.