Disk usage of a directory in Python

Question

I have some bash code which moves files and directory to /tmp/rmf rather than deleting them, for safety purposes.

I am migrating the code to Python to add some functionality. One of the added features is checking the available size on /tmp and asserting that the moved directory can fit in /tmp.

Checking for available space is done using os.statvfs, but how can I measure the disk usage of the moved directory?

I could either call du using subprocess, or recursively iterate over the directory tree and sum the sizes of each file. Which approach would be better?

Answer 1

This should do the trick:

import os
path = 'THE PATH OF THE DIRECTORY YOU WANT TO FETCH'
os.statvfs(path)

Answer 2

I think you might want to reconsider your strategy. Two reasons:

1. Checking if you can move a file, asserting you can move a file, and then moving a file provides a built-in race-condition to the operation. A big file gets created in /tmp/ after you've asserted but before you've moved your file.. Doh.

2. Moving the file across filesystems will result in a huge amount of overhead. This is why on OSX each volume has their own 'Trash' directory. Instead of moving the blocks that compose the file, you just create a new inode that points to the existing data.

I'd consider how long the file needs to be available and the visibility to consumers of the files. If it's all automated stuff happening on the backend - renaming a file to 'hide' it from computer and human consumers is easy enough in most cases and has the added benefit of being an atomic operation)

Occasionally scan the filesystem for 'old' files to cull and rm them after some grace period. No drama. Also makes restoring files a lot easier since it's just a rename to restore.