How to detect unhandled exception due to disk full?

Question

My application ran into a disk full error and somehow, as a result of the disk being full, an unhandled exception was thrown resulting in the set_terminate() handler being invoked.

Normally, I would get some kind of stack trace in my log file so I could see what went wrong, however, in this case, because the disk was full, no stack trace was recorded and it is not clear that the program terminated due to lack of disk space.

Reading what I can from the last things written to the disk, it appears std::clog was being written to, which has been set up to go to disk (the one which got full).

I'm wondering if using operator<< to write to clog could result in an exception being thrown and if so what exception might have been thrown?

Additionally, I'm interested in ideas of how to improve my application such that if this condition recurs in the future, I might update my application to leave behind a better trail of what exactly went wrong so that I can know the disk was full and not some other shortcoming of the application.

However, the key issue is detection of the failure, without that, ideas of how to mitigate are of no use.

Answer 1

In Linux, you can use filenames [in a special directory?] to create a trail of where you were - as files only use up "i-node space", which there is usually plenty of.

Another option is to create a large(ish) file as an "emergency log store" - if disk is full, then open your emergency log-store, and write to that file. Make it a few megabytes, and no one will notice on a modern disk, but it gives you plenty of space to dump the context of where you were.

Answer 2

I don't know the specifics of what would happen in the code per se, but I would like to address the issue of how to handle this kind of exception.

In essence this kind of issue where more logging would be useful. However you have to consider if the the logging mechanism is the issue here. You would need to have an alternative logging/reporting system that doesn't rely on the disk.

You could keep adding layers of redundancy, but in my opinion a primary that fails in exceptional circumstances, coupled with a backup that fails in even more exception circumstances is good enough for most apps. If the data resiliency was paramount, then of course you'd be monitoring your resources and mitigating (issue operator warning or what ever you chose as your fall-back mechanism - e.g. backup spool space etc) as the application processed.

In general the cost of all ways up compared to nearly always up follows the 80/20 rule in costs and time in development too.