Delay during disk writes when logging data

Question

I'm currently running raspbian on a raspberry pi, within which I'm running an application that is sampling raw data at 400Hz from a sensor and logging it to a file. My problem is that the program hangs for around 500ms to 1s every 10-15 seconds, due to some internal buffer writing to the SD card.

I've tried setting the buffer to 0 using the following code, but this doesn't seem to have made any difference, and neither during using flush()

std::fstream Log;
Log.rdbuf()->pubsetbuf(0, 0);

I'm assuming this problem is being caused by something in the kernel as opposed to my application, but I have no idea where to start looking. An explanation of where my problem most likely lies would be greatly appreciated.

Thanks

Matt

Answer 1

Check the linux pdflush tunables: http://www.westnet.com/~gsmith/content/linux-pdflush.htm

The first thing I would try is making pages expire sooner, e.g. set /proc/sys/vm/dirty_expire_centiseconds to 500. If that does not work, decrease /proc/sys/vm/dirty_writeback_centisecs and dirty_expire_centiseconds along with it.

Note that in general, this may decrease the overall performance of your system - the aggressive caching is there for a reason. In your case though, this might help by making the writes more regular.